¶ Install prerequisites
sudo apt update -y && sudo apt upgrade -y
sudo apt install -y software-properties-common cmake libopencv-dev python3-dev python3-pip python3-opencv python3-numpy
sudo add-apt-repository ppa:liujianfeng1994/rockchip-multimedia
sudo apt install -y rockchip-multimedia-config
¶ Get NPU demos
git clone https://github.com/rippanda12/rk3588-npu-demos
¶ Python
¶ Install Python RKNN module
cd ~/rk3588-npu-demos/Python
sudo cp ../C++/runtime/librknn_api/aarch64/librknnrt.so /usr/lib
pip3 install ./wheel/rknn_toolkit_lite2-1.3.0-cp310-cp310-linux_aarch64.whl
¶ Try the Python model that detects people with face masks
Find a photo of you and your friends in which you are wearing a face mask.
Open the File Browser and go to Home → rk3588-npu-demos → Python → face_mask → data → img
Delete the face.jpg file and replace it with the group photo.
Then open a terminal and run:
cd ~/rk3588-npu-demos/Python/face_mask
python3 ./face_mask.py
Next, open the face_mask folder and look for out.jpg. You should see everyone with a mask surrounded by a “bounding box” showing what part of the image was detected to contain a face mask.
Examples:
This is what we put as input.
This is the result.
¶ C++
¶ Try the yolov5 model that detects objects
Compile the yolov5 project
cd ~/rk3588-npu-demos/C++/yolov5/
./build.sh
Get a random photo of a city or a busy street and run but replace the example image with your own:
cd ~/rk3588-npu-demos/C++/yolov5/install/yolov5
./yolov5 data/model/yolov5s-640-640.rknn ~/Download/medrudnikterminal.jpg
Heres a sample output of the console:
./yolov5 data/model/yolov5s-640-640.rknn ./medenrudnikterminal.jpg
post process config: box_conf_threshold = 0.50, nms_threshold = 0.60
Read ./medenrudnikterminal.jpg ...
img width = 1200, img height = 630
Loading mode...
sdk version: 1.3.0 (c193be371@2022-05-04T20:16:33) driver version: 0.8.2
model input num: 1, output num: 3
index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
index=0, name=output, n_dims=5, dims=[1, 3, 85, 80], n_elems=1632000, size=1632000, fmt=UNDEFINED, type=INT8, qnt_type=AFFINE, zp=77, scale=0.080445
index=1, name=371, n_dims=5, dims=[1, 3, 85, 40], n_elems=408000, size=408000, fmt=UNDEFINED, type=INT8, qnt_type=AFFINE, zp=56, scale=0.080794
index=2, name=390, n_dims=5, dims=[1, 3, 85, 20], n_elems=102000, size=102000, fmt=UNDEFINED, type=INT8, qnt_type=AFFINE, zp=69, scale=0.081305
model is NHWC input fmt
model input height=640, width=640, channel=3
resize with RGA!
rga_api version 1.6.0_[4] (d041200 build: 2021-12-01 15:26:14 base: )
rk-debug out_fence_fd = 0
once run use 40.766000 ms
loadLabelName ./data/coco_80_labels_list.txt
person @ (764 257 884 600) 0.994352
person @ (710 267 774 384) 0.985245
person @ (866 246 907 344) 0.982702
chair @ (1012 360 1171 517) 0.979730
person @ (907 267 995 405) 0.962752
car @ (751 255 813 281) 0.962012
chair @ (944 324 1038 407) 0.918626
train @ (0 37 757 623) 0.812066
loop count = 10 , average run 39.756800 ms
You should see it output an out.jpg image that has everything it detected.
Examples:
This is what we put as input.
This is the result.
¶ Additional Discussion
Please see the following thread on the Indiedroid-Nova Discord Channel
https://discordapp.com/channels/1072231504295493742/1073693147785338950/1127675359274487898