Here you can find all inference results: Accuracy and Inference Time for all networks in Ai4prod inference Library
INFERENCE TIME
results are FPS on 1000 iterations
A) FPS wit model inference time but WITHOUT preprocessing or post processing execution time
Model | GPU/Backend | CPU Only(no Gpu) | FP32 B=1 | FP16 B=1 | OS |
yolov3-spp-base-608 | 2070 Rtx/ Tensorrt | 0,9 | 42,54 | 125,13 | ubuntu 18.04 |
yolov4-608 | 2070 Rtx/ Tensorrt | 41,66 | 124,96 | ubuntu 18.04 | |
yolov3-spp-base-608 | 2070 Rtx/ Tensorrt | 41,23 | 115,23 | Windows 10 | |
yolov3-spp-base-608 | Xavier NX | 5,18 | 18,86 | Jetpack 4.4 | |
resnet50-base | 2070 Rtx/ Tensorrt | 14 | 320,43 | 667,63 | ubuntu 18.04 |
resnet50-base | 2070 Rtx/ Tensorrt | 14 | 315,83 | 650,19 | Windows 10 |
resnet50-base | 2070/ DirectML | 142,85 | Windows 10 | ||
resnet50-base | Xavier NX | 50,96 | 83,74 | Jetpack 4.4 | |
yolact-resnet50-550 | 2070 Rtx/ Tensorrt | 2,8 | 37,03 | 102,04 | ubuntu 18.04 |
yolact-resnet30-550 | 2070 Rtx/ Tensorrt | 2,7 | 32,25 | 92,96 | Windows 10 |
NOTE:
- CPU Only is executed on Amd ryzen 5 3600.
B) FPS with model inference time with preprocessing and postprocessing
Model | GPU/Backend | CPU Only(no Gpu) | FP32 B=1 | FP16 B=1 | OS |
yolov3-spp-base-608 | 2070 Rtx/ Tensorrt | 34,87 | 77,34 | ubuntu 18.04 | |
yolov4-608 | 2070 Rtx/ Tensorrt | 31,25 | 58,52 | ubuntu 18.04 | |
yolov3-spp-base-608 | 2070 Rtx/ Tensorrt | 34,25 | 75,62 | Windows 10 | |
yolov3-spp-base-608 | Xavier | 4,69 | 14,28 | Jetpack 4.4 | |
resnet50-base | 2070 Rtx/ Tensorrt | 250,76 | 454,72 | ubuntu 18.04 | |
resnet50-base | 2070 Rtx/ Tensorrt | 245.56 | 447,17 | Windows 10 | |
yolact-resnet50-550 | 2070 Rtx/ Tensorrt | 1,68 | 27,02 | 49,73 | ubuntu18.04 |
NOTE:
- This time depend also on Cpu because usually preprocessing and postprocessing are done on cpu
ACCURACY
each network is tested on proper benchmark. For object detection is MAP
PAPER RESULTS:
Model | Dataset(validation) | Metrics | FP32 |
yolov3-spp-base-608 | Coco 2017 | MAP(AP50) | 60.6 |
resnet50-base | Imagenet 2012 | Accuracy | 76%-92% |
yolact-resnet50-550 | Coco 2017 | MAP(AP50) | 28.2 |
OUR RESULTS:
Model | Dataset(validation) | Metrics | Backend | FP32 | FP16 | OS |
yolov3-spp-base-608 | Coco 2017 | MAP(AP50) | Tensorrt | 66.1 | 66.1 | ubuntu 18.04 |
yolov4-608 | Coco 2017 | MAP(AP50) | Tensorrt | 72.3 | 72.3 | ubuntu 18.04 |
yolov3-spp-base-608 | Coco 2017 | MAP(AP50) | Tensorrt | 66.1 | 66.1 | Windows 10 |
yolov3-spp-base-608 | Coco 2017 | MAP(AP50) | Tensorrt | 65.1 | 65.2 | Jetson Xavier |
resnet50-base | Imagenet 2012 | Accuracy | Tensorrt | 75%-92% | 74%-92% | ubuntu 18.04 |
resnet50-base | Imagenet 2012 | Accuracy | Tensorrt | 75%-92% | 74%-92% | Windows 10 |
yolact-resnet50-550 | Coco 2017 | MAP(AP50) | Tensorrt | 42.1 | 41.9 | ubuntu 18.04 |
yolact-resnet50-550 | Coco 2017 | MAP(AP50) | Tensorrt | 42.1 | 41.9 | Windows 10 |
yolact-resnet50-550 | Coco 2017 | MAP(AP50) | Tensorrt | 36.1 | Jetson Xavier |
Note:
- For Accuracy metric value are 1 prediction-5 prediction.
- Fp32 e Fp16 are obtained when neural network is used in TensorRt Mode.
Yolov3-spp-608: Accuracy is taken from this https://pjreddie.com/darknet/yolo/
Yolact–550: Accuracy is taken from the paper