Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add vgg16 & resnet50 to supported models #32

Merged
merged 1 commit into from
Apr 25, 2018

Conversation

kuke
Copy link

@kuke kuke commented Apr 24, 2018

VGG16 Validation. The size of two intermediate fully-connected layers is set to 512, for the case which is set to 4096, the inference of ONNX model gets too slow.

python validate.py --fluid_model image_classification_vgg_512.inference.model/ --onnx_model image_classification_vgg_512.onnx
-----------  Configuration Arguments -----------
a: 0.0
b: 1.0
batch_size: 10
expected_decimal: 5
fluid_model: image_classification_vgg_512.inference.model/
onnx_model: image_classification_vgg_512.onnx
------------------------------------------------
Inference results for fluid model:
[array([[0.10089886, 0.09871482, 0.10326847, 0.10127534, 0.09726173,
        0.10202399, 0.09696591, 0.09893821, 0.10302259, 0.09763013],
       [0.10091323, 0.09872197, 0.10325751, 0.10122918, 0.09734982,
        0.10196733, 0.09698458, 0.09890091, 0.10304806, 0.09762745],
       [0.10088442, 0.09874254, 0.10324674, 0.10130584, 0.09730507,
        0.10196551, 0.09692121, 0.09891873, 0.10305765, 0.09765223],
       [0.1009225 , 0.09880089, 0.10329139, 0.10122177, 0.09734076,
        0.1019473 , 0.09699273, 0.09891348, 0.10295378, 0.09761531],
       [0.10091846, 0.09871273, 0.10328399, 0.10130198, 0.09726519,
        0.10197122, 0.09687649, 0.09894703, 0.1030934 , 0.09762951],
       [0.10092277, 0.09875839, 0.1032907 , 0.10128019, 0.0973415 ,
        0.10200986, 0.09692244, 0.09887472, 0.10299625, 0.09760312],
       [0.10092793, 0.09872951, 0.10329424, 0.1012623 , 0.09729295,
        0.10197538, 0.09699677, 0.0988729 , 0.10303052, 0.09761748],
       [0.10088892, 0.09871186, 0.10328761, 0.10128257, 0.09727918,
        0.10195947, 0.09692416, 0.09894343, 0.10303068, 0.09769206],
       [0.10086466, 0.09880146, 0.10325968, 0.10126632, 0.09731811,
        0.10198917, 0.09694263, 0.09888101, 0.10306235, 0.09761455],
       [0.10084924, 0.09872525, 0.10329291, 0.10123692, 0.09729046,
        0.10201162, 0.096938  , 0.09895328, 0.10304983, 0.09765254]],
      dtype=float32)]


Inference results for ONNX model:
Outputs(_0=array([[0.10089886, 0.09871481, 0.10326847, 0.10127534, 0.09726173,
        0.10202398, 0.09696591, 0.09893821, 0.10302258, 0.09763013],
       [0.10091322, 0.09872196, 0.10325751, 0.10122917, 0.09734982,
        0.10196733, 0.09698458, 0.0989009 , 0.10304805, 0.09762745],
       [0.10088442, 0.09874254, 0.10324674, 0.10130584, 0.09730507,
        0.10196552, 0.09692121, 0.09891874, 0.10305766, 0.09765223],
       [0.1009225 , 0.09880089, 0.10329139, 0.10122177, 0.09734076,
        0.1019473 , 0.09699273, 0.09891348, 0.10295378, 0.09761531],
       [0.10091845, 0.09871273, 0.10328399, 0.10130197, 0.09726518,
        0.1019712 , 0.09687648, 0.09894701, 0.10309339, 0.09762949],
       [0.10092278, 0.0987584 , 0.10329071, 0.1012802 , 0.09734151,
        0.10200986, 0.09692245, 0.09887473, 0.10299627, 0.09760312],
       [0.10092794, 0.09872951, 0.10329424, 0.1012623 , 0.09729295,
        0.10197538, 0.09699677, 0.0988729 , 0.10303052, 0.09761748],
       [0.10088893, 0.09871187, 0.10328761, 0.10128257, 0.09727918,
        0.10195947, 0.09692415, 0.09894343, 0.10303069, 0.09769207],
       [0.10086467, 0.09880148, 0.10325969, 0.10126634, 0.09731812,
        0.10198919, 0.09694266, 0.09888103, 0.10306237, 0.09761456],
       [0.10084924, 0.09872525, 0.10329291, 0.10123691, 0.09729046,
        0.10201162, 0.09693799, 0.09895328, 0.10304983, 0.09765254]],
      dtype=float32))


The exported model achieves 5-decimal precision.

@kuke
Copy link
Author

kuke commented Apr 24, 2018

ResNet50

python validate.py --fluid_model extras/image_classification_resnet.inference.model/ --onnx_model image_classification_resnet.onnx
-----------  Configuration Arguments -----------
a: 0.0
b: 1.0
batch_size: 10
expected_decimal: 5
fluid_model: extras/image_classification_resnet.inference.model/
onnx_model: image_classification_resnet.onnx
------------------------------------------------
Inference results for fluid model:
[array([[1.6038109e-28, 1.0000000e+00, 1.6038109e-28, 1.6038109e-28,
        1.6038109e-28, 1.6038109e-28, 1.6038109e-28, 1.6038109e-28,
        1.6038109e-28, 1.6038109e-28],
       [1.6038109e-28, 1.0000000e+00, 1.6038109e-28, 1.6038109e-28,
        1.6038109e-28, 1.6038109e-28, 1.6038109e-28, 1.6038109e-28,
        1.6038109e-28, 1.6038109e-28],
       [1.6038109e-28, 1.0000000e+00, 1.6038109e-28, 1.6038109e-28,
        1.6038109e-28, 1.6038109e-28, 1.6038109e-28, 1.6038109e-28,
        1.6038109e-28, 1.6038109e-28],
       [1.6038109e-28, 1.0000000e+00, 1.6038109e-28, 1.6038109e-28,
        1.6038109e-28, 1.6038109e-28, 1.6038109e-28, 1.6038109e-28,
        1.6038109e-28, 1.6038109e-28],
       [1.6038109e-28, 1.0000000e+00, 1.6038109e-28, 1.6038109e-28,
        1.6038109e-28, 1.6038109e-28, 1.6038109e-28, 1.6038109e-28,
        1.6038109e-28, 1.6038109e-28],
       [1.6038109e-28, 1.0000000e+00, 1.6038109e-28, 1.6038109e-28,
        1.6038109e-28, 1.6038109e-28, 1.6038109e-28, 1.6038109e-28,
        1.6038109e-28, 1.6038109e-28],
       [1.6038109e-28, 1.0000000e+00, 1.6038109e-28, 1.6038109e-28,
        1.6038109e-28, 1.6038109e-28, 1.6038109e-28, 1.6038109e-28,
        1.6038109e-28, 1.6038109e-28],
       [1.6038109e-28, 1.0000000e+00, 1.6038109e-28, 1.6038109e-28,
        1.6038109e-28, 1.6038109e-28, 1.6038109e-28, 1.6038109e-28,
        1.6038109e-28, 1.6038109e-28],
       [1.6038109e-28, 1.0000000e+00, 1.6038109e-28, 1.6038109e-28,
        1.6038109e-28, 1.6038109e-28, 1.6038109e-28, 1.6038109e-28,
        1.6038109e-28, 1.6038109e-28],
       [1.6038109e-28, 1.0000000e+00, 1.6038109e-28, 1.6038109e-28,
        1.6038109e-28, 1.6038109e-28, 1.6038109e-28, 1.6038109e-28,
        1.6038109e-28, 1.6038109e-28]], dtype=float32)]


Inference results for ONNX model:
Outputs(_0=array([[0., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
       [0., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
       [0., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
       [0., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
       [0., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
       [0., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
       [0., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
       [0., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
       [0., 1., 0., 0., 0., 0., 0., 0., 0., 0.],
       [0., 1., 0., 0., 0., 0., 0., 0., 0., 0.]], dtype=float32))


The exported model achieves 5-decimal precision.

Copy link
Collaborator

@varunarora varunarora left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting! why is it slow?

@kuke
Copy link
Author

kuke commented Apr 25, 2018

@varunarora I haven't had a detailed analysis. I think we'd better profile the network and find the bottleneck if we care about computation performance someday.

@kuke kuke merged commit 5441d5f into PaddlePaddle:develop Apr 25, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants