Skip to content
This repository was archived by the owner on Jun 3, 2025. It is now read-only.

Conversation

@mgoin
Copy link
Member

@mgoin mgoin commented Mar 30, 2022

Adds a simple to run script that provides a simplified table to view the results of analyze_model(), so it is quick to view the performance profile of a model.

deepsparse.analyze zoo:nlp/question_answering/bert-base/pytorch/huggingface/squad/pruned_quant-aggressive_95 -i "[1,128]" -q
Analyzing model: zoo:nlp/question_answering/bert-base/pytorch/huggingface/squad/pruned_quant-aggressive_95
DeepSparse Engine, Copyright 2021-present / Neuralmagic, Inc. version: 0.12.0 (514d3953) (optimized) (system=avx512, binary=avx512)
== Layer Breakdown ==
Name                           | Summed Time | Percent Taken
Naive Subgraph 0               |    0.181    | 3.80%
elementwise                    |    0.232    | 4.88%
reduce                         |    0.573    | 12.04%
shuffle                        |    0.785    | 16.49%
ks_gemm                        |    1.975    | 41.50%
  kernel=[768, 768, 1, 1, 1]   |    0.703    | 14.76%
  kernel=[768, 3072, 1, 1, 1]  |    1.272    | 26.74%
elementwise_sub_2              |    0.009    | 0.18%
gemm                           |    0.283    | 5.95%
  kernel=[768, 2, 1, 1, 1]     |    0.012    | 0.26%
softmax                        |    0.108    | 2.28%
elementwise_plus_2             |    0.425    | 8.94%
Naive Subgraph 1               |    0.187    | 3.93%
== Summed Total Time: 4.7583 ms
== Items per second: 210.1591

@mgoin mgoin marked this pull request as ready for review April 1, 2022 14:55
@dhuangnm
Copy link
Member

dhuangnm commented Apr 1, 2022

Thanks Michael.

@dbarbuzzi can you please add some tests for this utility after the PR lands? Thanks.

@mgoin mgoin merged commit bb475d9 into main Apr 4, 2022
@mgoin mgoin deleted the analyze-model branch April 4, 2022 17:40
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants