sparsezoo.deployment_package

[rahul-tuli]

sparsezoo.deployment_package

This utility provides an easy to use python-api and a command-line-interface
to find pre-trained, packaged models across both NLP and CV domains that are
ready for deployment out-of-the-box and are optimized for metrics that are most important to a user like accuracy, throughput, latency,
etc.

python-api

The python-api can be used to quickly fetch a model from SparseZoo, the
model.onnx path printed the end of the following code snippet can be directly
used with the Dockerfile provided with this diff

For deployment instructions refer here

from sparsezoo import deployment_package, Model

deployment_package(
    task="image_classification",
    dataset="imagenette",
    optimizing_metric=["accuracy", "compression"],
)

Output:

{'metrics': {'accuracy': 99.9, 'compression': 18055005},
 'stub': 'zoo:cv/classification/resnet_v1-50/pytorch/sparseml/imagenette/pruned-conservative'}

cli

The sparsezoo.deployment_package cli provides an interface to the python-api
and also prints out instructions to deploy the saved model directory with
Dockerfile present here

(sparsezoo3.9) ~ sparsezoo.deployment_package \
    --dataset imagenette \
    --optimizing_metric compression

Output:

downloading...: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 12.4M/12.4M [00:01<00:00, 12.0MB/s]

    Relevant Stub: zoo:cv/classification/mobilenet_v1-1.0/pytorch/sparseml/imagenette/base-none
    
    Model Metrics: {'compression': 12474452}
    
    Use the dockerfile in /home/ubuntu/projects/sparsezoo/deployment_package/docker/Dockerfile to build deepsparse
    image and run the `deepsparse.server` container.
    
    Run the following command inside `/home/ubuntu/projects/sparsezoo/deployment_package/docker`
    directory (Note: replace <TASK-NAME> with appropriate task):
    
    ```bash
    docker build -t deepsparse_docker . && docker run -it \
    -v /home/ubuntu/.cache/sparsezoo/f780f407-3213-4a79-8d50-a5759822410d/deployment:/home/deployment  deepsparse_docker \
     deepsparse.server --task <TASK-NAME> --model_path /home/deployment
    ```

Usage

Usage: sparsezoo.deployment_package [OPTIONS] [DIRECTORY]

  Utility to fetch a deployment directory for a task based on specified
  optimizing-metric

  Example for using sparsezoo.deployment_package:

       1) `sparsezoo.deployment_package --task image_classification
       --optimizing_metric accuracy

       2) `sparsezoo.deployment_package --task ic
       --optimizing_metric "accuracy, compression"                 --target
       VNNI`

Options:
  --version                       Show the version and exit.
  --task [image_classification|image_classification|image-classification|ic|classification|object_detection|object_detection|object-detection|od|detection|segmentation|segmentation|question_answering|question_answering|question-answering|qa|text_classification|text_classification|text-classification|glue|sentiment_analysis|sentiment_analysis|sentiment-analysis|sentiment|token_classification|token_classification|token-classification|ner|named_entity_recognition|named-entity-recognition]
                                  The task to find model for, must be
                                  specified if `--dataset` not provided
  --dataset [imagenette|imagenet|coco|squad|mnli|qqp|sst2|conll2003]
                                  The public dataset used to train this model,
                                  must be specified if `--task` not provided
  -m, --optimizing-metric TEXT    The criterion to search model for, multiple
                                  metrics can be specified as comma separated
                                  values, supported metrics are ['accuracy',
                                  'f1', 'recall', 'mAP', 'latency',
                                  'throughput', 'compression', 'file_size',
                                  'memory_usage']  [default: accuracy]
  --target [VNNI|DEFAULT]         Deployment target scenario (ie 'VNNI' for
                                  VNNI capable CPUs)  [default: DEFAULT]
  --help                          Show this message and exit.
  

Examples:

• Fetch the smallest Image Classification Model trained on imagenette
  sparsezoo.deployment_package --dataset imagenette --optimizing_metric compression
• Fetch the most accurate Image Classification Model trained on imagenette
  sparsezoo.deployment_package --dataset imagenette --optimizing_metric accuracy
• Fetch the most performant Question Answering model trained on squad
  sparsezoo.deployment_package --task qa --dataset squad --optimizing_metric latency
• Fetch the smallest most performant Question Answering model trained on squad

  sparsezoo.deployment_package --task qa --dataset squad \
       --optimizing_metric "compression, accuracy"

Deployment

Assuming we know the path to the directory containing model.onnx either
with or without configuration files, the following instructions can be used to
deploy a deepsparse container serving the onnx model using the Dockerfile,
exact instructions are also shown with the cli

---

• To see the specific tasks where the Asana app for GitHub is being used, see below:
  • https://app.asana.com/0/0/1203968932439036

[corey-nm]

lgtm!