Skip to content
This repository was archived by the owner on Jun 3, 2025. It is now read-only.

deepsparse.Pipeline - generic pipeline, deepsparse.server support, NLP,IC,OD pipelines #317

Merged
merged 14 commits into from
Apr 29, 2022
Merged

deepsparse.Pipeline - generic pipeline, deepsparse.server support, NLP,IC,OD pipelines #317

merged 14 commits into from
Apr 29, 2022

Conversation

@bfineran
Copy link
Contributor

@bfineran bfineran commented Apr 13, 2022 •
edited
Loading

co-authored by @rahul-tuli

The goal of this feature is to create a generic Pipeline class to abstract the general flow of wrapping Engine objects with pre/post processing and standardizing inputs/outputs with Pydantic models. These Pipelines should be easily dropped into deepsparse scripts for serving, inference, benchmarking, etc

instantiate a Pipeline using Pipeline.create

from deepsparse import Pipeline

qa_pipeline = Pipeline.create("qa")
qa_pipeline(question="where do i live", context="i am an engineer and i live in america")

TODO:

@bfineran bfineran self-assigned this Apr 13, 2022
@bfineran bfineran changed the title [WIP] generic deepsparse.Pipeline class [WIP] deepsparse.Pipeline - generic pipeline, deepsparse.server support, transformers refactor Apr 13, 2022
This was referenced Apr 13, 2022
Merged
Merged
Merged
@bfineran
Pipeline base class implementation (#315)
34fbd20 
* Pipeline base class implementation

* constructor default values

* __call__ inputs/outputs parsing + validation

* documentation

* pipeline 'alias' argument

* review fixes
@bfineran bfineran mentioned this pull request Apr 14, 2022
Merged
5 tasks
@bfineran bfineran added the 0.13 release label for PRs targeting release/0.13 label Apr 19, 2022
rahul-tuli
rahul-tuli reviewed Apr 19, 2022
View reviewed changes
rahul-tuli
rahul-tuli reviewed Apr 19, 2022
View reviewed changes
dbogunowicz
dbogunowicz reviewed Apr 20, 2022
View reviewed changes
Copy link
Contributor

@dbogunowicz dbogunowicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat.

Silly question, but I think I have missed some context while on my short break: this object would be the template Pipeline for any future ML inference tasks derived from our SparseML pipelines?
i.e. we would use this object to build scripts to run deepsparse inference for IC, transformers, Unet, etc... is that right?

dbogunowicz
dbogunowicz reviewed Apr 20, 2022
View reviewed changes
bfineran
bfineran commented Apr 20, 2022
View reviewed changes
bfineran and others added 6 commits April 29, 2022 14:19
@bfineran
[feature/Pipeline] PipelineConfig (#318)
890fcb9 
* PipelineConfig pydantic model + Pipeline.from_config

* Pipeline.to_config() function
@bfineran
[feature/Pipeline] refactor deepsparse.server to use deepsparse.Pipel…
1438ae9 
…ine (#319)

* PipelineConfig pydantic model + Pipeline.from_config

* Pipeline.to_config() function

* refactor deepsparse.server to use deepsparse.Pipeline

* review nit fix

remove files for separate feature
@rahul-tuli @bfineran
Image Classification Pipeline Integration (#322)
b7cb7a6 
* Create a command line installable for image classification pipeline

* Intermediate Commit

* Image Classification pipeline implementation

* Remove faulty entry point

* Apply suggestions from @bogunowicz


* Changed function  name from `_infer_input_shape` to `_infer_image_shape`

* Add validation script for Image Classification pipeline (#328)

* Add Validation Script for Image Classification Models

* Update pipelines and corresponding schemas to work with numpy arrays

* Bugfix if prediction to be converted to int if it's a string

* Update docstring

* Update src/deepsparse/image_classification/validation_script.py

* [feature/Pipeline] fixes for ic-pipelines implementation (#336)

* fixes for ic-pipelines implementation

* sparsezoo support

Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com>
@bfineran @dbogunowicz
Update src/deepsparse/pipeline.py
8d9653c 
Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com>
@bfineran
Merge branch 'main' into pipeline-feature-main
c06312b
quality
0408098
@bfineran bfineran mentioned this pull request Apr 29, 2022
Merged
@bfineran
[feature/Pipeline] deepsparse.Pipeline implementations for transforme…
08f221d 
…rs (#320)

* add parsing layer for deepsparse.Pipeline for implementation flexibility

* initial deepsparse.transformers.pipelines implementation base class + text_classification

* make tokenizer and config attributes 'public', but not properties

* decorator fix

* use kwargs for transforemrs pipeline parent class args

* token classification impl

* token classification output schema parsing

* quality

* question answering pipeline impl

* fixes for pipline impls - bs1 santity check inferences working

* [feature/Pipeline] deprecate and migrate existing transformers pipelines (#335)

* remove old pipeline pathway and files, add API for deprecated pathway

* migrate eval_downstream

* update readme

* server pipeline input fix

* hf license attribution
@bfineran bfineran force-pushed the pipeline-feature-main branch from ed145a1 to 08f221d Compare April 29, 2022 20:32
@bfineran bfineran mentioned this pull request Apr 29, 2022
Merged
@bfineran bfineran changed the title [WIP] deepsparse.Pipeline - generic pipeline, deepsparse.server support, transformers refactor [WIP] deepsparse.Pipeline - generic pipeline, deepsparse.server support, NLP,IC,OD pipelines Apr 29, 2022
rahul-tuli and others added 3 commits April 29, 2022 17:51
@rahul-tuli
YOLO pipeline integration for deepsparse (#327)
47e0074 
* Added YOLO pipeline
Add an installable for yolo integration
Added a task for YOLO

To install run:
* `pip install --editable "./[yolo]"`

* Changed function  name from `_infer_input_shape` to `_infer_image_shape`

* Update docstring

* Comments from @bogunowicz
* Moved COCO classes to a file

* Adds support to annotate images using YOLO (#332)

* Adds support to annotate images using YOLO

* Makes `YOLOOutput` iterable
* Returns a named tuple of image outputs when `next` is called on `YOLOOutput`
* Adds an annotate function to yolo utils

* Adds an annotation script, testing + minor fixes remain

* Intermediate-commit

* Intermediate WIP

* Working State with required bugfixes

* style fixes

Co-authored-by: Benjamin <ben@neuralmagic.com>

Co-authored-by: Benjamin <ben@neuralmagic.com>
@bfineran
[feature/Pipeline] rename input/output _model to _schema (#340)
9e45921 
* rename input/output _model to _schema

* refactor yolo pipeline
@bfineran
default model support for Pipeline.register (#339)
2fadc90 
* default model support for Pipeline.register

* update default stubs for transformers and IC

* yolo default model
@bfineran bfineran changed the title [WIP] deepsparse.Pipeline - generic pipeline, deepsparse.server support, NLP,IC,OD pipelines deepsparse.Pipeline - generic pipeline, deepsparse.server support, NLP,IC,OD pipelines Apr 29, 2022
@bfineran bfineran marked this pull request as ready for review April 29, 2022 22:00
@bfineran bfineran requested review from a team, KSGulin, natuan and rahul-tuli and removed request for a team April 29, 2022 22:03
markurtz
markurtz previously approved these changes Apr 29, 2022
View reviewed changes
markurtz
markurtz previously approved these changes Apr 29, 2022
View reviewed changes
KSGulin
KSGulin previously approved these changes Apr 29, 2022
View reviewed changes
Copy link
Contributor

@KSGulin KSGulin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved pending fix for pipeline.output_model discussed previously

@bfineran bfineran dismissed stale reviews from KSGulin and markurtz via 1566497 April 29, 2022 22:53
@bfineran bfineran merged commit 20a4cb3 into main Apr 29, 2022
@bfineran bfineran deleted the pipeline-feature-main branch April 29, 2022 22:56
markurtz added a commit that referenced this pull request May 2, 2022
@markurtz @InquestGeronimo
@mgoin @dhuangnm @bfineran @jeanniefinks @govindr-nm @rahul-tuli @KSGulin
[cherry-pick] Release 0.12.1 cherry pick (#343)
ddf0ed6 
* SparseServer.UI example [WIP] (#308)

* add megasparse dir

* edited readme to support DS integration

* small edit to docstring

* edited settings.py module so only 2 models are loaded.

* added more context to the readme about adding more models.

* fixed image

* added different  host  to streamlit client as default

* quality check edits

* quality check commit

* passing copyright quality check

* content edits

* rename dir to sparseserver-ui

* added new config file for quick start

* edited multipipelineclient in settings.py

* changed name of config file

* edited model stubs

* edited readme

* added dependency pins

* changed server pin

* edited model choice logic

* altered front-end features

* style update

* renamed samples file

* added variant descriptions

* style changes

* edited samples.py module

* style changes

* added new pic

* edited README

* updated readme cmds

* SparseServer edit (#314)

* edit number of models

* edit settings.py

* edit readme

* Update label mapping for deepsparse.transformers.eval_downstream (#323)

* Update label mapping for deepsparse.transformers.eval_downstream

* Fix MNLI as well

* bump up main to 0.13.0 (#313)

Co-authored-by: dhuang <dhuang@dhuangs-MacBook-Pro.local>

* AWS Sagemaker example integration (#305)

* AWS Sagemaker example integration

* documentation, sample config, dockerfile fixes

* fix ecr repo name

* readme code changes from testing

* Update huggingface-transformers/README.md with new models (#329)

* Update README.md (#330)

* Update README.md

various grammatical edits
additional edits for section headline consistency

* Topology file for HB120rs_v3 (#334)

* Topology file for HB120rs_v3

Specifies core-per-CCX grouping for HB120rs_v3 VM's, used by multi-process script.

* Update README.md to reference Azure topo file

* Move all benchmarking within deepsparse/benchmark/ (#333)

* Move all benchmarking within deepsparse/benchmark/

* Update benchmark_model

* Expose results at benchmark base

* isort

* Skip flake8

* server integration check bug fix (#331)

* server integration check bug fix

need to verify integration is set before calling `integration.lower()`

* respond to review - click choice

* add default integration val to server config schema (#337)

* deepsparse.Pipeline - generic pipeline, deepsparse.server support, NLP,IC,OD pipelines (#317)

* base commit - make pydantic a general req

* Pipeline base class implementation (#315)

* Pipeline base class implementation

* constructor default values

* __call__ inputs/outputs parsing + validation

* documentation

* pipeline 'alias' argument

* review fixes

* [feature/Pipeline] PipelineConfig (#318)

* PipelineConfig pydantic model + Pipeline.from_config

* Pipeline.to_config() function

* [feature/Pipeline] refactor deepsparse.server to use deepsparse.Pipeline (#319)

* PipelineConfig pydantic model + Pipeline.from_config

* Pipeline.to_config() function

* refactor deepsparse.server to use deepsparse.Pipeline

* review nit fix

remove files for separate feature

* Image Classification Pipeline Integration (#322)

* Create a command line installable for image classification pipeline

* Intermediate Commit

* Image Classification pipeline implementation

* Remove faulty entry point

* Apply suggestions from @bogunowicz


* Changed function  name from `_infer_input_shape` to `_infer_image_shape`

* Add validation script for Image Classification pipeline (#328)

* Add Validation Script for Image Classification Models

* Update pipelines and corresponding schemas to work with numpy arrays

* Bugfix if prediction to be converted to int if it's a string

* Update docstring

* Update src/deepsparse/image_classification/validation_script.py

* [feature/Pipeline] fixes for ic-pipelines implementation (#336)

* fixes for ic-pipelines implementation

* sparsezoo support

Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com>

* Update src/deepsparse/pipeline.py


* quality

* [feature/Pipeline] deepsparse.Pipeline implementations for transformers (#320)

* add parsing layer for deepsparse.Pipeline for implementation flexibility

* initial deepsparse.transformers.pipelines implementation base class + text_classification

* make tokenizer and config attributes 'public', but not properties

* decorator fix

* use kwargs for transforemrs pipeline parent class args

* token classification impl

* token classification output schema parsing

* quality

* question answering pipeline impl

* fixes for pipline impls - bs1 santity check inferences working

* [feature/Pipeline] deprecate and migrate existing transformers pipelines (#335)

* remove old pipeline pathway and files, add API for deprecated pathway

* migrate eval_downstream

* update readme

* server pipeline input fix

* hf license attribution

* `YOLO` pipeline integration for deepsparse (#327)

* Added YOLO pipeline
Add an installable for yolo integration
Added a task for YOLO

To install run:
* `pip install --editable "./[yolo]"`

* Changed function  name from `_infer_input_shape` to `_infer_image_shape`

* Update docstring

* Comments from @bogunowicz
* Moved COCO classes to a file

* Adds support to annotate images using YOLO (#332)

* Adds support to annotate images using YOLO

* Makes `YOLOOutput` iterable
* Returns a named tuple of image outputs when `next` is called on `YOLOOutput`
* Adds an annotate function to yolo utils

* Adds an annotation script, testing + minor fixes remain

* Intermediate-commit

* Intermediate WIP

* Working State with required bugfixes

* style fixes

Co-authored-by: Benjamin <ben@neuralmagic.com>

Co-authored-by: Benjamin <ben@neuralmagic.com>

* [feature/Pipeline] rename input/output _model to _schema (#340)

* rename input/output _model to _schema

* refactor yolo pipeline

* default model support for Pipeline.register (#339)

* default model support for Pipeline.register

* update default stubs for transformers and IC

* yolo default model

* minor fixes

* model->schema for server

Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>

* Remove: startlette dep (#338)

* Update src/deepsparse/version.py

Co-authored-by: Ricky Costa <79061523+InquestGeronimo@users.noreply.github.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
Co-authored-by: dhuangnm <74931910+dhuangnm@users.noreply.github.com>
Co-authored-by: dhuang <dhuang@dhuangs-MacBook-Pro.local>
Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com>
Co-authored-by: Jeannie Finks <74554921+jeanniefinks@users.noreply.github.com>
Co-authored-by: Govind Ramnarayan <77341216+govindr-nm@users.noreply.github.com>
Co-authored-by: Rahul Tuli <rahul@neuralmagic.com>
Co-authored-by: Konstantin Gulin <66528950+KSGulin@users.noreply.github.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@natuan natuan Awaiting requested review from natuan natuan was automatically assigned from neuralmagic/machine-learning

@rahul-tuli rahul-tuli Awaiting requested review from rahul-tuli rahul-tuli was automatically assigned from neuralmagic/machine-learning

3 more reviewers

@dbogunowicz dbogunowicz dbogunowicz left review comments

@markurtz markurtz markurtz approved these changes

@KSGulin KSGulin KSGulin approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@bfineran bfineran

Labels

0.13 release label for PRs targeting release/0.13

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@bfineran @markurtz @rahul-tuli @KSGulin @dbogunowicz