SparseServer.UI example [WIP] #308

InquestGeronimo · 2022-04-05T20:21:37Z

Adding MegaSparse demo to the examples folder. Megasparse is a streamlit app that runs on top of the DeepSparse Server for users to interact with and compare latency and performances of various sparse models on the question answering task.

Pushing for a new style in readme. Currently have 16 of the 18 models commented out so peeps computers don't explode, I discuss more on this in the readme. Looking for code reviews, then content review.

examples/megasparse/README.md

examples/megasparse/client/app.py

markurtz

A few blockers from my side:

Replace megasparse with a descriptive name of what this application is doing so users going through the repo can make sense of the multiple examples that will be in this folder without clicking into everyone -- offering a sample streamlit integration on top of our server so users can test it out.
Additionally would like to explain the need for this on top of the swagger API docs that offer the same functionality and why we wouldn't want to surface that as a test use case. Ultimately trying to figure out the value of this example -- specifically, want to orient all these examples around what the end-user gets out of this that relates to their day to day pipelines they use now. If there's any way we can add auto-generation / ingestion of data from sample datasets?
Need to limit the number of models shown and give the user some context as to why the models are different and what the tradeoffs are between them (accuracy and model size vs performance).
We're pushing up new model names this week along with more domains, would be good to get this migrated to those and I'll keep you updated as those land.
Can we update the color scheme a bit? IMO the sharp blue and black are a bit jarring

mgoin · 2022-04-07T16:32:56Z

@markurtz some thoughts on your points:

If there's any way we can add auto-generation / ingestion of data from sample datasets?

I think this can be added later as we decide to take this specific example further. We can add sample sets of inputs maybe as a dropdown selection to demonstrate the flexibility of the model/task, but these can be handpicked to start.

Need to limit the number of models shown and give the user some context

I already mentioned this and offered a suggestion above in a comment thread if you could reply there to push a solution.

mgoin

looks good to me

examples/sparseserver-ui/client/samples.py

markurtz

LGTM

mgoin

some spaces got removed with the last change

examples/sparseserver-ui/client/samples.py

dbarbuzzi

Was able to run through the sample fine with the one minor change requested.

examples/sparseserver-ui/README.md

* SparseServer.UI example [WIP] (#308) * add megasparse dir * edited readme to support DS integration * small edit to docstring * edited settings.py module so only 2 models are loaded. * added more context to the readme about adding more models. * fixed image * added different host to streamlit client as default * quality check edits * quality check commit * passing copyright quality check * content edits * rename dir to sparseserver-ui * added new config file for quick start * edited multipipelineclient in settings.py * changed name of config file * edited model stubs * edited readme * added dependency pins * changed server pin * edited model choice logic * altered front-end features * style update * renamed samples file * added variant descriptions * style changes * edited samples.py module * style changes * added new pic * edited README * updated readme cmds * SparseServer edit (#314) * edit number of models * edit settings.py * edit readme * Update label mapping for deepsparse.transformers.eval_downstream (#323) * Update label mapping for deepsparse.transformers.eval_downstream * Fix MNLI as well * bump up main to 0.13.0 (#313) Co-authored-by: dhuang <dhuang@dhuangs-MacBook-Pro.local> * AWS Sagemaker example integration (#305) * AWS Sagemaker example integration * documentation, sample config, dockerfile fixes * fix ecr repo name * readme code changes from testing * Update huggingface-transformers/README.md with new models (#329) * Update README.md (#330) * Update README.md various grammatical edits additional edits for section headline consistency * Topology file for HB120rs_v3 (#334) * Topology file for HB120rs_v3 Specifies core-per-CCX grouping for HB120rs_v3 VM's, used by multi-process script. * Update README.md to reference Azure topo file * Move all benchmarking within deepsparse/benchmark/ (#333) * Move all benchmarking within deepsparse/benchmark/ * Update benchmark_model * Expose results at benchmark base * isort * Skip flake8 * server integration check bug fix (#331) * server integration check bug fix need to verify integration is set before calling `integration.lower()` * respond to review - click choice * add default integration val to server config schema (#337) * deepsparse.Pipeline - generic pipeline, deepsparse.server support, NLP,IC,OD pipelines (#317) * base commit - make pydantic a general req * Pipeline base class implementation (#315) * Pipeline base class implementation * constructor default values * __call__ inputs/outputs parsing + validation * documentation * pipeline 'alias' argument * review fixes * [feature/Pipeline] PipelineConfig (#318) * PipelineConfig pydantic model + Pipeline.from_config * Pipeline.to_config() function * [feature/Pipeline] refactor deepsparse.server to use deepsparse.Pipeline (#319) * PipelineConfig pydantic model + Pipeline.from_config * Pipeline.to_config() function * refactor deepsparse.server to use deepsparse.Pipeline * review nit fix remove files for separate feature * Image Classification Pipeline Integration (#322) * Create a command line installable for image classification pipeline * Intermediate Commit * Image Classification pipeline implementation * Remove faulty entry point * Apply suggestions from @bogunowicz * Changed function name from `_infer_input_shape` to `_infer_image_shape` * Add validation script for Image Classification pipeline (#328) * Add Validation Script for Image Classification Models * Update pipelines and corresponding schemas to work with numpy arrays * Bugfix if prediction to be converted to int if it's a string * Update docstring * Update src/deepsparse/image_classification/validation_script.py * [feature/Pipeline] fixes for ic-pipelines implementation (#336) * fixes for ic-pipelines implementation * sparsezoo support Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com> * Update src/deepsparse/pipeline.py * quality * [feature/Pipeline] deepsparse.Pipeline implementations for transformers (#320) * add parsing layer for deepsparse.Pipeline for implementation flexibility * initial deepsparse.transformers.pipelines implementation base class + text_classification * make tokenizer and config attributes 'public', but not properties * decorator fix * use kwargs for transforemrs pipeline parent class args * token classification impl * token classification output schema parsing * quality * question answering pipeline impl * fixes for pipline impls - bs1 santity check inferences working * [feature/Pipeline] deprecate and migrate existing transformers pipelines (#335) * remove old pipeline pathway and files, add API for deprecated pathway * migrate eval_downstream * update readme * server pipeline input fix * hf license attribution * `YOLO` pipeline integration for deepsparse (#327) * Added YOLO pipeline Add an installable for yolo integration Added a task for YOLO To install run: * `pip install --editable "./[yolo]"` * Changed function name from `_infer_input_shape` to `_infer_image_shape` * Update docstring * Comments from @bogunowicz * Moved COCO classes to a file * Adds support to annotate images using YOLO (#332) * Adds support to annotate images using YOLO * Makes `YOLOOutput` iterable * Returns a named tuple of image outputs when `next` is called on `YOLOOutput` * Adds an annotate function to yolo utils * Adds an annotation script, testing + minor fixes remain * Intermediate-commit * Intermediate WIP * Working State with required bugfixes * style fixes Co-authored-by: Benjamin <ben@neuralmagic.com> Co-authored-by: Benjamin <ben@neuralmagic.com> * [feature/Pipeline] rename input/output _model to _schema (#340) * rename input/output _model to _schema * refactor yolo pipeline * default model support for Pipeline.register (#339) * default model support for Pipeline.register * update default stubs for transformers and IC * yolo default model * minor fixes * model->schema for server Co-authored-by: Rahul Tuli <rahul@neuralmagic.com> * Remove: startlette dep (#338) * Update src/deepsparse/version.py Co-authored-by: Ricky Costa <79061523+InquestGeronimo@users.noreply.github.com> Co-authored-by: Michael Goin <michael@neuralmagic.com> Co-authored-by: dhuangnm <74931910+dhuangnm@users.noreply.github.com> Co-authored-by: dhuang <dhuang@dhuangs-MacBook-Pro.local> Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com> Co-authored-by: Jeannie Finks <74554921+jeanniefinks@users.noreply.github.com> Co-authored-by: Govind Ramnarayan <77341216+govindr-nm@users.noreply.github.com> Co-authored-by: Rahul Tuli <rahul@neuralmagic.com> Co-authored-by: Konstantin Gulin <66528950+KSGulin@users.noreply.github.com>

* add megasparse dir * edited readme to support DS integration * small edit to docstring * edited settings.py module so only 2 models are loaded. * added more context to the readme about adding more models. * fixed image * added different host to streamlit client as default * quality check edits * quality check commit * passing copyright quality check * content edits * rename dir to sparseserver-ui * added new config file for quick start * edited multipipelineclient in settings.py * changed name of config file * edited model stubs * edited readme * added dependency pins * changed server pin * edited model choice logic * altered front-end features * style update * renamed samples file * added variant descriptions * style changes * edited samples.py module * style changes * added new pic * edited README * updated readme cmds

* Revert "[cherry-pick] Release 0.12.1 cherry pick (#343)" This reverts commit ddf0ed6. * SparseServer.UI example [WIP] (#308) * add megasparse dir * edited readme to support DS integration * small edit to docstring * edited settings.py module so only 2 models are loaded. * added more context to the readme about adding more models. * fixed image * added different host to streamlit client as default * quality check edits * quality check commit * passing copyright quality check * content edits * rename dir to sparseserver-ui * added new config file for quick start * edited multipipelineclient in settings.py * changed name of config file * edited model stubs * edited readme * added dependency pins * changed server pin * edited model choice logic * altered front-end features * style update * renamed samples file * added variant descriptions * style changes * edited samples.py module * style changes * added new pic * edited README * updated readme cmds * SparseServer edit (#314) * edit number of models * edit settings.py * edit readme * AWS Sagemaker example integration (#305) * AWS Sagemaker example integration * documentation, sample config, dockerfile fixes * fix ecr repo name * readme code changes from testing * Update huggingface-transformers/README.md with new models (#329) * Update README.md (#330) * Update README.md various grammatical edits additional edits for section headline consistency * Topology file for HB120rs_v3 (#334) * Topology file for HB120rs_v3 Specifies core-per-CCX grouping for HB120rs_v3 VM's, used by multi-process script. * Update README.md to reference Azure topo file * Move all benchmarking within deepsparse/benchmark/ (#333) * Move all benchmarking within deepsparse/benchmark/ * Update benchmark_model * Expose results at benchmark base * isort * Skip flake8 * server integration check bug fix (#331) * server integration check bug fix need to verify integration is set before calling `integration.lower()` * respond to review - click choice * add default integration val to server config schema (#337) * Remove: startlette dep (#338) * Update src/deepsparse/version.py Co-authored-by: Ricky Costa <79061523+InquestGeronimo@users.noreply.github.com> Co-authored-by: Michael Goin <michael@neuralmagic.com> Co-authored-by: Jeannie Finks <74554921+jeanniefinks@users.noreply.github.com> Co-authored-by: Govind Ramnarayan <77341216+govindr-nm@users.noreply.github.com> Co-authored-by: Konstantin Gulin <66528950+KSGulin@users.noreply.github.com> Co-authored-by: Mark Kurtz <mark.kurtz@neuralmagic.com>

InquestGeronimo added 7 commits April 5, 2022 15:06

add megasparse dir

a071ac5

edited readme to support DS integration

1d7712a

small edit to docstring

9db5ca8

edited settings.py module so only 2 models are loaded.

47bfdbb

added more context to the readme about adding more models.

9eea303

fixed image

03bc62e

added different host to streamlit client as default

dc09979

InquestGeronimo requested review from markurtz, mgoin and rahul-tuli April 5, 2022 20:21

InquestGeronimo added 2 commits April 5, 2022 16:27

quality check edits

d1ec856

quality check commit

5f26aae

InquestGeronimo changed the title ~~Megasparse example [WIP]~~ MegaSparse example [WIP] Apr 5, 2022

mgoin reviewed Apr 5, 2022

View reviewed changes

examples/megasparse/README.md Show resolved Hide resolved

examples/megasparse/README.md Outdated Show resolved Hide resolved

examples/megasparse/client/app.py Outdated Show resolved Hide resolved

InquestGeronimo added 2 commits April 5, 2022 16:53

passing copyright quality check

970e9dd

content edits

a7fe2fb

InquestGeronimo self-assigned this Apr 5, 2022

markurtz suggested changes Apr 6, 2022

View reviewed changes

InquestGeronimo added 8 commits April 8, 2022 14:01

rename dir to sparseserver-ui

f0e2e50

added new config file for quick start

9195b90

edited multipipelineclient in settings.py

7714dce

changed name of config file

29ada34

changed name of config file

dee99e5

edited model stubs

9f3fdda

edited readme

a1a0a5b

added dependency pins

d53f792

InquestGeronimo force-pushed the megasparse branch from 1ae6beb to d53f792 Compare April 8, 2022 19:15

Merge branch 'main' into megasparse

e0a7080

InquestGeronimo changed the title ~~MegaSparse example [WIP]~~ SparseServer.UI example [WIP] Apr 8, 2022

InquestGeronimo added 3 commits April 9, 2022 18:25

renamed samples file

deefbe5

added variant descriptions

8951944

edited variant descriptions

dceb0bc

InquestGeronimo requested review from mgoin and rahul-tuli April 11, 2022 14:58

mgoin previously approved these changes Apr 11, 2022

View reviewed changes

examples/sparseserver-ui/client/samples.py Outdated Show resolved Hide resolved

style changes

684b46f

InquestGeronimo dismissed mgoin’s stale review via 684b46f April 11, 2022 15:28

style changes

14ea603

markurtz previously approved these changes Apr 11, 2022

View reviewed changes

mgoin reviewed Apr 11, 2022

View reviewed changes

examples/sparseserver-ui/client/samples.py Outdated Show resolved Hide resolved

examples/sparseserver-ui/client/samples.py Outdated Show resolved Hide resolved

examples/sparseserver-ui/client/samples.py Outdated Show resolved Hide resolved

edited samples.py module

4ecec22

InquestGeronimo dismissed markurtz’s stale review via 4ecec22 April 11, 2022 18:00

style changes

11f3f4d

mgoin previously approved these changes Apr 11, 2022

View reviewed changes

added new pic

476dfc6

InquestGeronimo dismissed mgoin’s stale review via 476dfc6 April 11, 2022 19:10

InquestGeronimo added 3 commits April 11, 2022 15:16

edited README

bc81378

edited README

ead19b8

edited README

96b2c76

InquestGeronimo requested a review from dbarbuzzi April 11, 2022 19:51

dbarbuzzi suggested changes Apr 11, 2022

View reviewed changes

examples/sparseserver-ui/README.md Outdated Show resolved Hide resolved

updated readme cdms

4f50de8

InquestGeronimo requested a review from dbarbuzzi April 11, 2022 20:15

dbarbuzzi approved these changes Apr 11, 2022

View reviewed changes

mgoin approved these changes Apr 11, 2022

View reviewed changes

InquestGeronimo merged commit 5e99ec7 into neuralmagic:main Apr 11, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

SparseServer.UI example [WIP] #308

SparseServer.UI example [WIP] #308

Uh oh!

InquestGeronimo commented Apr 5, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

markurtz left a comment •

edited

Loading

Uh oh!

mgoin commented Apr 7, 2022

Uh oh!

mgoin left a comment

Uh oh!

Uh oh!

markurtz left a comment

Uh oh!

mgoin left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dbarbuzzi left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

SparseServer.UI example [WIP] #308

SparseServer.UI example [WIP] #308

Uh oh!

Conversation

InquestGeronimo commented Apr 5, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

markurtz left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mgoin commented Apr 7, 2022

Uh oh!

mgoin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

markurtz left a comment

Choose a reason for hiding this comment

Uh oh!

mgoin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dbarbuzzi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

markurtz left a comment •

edited

Loading