Skip to content
This repository was archived by the owner on Jun 3, 2025. It is now read-only.

Conversation

@rahul-tuli
Copy link
Member

@rahul-tuli rahul-tuli commented May 3, 2022

This PR accomplishes the following:

  • Update deepsparse.transformers.pipelines_cli to work with new pipelines refactor
  • Update a few typos in docstring in pipeline.py
  • Also noting that from deepsparse import Pipeline works

The local tests performed are as follows:

deepsparse.run_inference \
    --model-path zoo:nlp/token_classification/bert-base/pytorch/huggingface/conll2003/12layer_pruned80_quant-none-vnni \
    --task ner \
    --data input.txt

Input (input.txt):

My name is rahul and i am a student of computer science and engineering.
Output in out.json
```json
"{\"predictions\": [[{\"entity\": \"LABEL_0\", \"score\": 0.9998608231544495, \"index\": 1, \"word\": \"my\", \"start\": 0, \"end\": 2, \"is_grouped\": false}, {\"entity\": \"LABEL_0\", \"score\": 0.9996172785758972, \"index\": 2, \"word\": \"name\", \"start\": 3, \"end\": 7, \"is_grouped\": false}, {\"entity\": \"LABEL_0\", \"score\": 0.9998311996459961, \"index\": 3, \"word\": \"is\", \"start\": 8, \"end\": 10, \"is_grouped\": false}, {\"entity\": \"LABEL_1\", \"score\": 0.999155580997467, \"index\": 4, \"word\": \"ra\", \"start\": 11, \"end\": 13, \"is_grouped\": false}, {\"entity\": \"LABEL_1\", \"score\": 0.7973057627677917, \"index\": 5, \"word\": \"##hul\", \"start\": 13, \"end\": 16, \"is_grouped\": false}, {\"entity\": \"LABEL_0\", \"score\": 0.9999052286148071, \"index\": 6, \"word\": \"and\", \"start\": 17, \"end\": 20, \"is_grouped\": false}, {\"entity\": \"LABEL_0\", \"score\": 0.999871015548706, \"index\": 7, \"word\": \"i\", \"start\": 21, \"end\": 22, \"is_grouped\": false}, {\"entity\": \"LABEL_0\", \"score\": 0.9999074339866638, \"index\": 8, \"word\": \"am\", \"start\": 23, \"end\": 25, \"is_grouped\": false}, {\"entity\": \"LABEL_0\", \"score\": 0.9998987913131714, \"index\": 9, \"word\": \"a\", \"start\": 26, \"end\": 27, \"is_grouped\": false}, {\"entity\": \"LABEL_0\", \"score\": 0.9998655319213867, \"index\": 10, \"word\": \"student\", \"start\": 28, \"end\": 35, \"is_grouped\": false}, {\"entity\": \"LABEL_0\", \"score\": 0.9998788237571716, \"index\": 11, \"word\": \"of\", \"start\": 36, \"end\": 38, \"is_grouped\": false}, {\"entity\": \"LABEL_0\", \"score\": 0.9884167313575745, \"index\": 12, \"word\": \"computer\", \"start\": 39, \"end\": 47, \"is_grouped\": false}, {\"entity\": \"LABEL_0\", \"score\": 0.9930113554000854, \"index\": 13, \"word\": \"science\", \"start\": 48, \"end\": 55, \"is_grouped\": false}, {\"entity\": \"LABEL_0\", \"score\": 0.9991596937179565, \"index\": 14, \"word\": \"and\", \"start\": 56, \"end\": 59, \"is_grouped\": false}, {\"entity\": \"LABEL_0\", \"score\": 0.9913791418075562, \"index\": 15, \"word\": \"engineering\", \"start\": 60, \"end\": 71, \"is_grouped\": false}, {\"entity\": \"LABEL_0\", \"score\": 0.9998722672462463, \"index\": 16, \"word\": \".\", \"start\": 71, \"end\": 72, \"is_grouped\": false}]]}"

Tests to check pipeline import:

Python 3.8.12 (default, Jan 15 2022, 18:39:47) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from deepsparse import Pipeline
>>> Pipeline.create(
... task="ner"
... )
DeepSparse Engine, Copyright 2021-present / Neuralmagic, Inc. version: 0.12.0 (5f03c9f3) (release) (optimized) (system=avx512, binary=avx512)
<deepsparse.transformers.pipelines.token_classification.TokenClassificationPipeline object at 0x7f37b31bc6d0>
>>> 

Updated a few typos in docstring in pipeline.py
Also noting that `from deepsparse import Pipeline` works now
@rahul-tuli rahul-tuli requested a review from bfineran May 3, 2022 16:42
@rahul-tuli rahul-tuli requested review from a team, anmarques and natuan and removed request for a team May 3, 2022 16:48
@rahul-tuli rahul-tuli self-assigned this May 3, 2022
@bfineran bfineran merged commit d4824e5 into main May 3, 2022
@bfineran bfineran deleted the bugfix-deepsparse.run_inference branch May 3, 2022 19:45
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants