Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
mgoin committed Apr 12, 2024
1 parent c87e2f9 commit cfe1baa
Showing 1 changed file with 6 additions and 3 deletions.
9 changes: 6 additions & 3 deletions src/deepsparse/transformers/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,9 +151,12 @@ https://sparsezoo.neuralmagic.com/?useCase=text_generation)
```python
from deepsparse import Pipeline

opt_pipeline = Pipeline.create(task="opt", model_path="zoo:opt-1.3b-opt_pretrain-quantW8A8")
llama_pipeline = Pipeline.create(
task="text-generation",
model_path="zoo:llama2-7b-ultrachat200k_llama2_pretrain-pruned50_quantized"
)

inference = opt_pipeline("Who is the president of the United States?")
inference = llama_pipeline("Who is the president of the United States?")

>> 'The president of the United States is the head of the executive branch of government...'
```
Expand All @@ -163,7 +166,7 @@ Spinning up:
```bash
deepsparse.server \
--task text-generation \
--model_path zoo:opt-1.3b-opt_pretrain-pruned50_quantW8A8
--model_path zoo:llama2-7b-ultrachat200k_llama2_pretrain-pruned50_quantized
```

Making a request:
Expand Down

0 comments on commit cfe1baa

Please sign in to comment.