Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update llama triton example #1153

Merged
merged 4 commits into from
Feb 21, 2024
Merged

update llama triton example #1153

merged 4 commits into from
Feb 21, 2024

Conversation

zhyncs
Copy link
Contributor

@zhyncs zhyncs commented Feb 19, 2024

Motivation

After seeing #1101, I verified it in my local development environment and it ran normally. At the same time, I found that this example was not easy to use, so I added some comments and synchronized start_ids.csv from FasterTransformer.

Modification

as titled

Checklist

  1. Pre-commit or other linting tools are used to fix the potential lint issues.
  2. The modification is covered by complete unit tests. If not, please add more unit tests to ensure the correctness.
  3. If the modification has a dependency on downstream projects of a newer version, this PR should be tested with all supported versions of downstream projects.
  4. The documentation has been modified accordingly, like docstring or example tutorials.

@zhyncs
Copy link
Contributor Author

zhyncs commented Feb 19, 2024

e.g.

CUDA_VISIBLE_DEVICES=0 ./bld/bin/llama_triton_example /workdir/lmdeploy/examples/cpp/llama/llama_config.ini /workdir/lmdeploy/examples/cpp/llama/start_ids.csv

@zhyncs
Copy link
Contributor Author

zhyncs commented Feb 21, 2024

Hi @lzhangzz May you help review this pr? Thanks.

@lvhan028 lvhan028 merged commit 685070f into InternLM:main Feb 21, 2024
7 checks passed
@zhyncs zhyncs deleted the patch-4 branch February 21, 2024 10:29
grimoire pushed a commit to grimoire/lmdeploy that referenced this pull request Feb 22, 2024
* update llama triton example

* update start_ids.csv

* fix lint
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants