Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

export "num_tokens_per_iter", "max_prefill_iters" and etc when converting a model #845

Merged
merged 3 commits into from
Dec 21, 2023

Conversation

lvhan028
Copy link
Collaborator

Motivation

    num_tokens_per_iter: int = 1
    max_prefill_iters: int = 1
    extra_tokens_per_iter: int = 0

are used to adjust the performance of prefill

Modification

base.py

BC-breaking (Optional)

No

Use cases (Optional)

If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.

Checklist

  1. Pre-commit or other linting tools are used to fix the potential lint issues.
  2. The modification is covered by complete unit tests. If not, please add more unit tests to ensure the correctness.
  3. If the modification has a dependency on downstream projects of a newer version, this PR should be tested with all supported versions of downstream projects.
  4. The documentation has been modified accordingly, like docstring or example tutorials.

@lvhan028 lvhan028 requested a review from irexyc December 20, 2023 11:34
@lvhan028 lvhan028 merged commit cc316a3 into InternLM:main Dec 21, 2023
3 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants