Skip to content

Conversation

@p-ferreira
Copy link
Contributor

@p-ferreira p-ferreira commented May 13, 2024

  • update vllm memory requirements
  • update system prompt
  • drops python 3.9 from ci
  • adds --neuron.gpus and --neuron.llm_max_allowed_memory_in_gb new parameters and integration
  • changes base model to casperhansen/llama-3-70b-instruct-awq
  • adjust unit tests

@p-ferreira p-ferreira requested review from bkb2135 and steffencruz May 14, 2024 15:48
@p-ferreira p-ferreira marked this pull request as ready for review May 14, 2024 15:49
@p-ferreira
Copy link
Contributor Author

p-ferreira commented May 16, 2024

@dbobrenko Unfortunately I cannot add you as a reviewer in this space but please consider yourself as a reviewer of all PRs of this repo (specially the ones with tag 2.3.0)

Copy link
Collaborator

@dbobrenko dbobrenko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we update min_compute.yaml also, and README.md?

Overall, LGTM!

Copy link
Collaborator

@dbobrenko dbobrenko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

p-ferreira and others added 3 commits May 17, 2024 11:51
Co-authored-by: Dmytro Bobrenko <17252809+dbobrenko@users.noreply.github.com>
Co-authored-by: Dmytro Bobrenko <17252809+dbobrenko@users.noreply.github.com>
Copy link
Collaborator

@bkb2135 bkb2135 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good and has been run on main stabely.

@p-ferreira p-ferreira merged commit 9c5ce89 into staging May 17, 2024
@p-ferreira p-ferreira mentioned this pull request May 17, 2024
@Hollyqui Hollyqui deleted the features/validator-model branch August 2, 2024 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants