Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(container): vLLM build and base image strategies #142

Merged
merged 42 commits into from
Jul 31, 2023
Merged

feat(container): vLLM build and base image strategies #142

merged 42 commits into from
Jul 31, 2023

Conversation

aarnphm
Copy link
Collaborator

@aarnphm aarnphm commented Jul 25, 2023

Signed-off-by: aarnphm-ec2-dev 29749331+aarnphm@users.noreply.github.com

@github-actions
Copy link
Contributor

github-actions bot commented Jul 25, 2023 •
edited
Loading

Code Coverage

Package Statements
openllm 31.55% (1600 / 5071)
Summary 31.55% (1600 / 5071)

@aarnphm aarnphm force-pushed the build-vllm branch 13 times, most recently from 9e422c2 to c16da45 Compare July 29, 2023 06:42
@aarnphm aarnphm marked this pull request as ready for review July 29, 2023 09:34
@aarnphm
fix(models): update configuration exception message
80911fb 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
fix(logs): correctly ignore ConflictError from circus
35f83e0 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore(llama): rename to convention
6632743 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
fix: Using ray workers for multiple GPUs with vLLM
bc1e28b 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
perf: reduce save time and genearl CLI docs enhancement
816fcfa 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore(cli): improve tiny configuration and factory options
688db14 
feat(ci): add jobs for building and publishing base container image

Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore: update changelog
dff3c4d 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore: only login for GHCR when main and release not PR
bc88650 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore: update annotation
72dfd30 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore: update subnets and security id
2edbc10 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore: try to use smaller instance typpppe
e9fd907 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
@aarnphm
fix: remove returning runner for for_model
e4c3775 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore: running pre-commit [generated]
cd8be6a 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore: update ami template
ad61100 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore: ok I updated subnet and correct set ACLs
3ed5571 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
@aarnphm
fix: correct ami
8170dee 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore(ci): using my fork of ec2-hosted runner
c4a784c 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore: update AMI with more simple template
8284922 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore: remove runner-home-dir
81a418a 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore: update ami id
3f0fbf1 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore: update to latest aws sdk
6800431 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
@aarnphm
fix: set to all public subnet for now
716e491 
Signed-off-by: aarnphm-ec2-dev <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore: using different security group
a410a66 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
fix: update correct security group
9b9080b 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
fix(ec2): to use public registry
3dcfe20 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore: fix correct tag suffix
f3c1379 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore(ci): add more GPUs for faster build
23fccfe 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore: update cache-to to max mode to minimize build time
ced5542 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
ci: saving interim cache to S3 buckets
31cb863 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
fix: correct set manifest
d0261b0 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
Copy link
Collaborator Author

aarnphm commented Jul 30, 2023 •
edited
Loading

Will update the option for different base image with a follow up PR, this PR is already a lot of changes

@aarnphm
fix: typo and add access_key_id
b665b28 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
revert: remove secret_access_key and access_key_id inline as it respects
56c066e 
envvar

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore: using public registry
98fb429 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore: using smaller instance for more availability
0f093a4 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
revert: "chore: using smaller instance for more availability"
0a7b902 
This reverts commit 0f093a4.
@aarnphm
chore: fix container registry and update CLI args
68b367a 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore: update client annotations downstream
ebfeac7 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
chore: update VPC to change availability zone
9593715 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm
fix: remove private subnets and update security group
ee6b2e1 
Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>
@aarnphm aarnphm merged commit ef94c6b into main Jul 31, 2023
25 checks passed
@aarnphm aarnphm deleted the build-vllm branch July 31, 2023 06:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@aarnphm