Skip to content

Add vLLM logic #35

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 179 commits into from
Oct 30, 2024
Merged

Add vLLM logic #35

merged 179 commits into from
Oct 30, 2024

Conversation

@trujillm
Copy link
Contributor

@trujillm trujillm commented Oct 2, 2024

We want to implement VLLM as part of the our RAG LLM demo.
These changes implement the following components to be deployed and available:

Serverless
serviceMesh
RHODS
Minio
with bucket creation
RHODS automated data connection
Populating minio bucket with model set

We have also updated the endpoint for the new model in the rag llm ui config

trujillm and others added 30 commits August 14, 2024 14:17
@trujillm
add infra
71a4aca
@trujillm
add values hub updates
00b3a0f
@trujillm
add bucket and expose configuration
dfe721b
@trujillm
enable ingress and dns on bucket
769ef93
@trujillm
remove dns for bucket
ed04af2
@trujillm
adjust port
5a4e9fa
@trujillm
adjust permissions
5b87e52
@trujillm
adjust security context
9d55a9a
@trujillm
remove allow privilegeEscalation
2a1e13b
@trujillm
update bucket to use dns
1ff7bec
@trujillm
remove dns
380af57
@trujillm
remove ingress and expose
e21a246
@trujillm
update securityContext
110bbd0
@trujillm
add openshift AI
06a62af
@trujillm
update security context and auto cert
ff7de63
@trujillm
Merge branch 'validatedpatterns:main' into add_rhods
c97e66d
@trujillm
add route update namespace
0366199
@trujillm
expose console and api
598259c
@trujillm
Add route for api
bd5ab29
@trujillm
adjust route minio api
034010f
@trujillm
adjust ROHDS operator deployment remove project
8abe3bf
@trujillm
add rhods operator adjustment
b2cfc51
@trujillm
remove rag-llm for testing
399ddcf
@trujillm
add rag-llm back adjust vault status check
9aee792
@trujillm
update rhods namespace
0b8e1ff
@trujillm
change minio deployment to use non operator
20514a9
@trujillm
Add DSC setup
db13371
@trujillm
add dsc config
add266e
@trujillm
Merge branch 'validatedpatterns:main' into add_rhods
85007c4
@trujillm
remove minio deployement for testing
51697c9
trujillm and others added 26 commits October 20, 2024 20:31
@trujillm
remove python get modelId
d0258ea
@trujillm
adjust var for model name using helm values
cc7b469
@trujillm
add env var for hugging face
626a779
@trujillm
add bash to extract model id name
3101007
@trujillm
adjust command to format the name
8c2d6c9
@trujillm
adjust format name for serving runtime
ce5ac44
@trujillm
fix typo
dfa991d
@trujillm
fix var names missed
833530e
@trujillm
format adjustment
1015606
@trujillm
fix typo
4d18e71
@trujillm
adjust model path
88ebf78
@redmikhail
Fixes networkpolicy to allow traffic from all ns
1e1acac
@trujillm
add dynamic name for config
e70102d
@trujillm
adjust config dynamic name
009c35f
@trujillm
update lookup for config
dccb5db
@trujillm
adjust lookup for syntax
2232499
@trujillm
adjust config again
7595d04
@trujillm
testing config lookup
79b87e4
@trujillm
add value for model id
ffba5a9
@trujillm
fix values path
1a325ba
@trujillm
format name for model id
3dbdb06
@trujillm
adjust value for global scope
c7180b7
@trujillm
testing helm template
e8efcb4
@trujillm
add format function
1b466f0
@trujillm
Merge pull request #5 from redmikhail/add_rhods
8174f82 
Fixes networkpolicy to allow traffic from all ns
@trujillm
update wait timeout add login hf
c9ed98e
@trujillm trujillm marked this pull request as ready for review October 29, 2024 23:44
sauagarwa
sauagarwa approved these changes Oct 30, 2024
View reviewed changes
@sauagarwa sauagarwa merged commit d172066 into validatedpatterns:main Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sauagarwa sauagarwa sauagarwa approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@trujillm @sauagarwa @redmikhail