Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.

refactor: shorten cortex setup - save cortex cpp serve configs #747

Merged
merged 7 commits into from
Jun 22, 2024
Merged

refactor: shorten cortex setup - save cortex cpp serve configs #747

merged 7 commits into from
Jun 22, 2024

Conversation

@louis-jan
Copy link
Contributor

@louis-jan louis-jan commented Jun 21, 2024 •
edited
Loading

Describe Your Changes

  • Refactor cortex setup: automatically detect and install CUDA dependencies when running on GPU, eliminating the need for user input.
  • Every server configuration should be persisted to a configuration file, ensuring that subsequent requests are directed to the correct endpoints, rather than incorrect ones.
  • Faster code compiling with Bun.
  • Update cortex-cpp port to 3929
  • Pull engine along with model
  • Gate ONNX on Linux & MacOS

Fixes Issues

  • Closes #
  • Closes #

Self Checklist

  • Added relevant comments, esp in complex areas
  • Updated docs (for bug fixes / features)
  • Created issues for follow-up changes or refactoring needed

@louis-jan louis-jan force-pushed the refactor/cortex-cpp-configs branch from 92341b8 to a31e4a2 Compare June 21, 2024 15:43
@louis-jan louis-jan force-pushed the refactor/cortex-cpp-configs branch from a31e4a2 to d627265 Compare June 21, 2024 15:44
@louis-jan louis-jan force-pushed the refactor/cortex-cpp-configs branch 2 times, most recently from 592c83b to d95e64a Compare June 21, 2024 16:19
@louis-jan louis-jan force-pushed the refactor/cortex-cpp-configs branch from d95e64a to b4a5f0a Compare June 21, 2024 16:20
@louis-jan louis-jan force-pushed the refactor/cortex-cpp-configs branch from eb6ade3 to d81e6c4 Compare June 21, 2024 17:59
@louis-jan louis-jan force-pushed the refactor/cortex-cpp-configs branch from d81e6c4 to 192a891 Compare June 21, 2024 18:03
@louis-jan louis-jan force-pushed the refactor/cortex-cpp-configs branch 2 times, most recently from 22abdc6 to f924509 Compare June 22, 2024 10:30
@louis-jan louis-jan force-pushed the refactor/cortex-cpp-configs branch from f924509 to f86e090 Compare June 22, 2024 10:37
@louis-jan louis-jan requested a review from vansangpfiev June 22, 2024 10:39
@louis-jan louis-jan merged commit f5e6b2f into dev Jun 22, 2024
@louis-jan louis-jan deleted the refactor/cortex-cpp-configs branch June 22, 2024 13:09
cahyosubroto pushed a commit that referenced this pull request Jun 26, 2024
@louis-jan @irfanpena
refactor: shorten cortex setup - save cortex cpp serve configs (#747)
c0e4c91
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@marknguyen1302 marknguyen1302 Awaiting requested review from marknguyen1302

@vansangpfiev vansangpfiev Awaiting requested review from vansangpfiev

1 more reviewer

@namchuai namchuai namchuai approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@louis-jan @namchuai