Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.

Conversation

@louis-jan
Copy link
Contributor

@louis-jan louis-jan commented Jul 2, 2024

Describe Your Changes

  • This is to support cortex engines init command, for the case that support engines ahead of time, eg docker build.
// Llama.cpp engine
cortex engines init cortex.llamacpp

// ONNX engine
cortex engines init cortex.onnx

// Tensorrt-LLM engine
cortex engines init cortex.tensorrt-llm

image

// DO NOT INIT REMOTE ENGINES

Fixes Issues

Self Checklist

  • Added relevant comments, esp in complex areas
  • Updated docs (for bug fixes / features)
  • Created issues for follow-up changes or refactoring needed

@louis-jan louis-jan force-pushed the feat/cortex-engines-init branch 2 times, most recently from f0ecf0a to 07c5b76 Compare July 2, 2024 08:38
@louis-jan louis-jan force-pushed the feat/cortex-engines-init branch from 07c5b76 to 7c2f1c1 Compare July 2, 2024 08:44
@louis-jan louis-jan requested a review from irfanpena July 2, 2024 08:47
@louis-jan louis-jan merged commit d87bbe2 into dev Jul 2, 2024
@louis-jan louis-jan deleted the feat/cortex-engines-init branch July 2, 2024 10:33
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: cortex should support engines init to reinit engines manually

4 participants