Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move model code into separate, feature-flagged crate #1335

Merged
merged 11 commits into from
May 9, 2024
Merged

Move model code into separate, feature-flagged crate #1335

merged 11 commits into from
May 9, 2024

Conversation

Jeadie
Copy link
Contributor

@Jeadie Jeadie commented May 8, 2024 •
edited

Changes

  • New feature-flagged crate model_components, flagged on models.
  • Takes model code from runtime crate.
  • New make target make install-with-models
  • Reduces binary size from 152 mb -> 138 mb
  • When not built with models, model HTTP endpoints aren't supported (e.g. GET v1/models).
  • Note: No functional changes are expected, except for:
    • Infer model format from model path (i.e. the path downloaded from ModelSource).
    • Infer what runtime to use for model format

@Jeadie Jeadie self-assigned this May 8, 2024
@Jeadie Jeadie marked this pull request as ready for review May 8, 2024 05:13
@Jeadie Jeadie requested a review from a team as a code owner May 8, 2024 05:13
Jeadie
Jeadie commented May 8, 2024
View reviewed changes
crates/models/src/model.rs Outdated Show resolved Hide resolved
Jeadie
Jeadie commented May 8, 2024
View reviewed changes
crates/runtime/src/model.rs Outdated Show resolved Hide resolved
phillipleblanc
phillipleblanc reviewed May 8, 2024
View reviewed changes
crates/runtime/Cargo.toml Outdated Show resolved Hide resolved
crates/runtime/Cargo.toml Outdated Show resolved Hide resolved
sgrebnov
sgrebnov reviewed May 8, 2024
View reviewed changes
Makefile Outdated Show resolved Hide resolved
@phillipleblanc phillipleblanc marked this pull request as draft May 9, 2024 01:44
@Jeadie Jeadie marked this pull request as ready for review May 9, 2024 04:08
@Jeadie Jeadie enabled auto-merge (squash) May 9, 2024 05:52
@Jeadie Jeadie merged commit 4de4bfe into trunk May 9, 2024
15 of 16 checks passed
@Jeadie Jeadie deleted the jeadie/24-05-08/packages branch May 9, 2024 05:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ewgenius ewgenius ewgenius left review comments

@sgrebnov sgrebnov sgrebnov left review comments

@phillipleblanc phillipleblanc phillipleblanc approved these changes

Assignees

@Jeadie Jeadie

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@Jeadie @ewgenius @phillipleblanc @sgrebnov