-
Notifications
You must be signed in to change notification settings - Fork 11
Respect context size from model config #93
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Emily Casey <emily.casey@docker.com>
if modelCfg.ContextSize != nil { | ||
args = append(args, "--ctx-size", strconv.FormatUint(*modelCfg.ContextSize, 10)) | ||
} | ||
|
||
// Add arguments from backend config | ||
if config != nil { | ||
if config.ContextSize > 0 && !containsArg(args, "--ctx-size") { | ||
args = append(args, "--ctx-size", fmt.Sprintf("%d", config.ContextSize)) | ||
} | ||
args = append(args, config.RuntimeFlags...) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would unify the number-to-string conversion; strconv.Itoa
is probably the most efficient (vs. strconv.FormatUint
or fmt.Sprintf
).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can use strconv.FormatInt
and strconv.FormatUint
for the backend config and artifact config respectively. strconv.Itoa
accepts an int
type rather than an int64
. I was assuming we chose int64 in the backend config for a reason and wouldn't want to risk losing precision.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, roger that.
Signed-off-by: Emily Casey <emily.casey@docker.com>
Signed-off-by: Emily Casey <emily.casey@docker.com> Co-authored-by: Jacob Howard <jacob.howard@docker.com>
Signed-off-by: Emily Casey <emily.casey@docker.com>
Uh oh!
There was an error while loading. Please reload this page.