-
Notifications
You must be signed in to change notification settings - Fork 39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Put caikit's base config back into the container #131
Conversation
Xaenalt
commented
Nov 6, 2023
Skipping CI for Draft Pull Request. |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: Xaenalt The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
@dtrifiro PTAL |
bcf04ca
to
fcf338c
Compare
Can you rebase this? It's likely only related to the apiversion changes |
Rename caikit-*.yaml to caikit-tgis*.yaml to better reflect actual state Update a few docs as well
- name: RUNTIME_LOCAL_MODELS_DIR | ||
value: /mnt/models | ||
ports: | ||
- containerPort: 8080 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Xaenalt it doesn't work for me if I use port 8080, it returns me HTTP invalid request error in the pod logs. It works if I use port 8085. Could you pls double check?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bdattoma: try changing:
diff --git a/demo/kserve/custom-manifests/caikit/caikit-tgis-servingruntime.yaml b/demo/kserve/custom-manifests/caikit/caikit-tgis-servingruntime.yaml
index 8528909..9fee801 100644
--- a/demo/kserve/custom-manifests/caikit/caikit-tgis-servingruntime.yaml
+++ b/demo/kserve/custom-manifests/caikit/caikit-tgis-servingruntime.yaml
@@ -21,7 +21,7 @@ spec:
# cpu: 8
# memory: 16Gi
- name: transformer-container
- image: quay.io/opendatahub/caikit-tgis-serving:stable
+ image: quay.io/opendatahub/caikit-tgis-serving:fast
env:
- name: RUNTIME_LOCAL_MODELS_DIR
value: /mnt/models
if you want to use http the port is 8080, if you want to use grpc, the port is 8085
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm using fast
.
I wouldn't suggest to change the port here because the rest of the documentation instructs on querying via gRPC, not HTTP - unless the doc gets updated as well to report the options.
Could you plz provide an example of HTTP query for this runtime?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.