Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Put caikit's base config back into the container #131

Merged
merged 1 commit into from
Nov 8, 2023

Conversation

Xaenalt
Copy link
Contributor

@Xaenalt Xaenalt commented Nov 6, 2023

Put caikit's base config back into the container
Rename caikit-.yaml to caikit-tgis.yaml to better reflect actual state
Update a few docs as well

Copy link
Contributor

openshift-ci bot commented Nov 6, 2023

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@Xaenalt Xaenalt marked this pull request as ready for review November 6, 2023 19:11
Copy link
Contributor

openshift-ci bot commented Nov 6, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Xaenalt

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@Xaenalt
Copy link
Contributor Author

Xaenalt commented Nov 6, 2023

@dtrifiro PTAL

@dtrifiro
Copy link
Contributor

dtrifiro commented Nov 7, 2023

Can you rebase this? It's likely only related to the apiversion changes

Rename caikit-*.yaml to caikit-tgis*.yaml to better reflect actual state
Update a few docs as well
@Xaenalt Xaenalt merged commit fa1e551 into opendatahub-io:main Nov 8, 2023
3 of 4 checks passed
@heyselbi heyselbi linked an issue Nov 9, 2023 that may be closed by this pull request
- name: RUNTIME_LOCAL_MODELS_DIR
value: /mnt/models
ports:
- containerPort: 8080
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Xaenalt it doesn't work for me if I use port 8080, it returns me HTTP invalid request error in the pod logs. It works if I use port 8085. Could you pls double check?

Copy link
Contributor

@dtrifiro dtrifiro Nov 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bdattoma: try changing:

diff --git a/demo/kserve/custom-manifests/caikit/caikit-tgis-servingruntime.yaml b/demo/kserve/custom-manifests/caikit/caikit-tgis-servingruntime.yaml
index 8528909..9fee801 100644
--- a/demo/kserve/custom-manifests/caikit/caikit-tgis-servingruntime.yaml
+++ b/demo/kserve/custom-manifests/caikit/caikit-tgis-servingruntime.yaml
@@ -21,7 +21,7 @@ spec:
       #     cpu: 8
       #     memory: 16Gi
     - name: transformer-container
-      image: quay.io/opendatahub/caikit-tgis-serving:stable
+      image: quay.io/opendatahub/caikit-tgis-serving:fast
       env:
         - name: RUNTIME_LOCAL_MODELS_DIR
           value: /mnt/models

if you want to use http the port is 8080, if you want to use grpc, the port is 8085

Copy link
Contributor

@bdattoma bdattoma Nov 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm using fast .

I wouldn't suggest to change the port here because the rest of the documentation instructs on querying via gRPC, not HTTP - unless the doc gets updated as well to report the options.
Could you plz provide an example of HTTP query for this runtime?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Include base config in the caikit container
4 participants