[Kubernetes operator] GitHub example not working

[rubenharutyunov]

I'm trying to run the GitHub MCP server as provided in the example:

apiVersion: toolhive.stacklok.dev/v1alpha1
kind: MCPServer
metadata:
  name: github
  namespace: toolhive-system
spec:
  env:
  - name: GITHUB_API_URL
    value: https://api.github.com
  - name: LOG_LEVEL
    value: info
  image: docker.io/mcp/github
  permissionProfile:
    name: network
    type: builtin
  port: 8080
  resources:
    limits:
      cpu: 200m
      memory: 256Mi
    requests:
      cpu: 100m
      memory: 128Mi
  secrets:
  - key: GITHUB_PERSONAL_ACCESS_TOKEN
    name: github-token
  transport: stdio

But I'm getting this error:

Error: error instantiating secret manager failed to get secrets password: failed to read password: failed to read password: inappropriate ioctl for device
ToolHive needs a password to secure your credentials in the OS keyring.
This password will be used to encrypt and decrypt API tokens and other secrets
that need to be accessed by MCP servers. It will be securely stored in your OS keyring
so you won't need to enter it each time.
Please enter your keyring password:

I tried to run it in detached mode, however I'm now getting another error:

Error: error instantiating secret manager failed to get secrets password: detached process detected, cannot ask for password

Version: 0.0.1-dev.400_bcba31b

[ChrisJBurns]

Thanks @rubenharutyunov I'll look into this ASAP

[danbarr]

Two things happening I think:

1. The secrets field in our example should look like:

     secrets:
       - name: github-token
         key: token
         targetEnvName: GITHUB_PERSONAL_ACCESS_TOKEN

   This assumes the secret is created like kubectl create secret generic github-token -n toolhive-system --from-literal=token=<YOUR_TOKEN>, which doesn't quite match the cmd/thv-operator/README.md example which uses the default namespace in the example manifest and command

2. Fixing the above, I also see the issue with thv trying to initialize the secrets provider, which shouldn't happen in a K8s context

[ChrisJBurns]

@rubenharutyunov Would you be able to test out @danbarr's first point and let us know if it fixes your issue?

@danbarr On point 2. I can imagine a point in the future where we would initialise secret providers such as AWS Secrets Manager, Vault etc. However for now, since we are only using Kubernetes secrets, we should probably stop the provider initialisation.

[danbarr]

@ChrisJBurns Yeah the problem (even with the fixed manifest) is that the thv CLI attempts to initialize the default encrypted secrets provider and prompts you interactively to set a password, which bombs out the container with the error that @rubenharutyunov noted.

[JAORMX]

We probably should have a none provider.

[ChrisJBurns]

Have created #536 to track point 2. made above. Shall wait to hear back from @rubenharutyunov on point 1.

[rubenharutyunov]

@ChrisJBurns

Would you be able to test out @danbarr's first point and let us know if it fixes your issue?

I initially created the secret in a correct namespace and referenced it correctly. While it's of course good to keep documentation correct, that unfortunately doesn't fix the issue.

On point 2. I can imagine a point in the future where we would initialise secret providers such as AWS Secrets Manager, Vault etc. However for now, since we are only using Kubernetes secrets, we should probably stop the provider initialization.

It's a bit unclear to me what's the use case of supporting other secrets providers than none in Kubernetes, which itself has a common ecosystem and tooling to support secrets management. Generally, I think it's more than enough to provide a way to propagate values from Kubernetes secrets to the actual MCP server container, the rest can be handled by Kubernetes (or any supported platform in the future).

[ChrisJBurns]

@rubenharutyunov So it's entirely possible that users will want external secrets provider integration like Vault/AWS Secrets Manager, so we'll need to stay open to requests for support.

On the Kubernetes secret side of things, there's a couple of things that we can do:

1. Provide a Kubernetes secrets provider that allows the code to directly read the secrets it needs, ensuring that the secret itself isn't exposed as an env var to the runner, however given the MCP servers themselves can vary, its likely that the secret gets sourced as an environment variable into the MCP container anyways
2. Provide a none secrets provider that in the Kubernetes case stops the initialisation of a provider, removing the error you have above. This then would allow for us to inject secrets as environment variables into the runner container which is currently what we expect to happen.

I think for now, we can go with option 2. so we can unblock yourself and others trying to use secrets, then in future possibly implement 1. to secure things a bit more.

[danbarr]

I tested again now that the none secrets provider landed in v0.0.45 and the chart was bumped to that version. Now instead of erroring out because of the prompt to set the encryption password, we get:

Error: failed to get secrets: secret not found: github-token/token (none provider doesn't store secrets)

@ChrisJBurns does the operator need to be updated to just inject the env vars itself instead of passing them to thv's --secrets flag?