Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

VSO failed to connect through CloudFlare Zero Trust #622

Closed
kalote opened this issue Feb 23, 2024 · 4 comments · Fixed by #629
Closed

VSO failed to connect through CloudFlare Zero Trust #622

kalote opened this issue Feb 23, 2024 · 4 comments · Fixed by #629
Labels
bug Something isn't working good first issue Good for newcomers
Milestone
v0.5.2

Comments

@kalote
Copy link

kalote commented Feb 23, 2024 •
edited

Describe the bug
We have our vault protected by CloudFlare Zero Trust. I have a client_id and client_secret tokens that I need to pass to go through the ZT. When using the CLI locally, the tokens works fine (using vault login -header=cf-client=xxx ...).
When adding those information as headers in the default vaultConnection, I get a 403:

Failed to check Vault seal status: Error making API request. URL: GET https://example.cloudflareaccess.com/cdn-cgi/access/login/vault.example.com Code: 403. Raw Message: <html> <head><title>403 Forbidden</title></head> <body> <center><h1>403 Forbidden</h1></center> <hr><center>cloudflare</center> </body> </html>

We're using helm charts with following info:

  • vault-secrets-operator version: 0.4.3
  • vault version: 0.27.0

The values.yaml (for the operator):

operator:
  enabled: true
  controller:
    replicas: 1
  defaultVaultConnection:
    enabled: true
    address: "https://vault.example.com"
    skipTLSVerify: false
    headers:
      CF-Access-Client-Id: "1f..."
      CF-Access-Client-Secret: "58..."
  defaultAuthMethod:
    enabled: false

When checking in my K8s cluster the vaultConnection resource, I can see the headers:

apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultConnection
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: > {longobject}
  finalizers:
    - vaultconnection.secrets.hashicorp.com/finalizer
  generation: 4
  labels:
    app.kubernetes.io/component: controller-manager
    app.kubernetes.io/instance: vault-dev
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: operator
    app.kubernetes.io/version: 0.4.3
    argocd.argoproj.io/instance: vault-dev
    component: controller-manager
    control-plane: controller-manager
    helm.sh/chart: operator-0.4.3
  name: default
  namespace: vault
spec:
  address: 'https://vault.example.com'
  headers:
    CF-Access-Client-Id: 1f...
    CF-Access-Client-Secret: 58...
  skipTLSVerify: false
status:
  valid: true

Expected behaviour
The vault-secrets-operator will use the headers to bypass CloudFlare Zero Trust

Environment

  • Kubernetes version:
    • Distribution or cloud vendor (OpenShift, EKS, GKE, AKS, etc.): EKS, version 1.28

Is there something that I miss here? I haven't been able to find a good documentation on this specific setup =/

Also, To make it work, do I need to update additional resources (VaultAuth, VaultStaticSecret, ...)?
Thanks 🙏
@kalote kalote added the bug Something isn't working label Feb 23, 2024
@kalote kalote changed the title Cannot connect through CloudFlare Zero Trust VSO failed to connect through CloudFlare Zero Trust Feb 23, 2024
@kalote
Copy link
Author

kalote commented Feb 23, 2024

I've update the vault-secrets-operator version to 0.5.1, but still the same issue.

@tvoran tvoran added this to the v0.5.2 milestone Feb 23, 2024
@tvoran
Copy link
Member

tvoran commented Feb 23, 2024 •
edited

Hi @kalote, thanks for pointing this out! Indeed it looks like spec.headers from VaultConnection is not being set correctly on the vault client. We should be able to get a fix for this into the next release.

@tvoran tvoran added the good first issue Good for newcomers label Feb 23, 2024
@kalote
Copy link
Author

kalote commented Feb 25, 2024

Thanks a lot @tvoran !

QQ:

  • Do you know when will be the next release?
  • Is there a flag that you could add to the helm chart to help debug a bit more ... e.g., being able to see the requests that are sent to the vault for instance? something like:
operator:
  enabled: true
  debug: true # or log-level: debug

Thanks 🙏

@kalote kalote mentioned this issue Feb 26, 2024
Open
@tvoran tvoran mentioned this issue Feb 29, 2024
Merged
@tvoran
Copy link
Member

tvoran commented Mar 1, 2024

Setting -zap-log-level=5 in controller.manager.extraArgs may give some of what you're looking for in terms of debugging.

No ETA yet on the next release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working good first issue Good for newcomers
Projects
None yet
Milestone
v0.5.2
Development

Successfully merging a pull request may close this issue.

Set VaultConnection headers on the vault client hashicorp/vault-secrets-operator
2 participants
@tvoran @kalote