[Question][DevOps] Deploy devlake with managed GCP Cloud database service (with SSL enabled) #7928
Comments
pmallampati-sift
changed the title
|
Hello @pmallampati-sift! I'm here to help you with any questions or issues you might have. I can assist with bug fixes, troubleshooting, and becoming a contributor. To deploy DevLake with a MySQL database hosted on GCP with SSL enabled using a Helm chart, you need to pass the SSL certificates in the DB connection string. Here are the steps to achieve this:
Replace |
|
Per above suggestion when deployed with above config it says unknown value error log from DB |
|
To correctly configure the Make sure to replace |
|
@dosu |
|
To resolve the "x509: cannot validate certificate for X.X.X.X(ip) because it doesn't contain any IP SANs" error when deploying DevLake with SSL, you can add your root CA to the devlake:
image: apache/devlake:v...
...
volumes:
...
- /path/to/your/rootCA.crt:/usr/local/share/ca-certificates/rootCA.crt
command: [ "sh", "-c", "update-ca-certificates; lake" ]
...This approach involves mounting your |
|
@dosu |
|
To address the issue with the missing SAN names in the SSL certificates generated by GCP Cloud SQL, you can add your root CA to the devlake:
image: apache/devlake:v...
...
volumes:
...
- /path/to/your/rootCA.crt:/usr/local/share/ca-certificates/rootCA.crt
command: [ "sh", "-c", "update-ca-certificates; lake" ]
...Since you are using Helm, you can adapt this approach by adding the root CA to your Helm chart configuration. Here is an example of how to do this:
This setup should help in cases where the generated SSL certificates have CN=Google Cloud SQL Server CA but are missing SAN names, and mounting the required certs and setting |
|
@dosu Also placed Also in the init container I am seeing below error.
|
|
To resolve the "x509: cannot validate certificate for 10.252.182.24 because it doesn't contain any IP SANs" error when deploying DevLake with SSL, you need to ensure that your root CA is properly added to the Here is a solution based on the provided documentation:
Here is an example of how you can modify your devlake:
image: apache/devlake:v...
...
volumes:
...
- /path/to/your/rootCA.crt:/usr/local/share/ca-certificates/rootCA.crt
command: [ "sh", "-c", "apt-get update && apt-get install -y ca-certificates && update-ca-certificates; lake" ]
...This command ensures that the Additionally, if you encounter issues with self-signed certificates, you can set the environment variable For more detailed troubleshooting, you can refer to the Configuration and Blueprint Troubleshooting section in the DevLake documentation. Specifically, the section on "Failed to collect data from the server with a self-signed certificate" provides relevant information and solutions [1][2][3][4][5]. |
|
@dosu |
|
To execute the steps for installing the necessary package and running
Replace This configuration ensures that the |
|
Per your suggestion if we do below step, do we still need to create the `lake: volumes: |
|
Yes, you still need to create the |
|
In that case its still not working with above suggestion I was able to get the both below init containers up and running.
But after that the lake container is still failing with |
|
To resolve the "x509: cannot validate certificate for 10.252.182.24 because it doesn't contain any IP SANs" error, you can bypass the certificate verification by setting the Here is how you can update your Helm chart configuration to include this environment variable:
Replace This configuration will bypass the certificate verification, allowing the |
|
It appears that MySQL SSL connections are currently not supported. For now, I recommend trying a connection without an SSL certificate. |
Startrekzky
changed the title
|
Thank you @klesh yes mysql SSL were not supported, but we were able to resolve the issue by connecting using cloudsql-auth-proxy. |
|
Hi @dosu |
|
To add new plugins to DevLake using a Helm chart, follow these steps:
This will apply the new configuration and include the new plugin in your DevLake installation [1][2]. |
|
hi @dosu |
|
Currently, there is no specific documentation or configuration guide for integrating DevLake with OKTA SSO. However, DevLake does support SSO integration through the Here is a general approach to integrate DevLake with OKTA SSO using
This setup should help you integrate DevLake with OKTA SSO using |
|
@dosu |
|
To generate a value for Here are a few ways to generate this secret: Using OpenSSLYou can use the openssl rand -base64 32Using PythonYou can also use Python to generate a random string: import os
import base64
print(base64.b64encode(os.urandom(32)).decode('utf-8'))Using Linux Command LineIf you are on a Linux system, you can use the following command: head -c32 /dev/urandom | base64ExampleOnce you have generated the secret, it will look something like this: You can then use this value in your authproxy:
enabled: true
image: quay.io/oauth2-proxy/oauth2-proxy:v7.4.0-amd64
env:
- name: OAUTH2_PROXY_PROVIDER
value: oidc
- name: OAUTH2_PROXY_PROVIDER_DISPLAY_NAME
value: OKTA
- name: OAUTH2_PROXY_COOKIE_SECRET
value: s3cr3tK3yTh4tIsV3ryS3cur3AndR4nd0m==
- name: OAUTH2_PROXY_COOKIE_DOMAINS
value: localhost:4180
- name: OAUTH2_PROXY_COOKIE_SECURE
value: 'false'
- name: OAUTH2_PROXY_EMAIL_DOMAINS
value: '*'
- name: OAUTH2_PROXY_OIDC_ISSUER_URL
value: https://<your-okta-domain>/oauth2/default
- name: OAUTH2_PROXY_CLIENT_ID
value: <your-client-id>
- name: OAUTH2_PROXY_CLIENT_SECRET
value: <your-client-secret>
- name: OAUTH2_PROXY_UPSTREAMS
value: http://localhost:4000
- name: OAUTH2_PROXY_HTTP_ADDRESS
value: http://0.0.0.0:4180
- name: OAUTH2_PROXY_REVERSE_PROXY
value: 'true'
- name: OAUTH2_PROXY_SKIP_AUTH_ROUTES
value: ^/grafana.*Replace |
|
@dosu Do we need to delete the pod resources and recreate from scratch...? |
|
To resolve the issue of the basic auth username/password pop-up still appearing after adding
After making these changes, redeploy the Helm chart: helm upgrade devlake devlake/devlake --values values.yamlBy following these steps, you should be able to resolve the issue and ensure that the |
|
@dosu |
|
To troubleshoot why the
If you still do not see the |
|
Hi @dosu We are have forked https://github.com/apache/incubator-devlake repo and developed a new internal custom plugin. Thanks in advance. |
|
To generate a new DevLake image that includes your custom plugin and deploy it in your own cloud using a Helm chart, follow these steps:
This will deploy DevLake with your custom plugin included in the image. For more detailed instructions and additional configuration options, you can refer to the Apache DevLake Helm Setup documentation. Additional Configuration Options
For more detailed deployment scenarios and configurations, refer to the Helm Setup documentation [1][2][3][4]. |
|
Thank you @dosu Can you also send me details on how to generate a image just for plugin by itself and have the plugin image in helm values file. |
|
To generate an image just for a custom plugin and configure the Helm values file to use this plugin image, follow these steps:
This will configure the Helm chart to use your custom plugin image [1][2]. |
|
Hi @dosu here ca-cert is the SOPS secret for ca-cert.crt data. I am trying to use SSL integration for Azure MySQL. Error: |
Question
Hi currently we are trying to deploy devlake in our dev env. We are trying to integrate devlake using helm chart with managed GCP Cloud database service.
Our DB is SSL enabled and we don't see how to pass the SSL certs in the DB conn string part of helm chart.
Could you please let us know what additional params we need to use for using SSL in additional to below params we pass to helm command.
Screenshots
If applicable, add screenshots to help explain.
Additional context
Add any other context here.
The text was updated successfully, but these errors were encountered: