Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix valkey url overrides #119

Merged
merged 1 commit into from
May 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 44 additions & 21 deletions AGENTS-GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ We recommend deploying Nx Agents onto a new cluster, but you can also deploy ont
Valkey is an in-memory key-value store (like Redis) that is used by the workflow controller to hold temporary state.

1. Create a secret similar to `agents-guide/agents-secrets.yml` and set your valkey password in there.
- Important: the key `valkey-password` itself shouldn't be changed, only its value
- Important: the key `valkey-password` itself shouldn't be changed, only its value
- You do not need to set the secret values for the S3 bucket yet. They are there as an example. Please refer to `charts/nx-agents/values.yaml` for info on what Agent storage options we support.
2. Apply the secret: `kubectl apply -f agents-secrets.yml`
3. Now let's deploy Valkey:
Expand All @@ -18,15 +18,38 @@ Valkey is an in-memory key-value store (like Redis) that is used by the workflow

### Deploy the Agents chart onto your cluster

Modify you `nx-agents.yml` values file, and make sure the secrets we created above are linked up:
1. Ensure `secret.name: nx-cloud-agents-secret` (see [here](https://github.com/nrwl/nx-cloud-helm/blob/main/charts/nx-agents/values.yaml#L132))
2. Ensure `secret.valkeyPassword: 'valkey-password'`. The name needs to match the exact key you declared in the secret above (example [here](https://github.com/nrwl/nx-cloud-helm/blob/main/charts/nx-agents/values.yaml#L132)).

Now you can push your chart changes so your controller can connect to valkey:

```bash
helm repo add nx-cloud https://nrwl.github.io/nx-cloud-helm
helm repo update nx-cloud
helm upgrade --install nx-agents nx-cloud/nx-agents --values=nx-agents.yml
```

Use the values file in `charts/nx-agents/values.yaml` as an example.
Again, you can use the values file in `charts/nx-agents/values.yaml` as an example.

###### Custom valkey URL

Note on storage:
If you have deployed valkey in a custom location you can overwrite the default url:

```yaml
controller:
useDefaultValkeyAddress: false # set this to false
deployment:
port: 9000
env:
- name: VALKEY_CONNECTION_STRING # declare the custom connection string
valueFrom: # you can insert the value from a secret or hardcode it in the nx-agents.yml
secretKeyRef:
name: nx-cloud-k8s-secret
key: valkey-connection-string
```
---
Note on storage:
1. The Agents need a storage bucket for storing logs as well as cached folders (such as `node_modules`)
2. You do not need to use S3, we also support Azure Blob Storage and GCloud buckets

Expand All @@ -41,24 +64,24 @@ Set these in your NxCloud `values.yaml` file:

```yaml
nxCloudWorkflows:
enabled: true
port: 9000
name: 'nx-cloud-workflow-controller-service'
workflowsNamespace: 'nx-cloud-workflows'

# If externalName is left unset, the applications will look for ane existing service with the name defined
# by `nxCloudWorkflows.name` in the namespace `nxCloudWorkflows.workflowsNamespace`. Use this option if you are
# also running the nx-agents chart in the same cluster as this nx-cloud chart
#
# If externalName is set, an additional service will be created with the name `nxCloudWorkflows.name`
# in the global namespace of this chart, and applications will use that service to connect to the workflow controller.
# Use this option if your nx-agents are running in a different cluster than this nx-cloud chart
externalName: ''

# If you find that an externalName service is not working as expected, you can set this to true to create a headless service
# which will create an endpoint group as an alternative. Please continue to set `externalName` to the IP address
# you wish to direct traffic to as we will use it to populate the endpoint slice.
headless: false
enabled: true
port: 9000
name: 'nx-cloud-workflow-controller-service'
workflowsNamespace: 'nx-cloud-workflows'

# If externalName is left unset, the applications will look for ane existing service with the name defined
# by `nxCloudWorkflows.name` in the namespace `nxCloudWorkflows.workflowsNamespace`. Use this option if you are
# also running the nx-agents chart in the same cluster as this nx-cloud chart
#
# If externalName is set, an additional service will be created with the name `nxCloudWorkflows.name`
# in the global namespace of this chart, and applications will use that service to connect to the workflow controller.
# Use this option if your nx-agents are running in a different cluster than this nx-cloud chart
externalName: ''

# If you find that an externalName service is not working as expected, you can set this to true to create a headless service
# which will create an endpoint group as an alternative. Please continue to set `externalName` to the IP address
# you wish to direct traffic to as we will use it to populate the endpoint slice.
headless: false
```

Please see `charts/nx-cloud/values.yaml` for up to date docs on the above options.
Expand Down
2 changes: 1 addition & 1 deletion charts/nx-agents/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ apiVersion: v2
name: nx-agents
description: Nx Cloud Agents Helm Chart
type: application
version: 1.2.1
version: 1.2.2
maintainers:
- name: nx
url: "https://nx.app/"
Expand Down
2 changes: 2 additions & 0 deletions charts/nx-agents/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -91,8 +91,10 @@ spec:
{{- toYaml .Values.controller.deployment.envFrom | nindent 12}}
{{- end }}
env:
{{- if .Values.controller.useDefaultValkeyAddress }}
- name: VALKEY_CONNECTION_STRING
value: valkey://valkey-master.{{ .Values.global.namespace }}.svc.cluster.local:6379
{{- end }}
{{- if gt (len .Values.controller.deployment.env) 0 }}
{{- toYaml .Values.controller.deployment.env | nindent 12 }}
{{- end }}
Expand Down
1 change: 1 addition & 0 deletions charts/nx-agents/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ controller:
pullPolicy: Always
# These are our recommended resources for the controller. However we have noted that a request as low as 0.2 CPU and 0.5Gi
# memory can work for many cases
useDefaultValkeyAddress: true
resources:
limits:
memory: '1Gi'
Expand Down
Loading