Skip to content

Commit

Permalink
fix valkey url overrides (#119)
Browse files Browse the repository at this point in the history
  • Loading branch information
rarmatei committed May 24, 2024
1 parent ba260c2 commit 4ff427c
Show file tree
Hide file tree
Showing 4 changed files with 48 additions and 22 deletions.
65 changes: 44 additions & 21 deletions AGENTS-GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ We recommend deploying Nx Agents onto a new cluster, but you can also deploy ont
Valkey is an in-memory key-value store (like Redis) that is used by the workflow controller to hold temporary state.

1. Create a secret similar to `agents-guide/agents-secrets.yml` and set your valkey password in there.
- Important: the key `valkey-password` itself shouldn't be changed, only its value
- Important: the key `valkey-password` itself shouldn't be changed, only its value
- You do not need to set the secret values for the S3 bucket yet. They are there as an example. Please refer to `charts/nx-agents/values.yaml` for info on what Agent storage options we support.
2. Apply the secret: `kubectl apply -f agents-secrets.yml`
3. Now let's deploy Valkey:
Expand All @@ -18,15 +18,38 @@ Valkey is an in-memory key-value store (like Redis) that is used by the workflow

### Deploy the Agents chart onto your cluster

Modify you `nx-agents.yml` values file, and make sure the secrets we created above are linked up:
1. Ensure `secret.name: nx-cloud-agents-secret` (see [here](https://github.com/nrwl/nx-cloud-helm/blob/main/charts/nx-agents/values.yaml#L132))
2. Ensure `secret.valkeyPassword: 'valkey-password'`. The name needs to match the exact key you declared in the secret above (example [here](https://github.com/nrwl/nx-cloud-helm/blob/main/charts/nx-agents/values.yaml#L132)).

Now you can push your chart changes so your controller can connect to valkey:

```bash
helm repo add nx-cloud https://nrwl.github.io/nx-cloud-helm
helm repo update nx-cloud
helm upgrade --install nx-agents nx-cloud/nx-agents --values=nx-agents.yml
```

Use the values file in `charts/nx-agents/values.yaml` as an example.
Again, you can use the values file in `charts/nx-agents/values.yaml` as an example.

###### Custom valkey URL

Note on storage:
If you have deployed valkey in a custom location you can overwrite the default url:

```yaml
controller:
useDefaultValkeyAddress: false # set this to false
deployment:
port: 9000
env:
- name: VALKEY_CONNECTION_STRING # declare the custom connection string
valueFrom: # you can insert the value from a secret or hardcode it in the nx-agents.yml
secretKeyRef:
name: nx-cloud-k8s-secret
key: valkey-connection-string
```
---
Note on storage:
1. The Agents need a storage bucket for storing logs as well as cached folders (such as `node_modules`)
2. You do not need to use S3, we also support Azure Blob Storage and GCloud buckets

Expand All @@ -41,24 +64,24 @@ Set these in your NxCloud `values.yaml` file:

```yaml
nxCloudWorkflows:
enabled: true
port: 9000
name: 'nx-cloud-workflow-controller-service'
workflowsNamespace: 'nx-cloud-workflows'

# If externalName is left unset, the applications will look for ane existing service with the name defined
# by `nxCloudWorkflows.name` in the namespace `nxCloudWorkflows.workflowsNamespace`. Use this option if you are
# also running the nx-agents chart in the same cluster as this nx-cloud chart
#
# If externalName is set, an additional service will be created with the name `nxCloudWorkflows.name`
# in the global namespace of this chart, and applications will use that service to connect to the workflow controller.
# Use this option if your nx-agents are running in a different cluster than this nx-cloud chart
externalName: ''

# If you find that an externalName service is not working as expected, you can set this to true to create a headless service
# which will create an endpoint group as an alternative. Please continue to set `externalName` to the IP address
# you wish to direct traffic to as we will use it to populate the endpoint slice.
headless: false
enabled: true
port: 9000
name: 'nx-cloud-workflow-controller-service'
workflowsNamespace: 'nx-cloud-workflows'

# If externalName is left unset, the applications will look for ane existing service with the name defined
# by `nxCloudWorkflows.name` in the namespace `nxCloudWorkflows.workflowsNamespace`. Use this option if you are
# also running the nx-agents chart in the same cluster as this nx-cloud chart
#
# If externalName is set, an additional service will be created with the name `nxCloudWorkflows.name`
# in the global namespace of this chart, and applications will use that service to connect to the workflow controller.
# Use this option if your nx-agents are running in a different cluster than this nx-cloud chart
externalName: ''

# If you find that an externalName service is not working as expected, you can set this to true to create a headless service
# which will create an endpoint group as an alternative. Please continue to set `externalName` to the IP address
# you wish to direct traffic to as we will use it to populate the endpoint slice.
headless: false
```

Please see `charts/nx-cloud/values.yaml` for up to date docs on the above options.
Expand Down
2 changes: 1 addition & 1 deletion charts/nx-agents/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ apiVersion: v2
name: nx-agents
description: Nx Cloud Agents Helm Chart
type: application
version: 1.2.1
version: 1.2.2
maintainers:
- name: nx
url: "https://nx.app/"
Expand Down
2 changes: 2 additions & 0 deletions charts/nx-agents/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -91,8 +91,10 @@ spec:
{{- toYaml .Values.controller.deployment.envFrom | nindent 12}}
{{- end }}
env:
{{- if .Values.controller.useDefaultValkeyAddress }}
- name: VALKEY_CONNECTION_STRING
value: valkey://valkey-master.{{ .Values.global.namespace }}.svc.cluster.local:6379
{{- end }}
{{- if gt (len .Values.controller.deployment.env) 0 }}
{{- toYaml .Values.controller.deployment.env | nindent 12 }}
{{- end }}
Expand Down
1 change: 1 addition & 0 deletions charts/nx-agents/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ controller:
pullPolicy: Always
# These are our recommended resources for the controller. However we have noted that a request as low as 0.2 CPU and 0.5Gi
# memory can work for many cases
useDefaultValkeyAddress: true
resources:
limits:
memory: '1Gi'
Expand Down

0 comments on commit 4ff427c

Please sign in to comment.