Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Add extraVolumes and extraVolumeMounts to all main containers #16361

Merged
merged 4 commits into from
Aug 22, 2021
Merged

feat: Add extraVolumes and extraVolumeMounts to all main containers #16361

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
cd7b76b
Add extraVolumes and extraVolumeMounts to all main containers
cccs-tom Aug 11, 2021
bc5ca68
Bump the chart version
cccs-tom Aug 19, 2021
f692612
Add examples in values.yaml
cccs-tom Aug 20, 2021
c876690
Use lists instead of maps for extraVolumes
cccs-tom Aug 20, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion helm/superset/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ maintainers:
- name: craig-rueda
email: craig@craigrueda.com
url: https://github.com/craig-rueda
version: 0.3.5
version: 0.3.6
dependencies:
- name: postgresql
version: 10.2.0
Expand Down
8 changes: 8 additions & 0 deletions helm/superset/templates/deployment-beat.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,10 @@ spec:
- name: superset-config
mountPath: {{ .Values.configMountPath | quote }}
readOnly: true
{{- range $volumeName, $mount := .Values.extraVolumeMounts }}
- name: {{ $volumeName }}
{{- tpl (toYaml $mount) $ | nindent 14 -}}
{{- end }}
resources:
{{ toYaml .Values.resources | indent 12 }}
{{- with .Values.nodeSelector }}
Expand All @@ -108,4 +112,8 @@ spec:
- name: superset-config
secret:
secretName: {{ tpl .Values.configFromSecret . }}
{{- range $volumeName, $volume := .Values.extraVolumes }}
- name: {{ $volumeName }}
{{- tpl (toYaml $volume) $ | nindent 10 -}}
{{- end }}
{{- end -}}
8 changes: 8 additions & 0 deletions helm/superset/templates/deployment-worker.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,10 @@ spec:
- name: superset-config
mountPath: {{ .Values.configMountPath | quote }}
readOnly: true
{{- range $volumeName, $mount := .Values.extraVolumeMounts }}
- name: {{ $volumeName }}
{{- tpl (toYaml $mount) $ | nindent 14 -}}
{{- end }}
resources:
{{ toYaml .Values.resources | indent 12 }}
{{- with .Values.nodeSelector }}
Expand All @@ -109,3 +113,7 @@ spec:
- name: superset-config
secret:
secretName: {{ tpl .Values.configFromSecret . }}
{{- range $volumeName, $volume := .Values.extraVolumes }}
- name: {{ $volumeName }}
{{- tpl (toYaml $volume) $ | nindent 10 -}}
{{- end }}
8 changes: 8 additions & 0 deletions helm/superset/templates/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,10 @@ spec:
mountPath: {{ .Values.extraConfigMountPath | quote }}
readOnly: true
{{- end }}
{{- range $volumeName, $mount := .Values.extraVolumeMounts }}
- name: {{ $volumeName }}
{{- tpl (toYaml $mount) $ | nindent 14 -}}
{{- end }}
ports:
- name: http
containerPort: {{ .Values.service.port }}
Expand Down Expand Up @@ -127,3 +131,7 @@ spec:
configMap:
name: {{ template "superset.fullname" . }}-extra-config
{{- end }}
{{- range $volumeName, $volume := .Values.extraVolumes }}
- name: {{ $volumeName }}
{{- tpl (toYaml $volume) $ | nindent 10 -}}
{{- end }}
8 changes: 8 additions & 0 deletions helm/superset/templates/init-job.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,10 @@ spec:
mountPath: {{ .Values.extraConfigMountPath | quote }}
readOnly: true
{{- end }}
{{- range $volumeName, $mount := .Values.extraVolumeMounts }}
- name: {{ $volumeName }}
{{- tpl (toYaml $mount) $ | nindent 12 -}}
{{- end }}
command: {{ tpl (toJson .Values.init.command) . }}
resources:
{{ toYaml .Values.init.resources | indent 10 }}
Expand All @@ -76,5 +80,9 @@ spec:
configMap:
name: {{ template "superset.fullname" . }}-extra-config
{{- end }}
{{- range $volumeName, $volume := .Values.extraVolumes }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just range over all of these and just call toyaml on them directly? That would allow users to customize extra volume options, etc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you suggesting changing this to a list instead of a map? The reason I implemented it this way is that my team manages multiple clusters (with one deployment of Superset per cluster). It is useful to have one "base configuration" with common values, which can then be appended to in the more specific values. Making this a list would force us to repeat this part of our configuration in multiple locations, which I was trying to avoid.

Having said that, I'm open to changing it. I'm just trying to better understand your comment:

That would allow users to customize extra volume options, etc.

With the way I implemented it, users are indeed free to specify whatever options they require. I've added a few extra options to the examples I committed in response to your previous comment. The only potential issue I can think of is that the volume names are slightly restricted by what Helm considers a valid key. Everything else will be used as-is by this line (85):

{{- tpl (toYaml $volume) $ | nindent 10 -}}

Am I misunderstanding your comment?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes total sense. I just think that most folks will run into a spot where they need an extra option for the volume, and will need to add a PR to get the additional config they need shoved in.

Something like this is what I was thinking:
https://github.com/Kong/charts/blob/main/charts/kong/values.yaml#L35

It really allows for the most flexibility possible.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. So using the example you linked to, that would turn into:

extraVolumes:
  volumeName:
    emptyDir: {}
extraVolumeMounts:
  volumeName:
    mountPath: "/opt/user/dir/mount"

The syntax is a little different, but I believe the flexibility is the same, is it not? With the added bonus that additional volumes can be appended without overriding the entire list (as per the use case I laid out in my previous reply).

Copy link
Member

@craig-rueda craig-rueda Aug 20, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, it would look like:

extraVolumes:
  - name: "cool-volume"
    emptyDir: {}

extraVolumeMounts:
 - name: "cool-volume"
   mountPath: "/opt/user/dir/mount"

This allows us to add things like defaultMode, etc. to the volume definitions. We can also mount from secrets, or configMaps without changing this chart.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I meant was that, if you modify the example in the way I wrote it in my reply, you would get the exact same YAML output as if I made the changes you requested.
My intent was to allow maximum flexibility. As far as I know, this implementation allows 100% of the volumes and volumeMounts spec. Could you provide an example of something that isn't supported with this implementation to help me better understand, please?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should use a list rather than a map here. Nested maps are a little tricky to follow in this context. The volumes and volumeMounts stanzas in Deployments, etc. use lists, so it makes the transition pretty straight forward for end users to just drop their list into this chart's values override.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've just pushed this change.

- name: {{ $volumeName }}
{{- tpl (toYaml $volume) $ | nindent 10 -}}
{{- end }}
restartPolicy: Never
{{- end }}
4 changes: 4 additions & 0 deletions helm/superset/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,10 @@ extraConfigs: {}

extraSecrets: {}

extraVolumes: {}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add an example of each so people know how to use these

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


extraVolumeMounts: {}

# A dictionary of overrides to append at the end of superset_config.py - the name does not matter
# WARNING: the order is not guaranteed
configOverrides: {}
Expand Down