Release helm chart #3312

mforutan · 2020-12-06T04:29:08Z

Using GitHub Actions to release the official helm chart

This PR is a starting point to introduce an official helm chart for locakstack,

I copied the helm chart from sidhuko/charts#1 without any modification (hope it is fine with @sidhuko)

There are couple of tasks that need to be done before this can be merged:

A gh-pages branch need to be created
GitHub Pages need to be enabled in the setting

TODO (can be done in this PR or next PRs):

A GitHub workflow to test the helm chart for PRs

Fixes #882

coveralls · 2020-12-06T05:03:28Z

Coverage remained the same at 77.111% when pulling da208f9 on mforutan:add-helm-chart into 9a4f516 on localstack:master.

macnev2013 · 2020-12-06T17:47:26Z

chart/localstack/templates/deployment.yaml

+          {{- end }}
+          livenessProbe:
+            httpGet:
+              path: /


Instead of this can we use something like /health for health check path?

macnev2013 · 2020-12-06T17:48:02Z

chart/localstack/templates/deployment.yaml

+              port: http
+          readinessProbe:
+            httpGet:
+              path: /


Instead of this can we use something like /health for health check path?

macnev2013 · 2020-12-06T17:58:48Z

chart/localstack/values-production.yaml

+    - name: apigateway
+      enabled: false
+      endpoint: ""
+      internalPort: 4567
+      externalPort: 4567
+      nodePort:  31567
+    - name: cloudformation
+      enabled: false
+      endpoint: ""
+      internalPort: 4581
+      externalPort: 4581
+      nodePort: 31581
+    - name: dynamodb
+      enabled: false
+      endpoint: ""
+      internalPort: 4569
+      externalPort: 4569
+      nodePort:  31569
+    - name: elasticsearch
+      enabled: false
+      endpoint: ""
+      internalPort: 4571
+      externalPort: 4571
+      nodePort: 31571
+    - name: kinesis
+      enabled: false
+      endpoint: ""
+      internalPort: 4568
+      externalPort: 4568
+      nodePort: 31568
+    - name: s3
+      enabled: false
+      endpoint: ""
+      internalPort: 4572
+      externalPort: 4572
+      nodePort: 31572
+    - name: sns
+      enabled: false
+      endpoint: ""
+      internalPort: 4575
+      externalPort: 4575
+      nodePort: 31575
+    - name: sqs
+      enabled: false
+      endpoint: ""
+      internalPort: 4576
+      externalPort: 4576
+      nodePort: 31576
+    - name: ssm
+      enabled: false
+      internalPort: 4583
+      externalPort: 4583
+      nodePort: 31583
+    - name: dynamodbstreams
+      enabled: false
+      internalPort: 4570
+      externalPort: 4570
+      nodePort:  31570
+    - name: firehose
+      enabled: false
+      internalPort: 4573
+      externalPort: 4573
+      nodePort: 31573
+    - name: lambda
+      enabled: false
+      internalPort: 4574
+      externalPort: 4574
+      nodePort:  31574
+    - name: redshift
+      enabled: false
+      internalPort: 4577
+      externalPort: 4577
+      nodePort: 31577
+    - name: es
+      enabled: false
+      internalPort: 4578
+      externalPort: 4578
+      nodePort: 31578
+    - name: ses
+      enabled: false
+      internalPort: 4579
+      externalPort: 4579
+      nodePort: 31579
+    - name: route53
+      enabled: false
+      internalPort: 4580
+      externalPort: 4580
+      nodePort:  31580
+    - name: cloudwatch
+      enabled: false
+      internalPort: 4582
+      externalPort: 4582
+      nodePort: 31582
+    - name: secretsmanager
+      enabled: false
+      internalPort: 4583
+      externalPort: 4583
+      nodePort: 31583


After version 0.11.5 of localstack we are using edge port 4566 to access all the services. So we just need to expose port 4566 and 4571. We can remove the rest.

How about setting <SERVICE>_PORT_EXTERNAL or is this not required anymore?

Yes, as per my knowledge it is deprecated. We are using single entrypoint for all the services (by default 4566) and 4571.

Ref. https://github.com/localstack/localstack#announcements
Ref. docker-compose.yaml

macnev2013 · 2020-12-06T18:06:45Z

chart/localstack/values.yaml

+    - name: apigateway
+      enabled: false
+      endpoint: ""
+      internalPort: 4567
+      externalPort: 4567
+      nodePort:  31567
+    - name: cloudformation
+      enabled: false
+      endpoint: ""
+      internalPort: 4581
+      externalPort: 4581
+      nodePort: 31581
+    - name: dynamodb
+      enabled: false
+      endpoint: ""
+      internalPort: 4569
+      externalPort: 4569
+      nodePort:  31569
+    - name: elasticsearch
+      enabled: false
+      endpoint: ""
+      internalPort: 4571
+      externalPort: 4571
+      nodePort: 31571
+    - name: kinesis
+      enabled: false
+      endpoint: ""
+      internalPort: 4568
+      externalPort: 4568
+      nodePort: 31568
+    - name: s3
+      enabled: false
+      endpoint: ""
+      internalPort: 4572
+      externalPort: 4572
+      nodePort: 31572
+    - name: sns
+      enabled: false
+      endpoint: ""
+      internalPort: 4575
+      externalPort: 4575
+      nodePort: 31575
+    - name: sqs
+      enabled: false
+      endpoint: ""
+      internalPort: 4576
+      externalPort: 4576
+      nodePort: 31576
+    - name: ssm
+      enabled: false
+      internalPort: 4583
+      externalPort: 4583
+      nodePort: 31583
+    - name: dynamodbstreams
+      enabled: false
+      internalPort: 4570
+      externalPort: 4570
+      nodePort:  31570
+    - name: firehose
+      enabled: false
+      internalPort: 4573
+      externalPort: 4573
+      nodePort: 31573
+    - name: lambda
+      enabled: false
+      internalPort: 4574
+      externalPort: 4574
+      nodePort:  31574
+    - name: redshift
+      enabled: false
+      internalPort: 4577
+      externalPort: 4577
+      nodePort: 31577
+    - name: es
+      enabled: false
+      internalPort: 4578
+      externalPort: 4578
+      nodePort: 31578
+    - name: ses
+      enabled: false
+      internalPort: 4579
+      externalPort: 4579
+      nodePort: 31579
+    - name: route53
+      enabled: false
+      internalPort: 4580
+      externalPort: 4580
+      nodePort:  31580
+    - name: cloudwatch
+      enabled: false
+      internalPort: 4582
+      externalPort: 4582
+      nodePort: 31582
+    - name: secretsmanager
+      enabled: false
+      internalPort: 4583
+      externalPort: 4583
+      nodePort: 31583


We can remove this except 4566 and 4571.

sidhuko · 2020-12-06T23:39:45Z

Sorry, been a busy week. I've created a new branch (localstack-chart) to submit as a helm chart on their repository but it needs some work. I am unsure about contributors comments above as it seems we still have <SERVICE>_PORT_EXTERNAL and <SERVICE>_BACKEND. I am not sure if both are still used? I've supported <SERVICE>_BACKEND which can be specified by passing an key value pair under services of the new configuration in new branch. If neither are support or will be deprecated it will make it easier. Otherwise I can still expose multiple ports by reverting to what we have here.

We will also need to expose the new environment variables as configuration in the deployment and values file.

macnev2013 · 2020-12-07T05:07:45Z

Sorry, been a busy week. I've created a new branch (localstack-chart) to submit as a helm chart on their repository but it needs some work. I am unsure about contributors comments above as it seems we still have <SERVICE>_PORT_EXTERNAL and <SERVICE>_BACKEND. I am not sure if both are still used? I've supported <SERVICE>_BACKEND which can be specified by passing an key value pair under services of the new configuration in new branch. If neither are support or will be deprecated it will make it easier. Otherwise I can still expose multiple ports by reverting to what we have here.

Hey @sidhuko, As far as I know we don't use it anymore. @whummer can you confirm it?

macnev2013 · 2020-12-07T05:10:10Z

We will also need to expose the new environment variables as configuration in the deployment and values file.

Yes @sidhuko, we need to add new variables as well. For the starting point maybe we can start importing variables present in the compose file.

Ref. docker-compose.yaml

mforutan · 2020-12-07T05:40:27Z

@macnev2013 @sidhuko, what do you think if we change the chart's version to 0.1.0-beta.0 and merge this PR to have the process of releasing the chart in place then we can fix the rest of issues and release the chart in next PRs, my aim here is to keep these two steps separated so everyone can contribute individually.

whummer · 2020-12-07T07:43:39Z

Thanks @mforutan for creating the initial chart version 👍 and thanks @macnev2013 @sidhuko for reviewing! Agreed that we can merge this initial version as-is, we'll make a couple of changes and enhancements (e.g., removing the legacy service ports) as we go.. Thanks

There are couple of tasks that need to be done before this can be merged:
A gh-pages branch need to be created
GitHub Pages need to be enabled in the setting

Github Pages has been enabled for this repo (currently on master branch) - wondering if we need to move the chart to a localstack.github.io repo to enable hosting of the Githube page..?

mforutan · 2020-12-07T07:53:07Z

hi @whummer, thank you for merging this PR but it failed to deploy the chart as the gh-pages branch wasn't created, can you please do these steps to get the chart deployed?

Remove the new tag: https://github.com/localstack/localstack/releases/tag/localstack-0.1.0
Create gh-pages branch
Enable GitHub pages in the setting
Re-run the GitHub Actions: https://github.com/localstack/localstack/actions/runs/405281754

whummer · 2020-12-07T08:02:19Z

Thanks @mforutan . Actually, I think we may need to move the chart to a separate repo in that case, or do you know if there is a way to prevent the helm/chart-releaser-action@v1.1.0 Github action plugin from creating tags (since we're already using tags to create LocalStack code releases)?

mforutan · 2020-12-07T08:22:47Z

helm/chart-releaser uses GitHub release to store the chart and also to compare it with the existing versions,

If you are able to create a new repo, I am more than happy to create a new PR against that or we may need to see how to ignore this new tags in code releases.

JamesArthurHolland · 2020-12-07T15:44:43Z

chart/localstack/templates/service.yaml

+      protocol: TCP
+      port: {{ $val.externalPort }}
+      targetPort: {{ $val.internalPort }}
+      {{- if contains "NodePort" .Values.service.type }}


This creates a scope error:
Error: template: localstack/templates/service.yaml:28:40: executing "localstack/templates/service.yaml" at <.Values.service.type>: nil pointer evaluating interface {}.service

I think it should be
$.Values.service.type

sidhuko · 2020-12-07T16:01:25Z

I cannot say as not a contributor here. It's early Monday morning for me. I will hopefully complete by end of day today and get the PR across to helm/charts. They are quite responsive. Merge in couple days as long as no significant changes are needed. I will leave notes here so you can try install locally and test for me. The chart will be better in helm/charts then here for continued support though.

…

On Sun, 6 Dec 2020, 10:40 pm Mohammad Forutan, ***@***.***> wrote: @macnev2013 <https://github.com/macnev2013> @sidhuko <https://github.com/sidhuko>, what do you think if we change the chart's version to 0.1.0-beta.0 and merge this PR to have the process of releasing the chart in place then we can fix the rest of issues and release the chart in next PRs, my aim here is to keep these two steps separated so everyone can contribute individually. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3312 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAVFBWMRJJQXZS6BT6F4FALSTRTFRANCNFSM4UPBHNHA> .

whummer · 2020-12-07T16:21:09Z

Thanks all for your inputs - we'll first migrate the chart into a separate repo later today, then we can continue making the incremental changes. Will share the details here shortly. Thanks

whummer · 2020-12-07T18:24:23Z

@mforutan @sidhuko @macnev2013 @JamesArthurHolland Moved the chart into this new repo: https://github.com/localstack/helm-charts . We'll create the gh-pages branch and the Github actions build shortly.. Thanks

mforutan · 2020-12-07T22:56:36Z

@whummer Awesome!

mforutan added 8 commits December 6, 2020 13:08

Initial version

55dfc32

change to push

55b44d6

change chart folder

4683b68

Update chart-release.yaml

bfb609f

fix github action triggers

d968b0f

Update README.md

3e5233a

Update README.md

3e4de65

fix indentation

da208f9

mforutan mentioned this pull request Dec 6, 2020

Official helm chart 🎉 #882

Closed

macnev2013 requested changes Dec 6, 2020

View reviewed changes

whummer merged commit 4f14113 into localstack:master Dec 7, 2020

JamesArthurHolland reviewed Dec 7, 2020

View reviewed changes

whummer pushed a commit that referenced this pull request Dec 7, 2020

release initial version of Helm chart (#3312)

fadaddc

mforutan deleted the add-helm-chart branch December 7, 2020 22:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release helm chart #3312

Release helm chart #3312

mforutan commented Dec 6, 2020

coveralls commented Dec 6, 2020

macnev2013 Dec 6, 2020

macnev2013 Dec 6, 2020

macnev2013 Dec 6, 2020

sidhuko Dec 6, 2020 •

edited

macnev2013 Dec 7, 2020

macnev2013 Dec 6, 2020

sidhuko commented Dec 6, 2020

macnev2013 commented Dec 7, 2020

macnev2013 commented Dec 7, 2020

mforutan commented Dec 7, 2020

whummer commented Dec 7, 2020 •

edited

mforutan commented Dec 7, 2020

whummer commented Dec 7, 2020

mforutan commented Dec 7, 2020

JamesArthurHolland Dec 7, 2020

sidhuko commented Dec 7, 2020 via email

whummer commented Dec 7, 2020

whummer commented Dec 7, 2020

mforutan commented Dec 7, 2020

Release helm chart #3312

Release helm chart #3312

Conversation

mforutan commented Dec 6, 2020

coveralls commented Dec 6, 2020

macnev2013 Dec 6, 2020

Choose a reason for hiding this comment

macnev2013 Dec 6, 2020

Choose a reason for hiding this comment

macnev2013 Dec 6, 2020

Choose a reason for hiding this comment

sidhuko Dec 6, 2020 • edited

Choose a reason for hiding this comment

macnev2013 Dec 7, 2020

Choose a reason for hiding this comment

macnev2013 Dec 6, 2020

Choose a reason for hiding this comment

sidhuko commented Dec 6, 2020

macnev2013 commented Dec 7, 2020

macnev2013 commented Dec 7, 2020

mforutan commented Dec 7, 2020

whummer commented Dec 7, 2020 • edited

mforutan commented Dec 7, 2020

whummer commented Dec 7, 2020

mforutan commented Dec 7, 2020

JamesArthurHolland Dec 7, 2020

Choose a reason for hiding this comment

sidhuko commented Dec 7, 2020 via email

whummer commented Dec 7, 2020

whummer commented Dec 7, 2020

mforutan commented Dec 7, 2020

sidhuko Dec 6, 2020 •

edited

whummer commented Dec 7, 2020 •

edited