improvement: split manifests for dev and release

[Shabirmean]

Description:

• The k8s manifests currently point to a concrete image URL for the containers.
• As a result even when doing local development the freshly built images are not deployed to the cluster
• Instead what is deployed is the publicly available image that is referenced in the manifest file
• We need to fix this by providing separate manifest files for dev and release

Changes:

• Create a new directory for k8 manifests (The old directory (kubernetes-manifests) is kept as is since there can be other documents referencing it. So we will gradually remove it once we have the rest of the setup clean)
• Place the common manifests in the root directory and move the manifests that need to differ based on environments to their own specific environment directories
• Update the skaffold.yaml file to have profiles for the different scenarios
• Add readme to the new manifests directory explaining each

Related Issues:

• Remove the kubernetes-manifests directory #96

[snippet-bot]

Here is the summary of changes.

You are about to add 16 region tags.

• point-of-sale-app/k8-manifests/api-server-lb.yaml:15, tag anthosbaremetal_k8_manifests_api_server_lb_service_api_server_lb
• point-of-sale-app/k8-manifests/dev/api-server.yaml:15, tag anthosbaremetal_k8_manifests_dev_api_server_deployment_api_server
• point-of-sale-app/k8-manifests/dev/api-server.yaml:68, tag anthosbaremetal_k8_manifests_dev_api_server_service_api_server_svc
• point-of-sale-app/k8-manifests/dev/inventory.yaml:15, tag anthosbaremetal_k8_manifests_dev_inventory_deployment_inventory
• point-of-sale-app/k8-manifests/dev/inventory.yaml:73, tag anthosbaremetal_k8_manifests_dev_inventory_service_inventory_svc
• point-of-sale-app/k8-manifests/dev/payments.yaml:15, tag anthosbaremetal_k8_manifests_dev_payments_deployment_payments
• point-of-sale-app/k8-manifests/dev/payments.yaml:57, tag anthosbaremetal_k8_manifests_dev_payments_service_payments_svc
• point-of-sale-app/k8-manifests/release/api-server.yaml:15, tag anthosbaremetal_k8_manifests_release_api_server_deployment_api_server
• point-of-sale-app/k8-manifests/release/api-server.yaml:69, tag anthosbaremetal_k8_manifests_release_api_server_service_api_server_svc
• point-of-sale-app/k8-manifests/release/inventory.yaml:15, tag anthosbaremetal_k8_manifests_release_inventory_deployment_inventory
• point-of-sale-app/k8-manifests/release/inventory.yaml:74, tag anthosbaremetal_k8_manifests_release_inventory_service_inventory_svc
• point-of-sale-app/k8-manifests/release/payments.yaml:15, tag anthosbaremetal_k8_manifests_release_payments_deployment_payments
• point-of-sale-app/k8-manifests/release/payments.yaml:58, tag anthosbaremetal_k8_manifests_release_payments_service_payments_svc
• point-of-sale-app/k8-manifests/service-configs.yaml:15, tag anthosbaremetal_k8_manifests_service_configs_configmap_service_configs
• point-of-sale-app/skaffold.yaml:15, tag anthosbaremetal_point_of_sale_app_skaffold_config_setup
• point-of-sale-app/skaffold.yaml:65, tag anthosbaremetal_point_of_sale_app_skaffold_config_load-balancer

---

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

• Refresh this comment

[xtineskim]

region tags LGTM! may want to wait for others to stamp it before merging :)

[bourgeoisor]

It's not obvious at a glance the difference between k8s-manifests and kubernetes-manifests before looking at them and opening various resources. Additionally, it looks like you have 2 or 3 different manifests to update every time you make a change, instead of having a single source of truth. This can be error-prone.

In Bank of Anthos (and soon Online Boutique as well) we have only two directories:

• dev-kubernetes-manifests (this is the latest dev / source of truth)
• kubernetes-manifests (this is read only and is basically just a snapshot of dev-kuberentes-manifests from the last time the make-release.sh ran

That means that users uses kubernetes-manifests, developers uses dev-kubernetes-manifests (and modify only that), and once a new release is ready, when they run make-release.sh it snapshots dev-kubernetes-manifests into kubernetes-manifests to reflect that new versioning

https://github.com/GoogleCloudPlatform/bank-of-anthos/blob/main/release/make-release.sh#L46-L55

Thoughts on doing something similar?

[Shabirmean]

@bourgeoisor Thank you very much for reviewing this fast.

It's not obvious at a glance the difference between k8s-manifests and kubernetes-manifests before looking at them and opening various resources.

Like explained in the Description k8s-manifests is what will stand. kubernetes-manifests will be deleted (see linked issues: #96). I am not deleting it yet because kubernetes-manifests might be referenced elsewhere and want to handle it separately.

Additionally, it looks like you have 2 or 3 different manifests to update every time you make a change, instead of having a single source of truth. This can be error-prone.

Looking at BoA, I see that it is still having multiple yamls in both directories. In any-case, I think it's a matter of keeping the resources nearby and having a consistent way of updating the files via the automation we choose. Scan and update one yaml or scan and update multiple yamls in the same directory. Personally I feel having the same look and feel across different environment helps identify diffs for an outsider.

In Bank of Anthos (and soon Online Boutique as well) we have only two directories:

• dev-kubernetes-manifests (this is the latest dev / source of truth)
• kubernetes-manifests (this is read only and is basically just a snapshot of dev-kuberentes-manifests from the last time the make-release.sh ran

IMHO, this is not straight-forward and evident for someone new to the repository. For a noogler onboarding they wouldn't recognized it easily if we didn't explain it or they read the readme. Same goes for the external contributors. I understand that this was a structure decided upon when the repos were created but I think we can adopt a better model.

As already explained above, the root will only have one directory k8-manifests. kuberenetes-manifests will be deleted. So for someone visiting the repo they have one destination to find manifest files: k8-manifests. When they are in it is clearly evident there are two different types of manifests: dev and release. Also the common ones are there flat within that repo. I think this model consolidate the yamls into one directory from the root and narrows down the context for users with minimal need for a README.

That means that users uses kubernetes-manifests, developers uses dev-kubernetes-manifests (and modify only that), and once a new release is ready, when they run make-release.sh it snapshots dev-kubernetes-manifests into kubernetes-manifests to reflect that new versioning

https://github.com/GoogleCloudPlatform/bank-of-anthos/blob/main/release/make-release.sh#L46-L55

I think the point of discussion is the structure and what makes it straight-forward. The rest as to how people will consume it will definitely require README guidance in either case.

Let me know if the above thought arguments I had when planning this structure makes sense.

[bourgeoisor]

I missed the bit where the kubernetes-manifests directory was going away, that clears things!

As for the way the release snapshot is generated, is that intended to be a manual process? Should we have that be automated or documented in some form in the "making a new release" processes? (The less error-prone method would be an automated part of the release making script, but in either way that should be documented somewhere I think)

[Shabirmean]

I missed the bit where the kubernetes-manifests directory was going away, that clears things!

As for the way the release snapshot is generated, is that intended to be a manual process? Should we have that be automated or documented in some form in the "making a new release" processes? (The less error-prone method would be an automated part of the release making script, but in either way that should be documented somewhere I think)

Sorry about the confusion on the kubernetes-manifests directory. I think there was a slight delay between when I opened the PR and when I added that detail. It may be that you had already loaded the page before that.

Yes, the release process should be automated and definitely documented for others to execute it. However, it is still in the works. So as I go adding these improvements I will also open PRs for the release procedure and supported documentation. I don't have it ready just yet!

[bourgeoisor]

LGTM!

Caveats for merge:

• I have not thoroughly tested these changes
• Documentation on these changes + automation of the release manifests generation incoming in a later PR
• This is not breaking anyone's workflow without letting them know

[Shabirmean]

@bourgeoisor - I made one additional commit (a96ec35) which changed the newly added manifests to point to the CI project in runcible.

Also for re: not tested, changes 1 and 2 is making sure that the new manifests are used to deploy the PR cluster. So if it passes then we should be good. I have tested it locally too.

[bourgeoisor]

LGTM