[Feature][installation] Make .env file readonly to fix the encKey ENCODE_KEY complication once and for all

[klesh]

Search before asking

• I had searched in the issues and found no similar feature requirement.

Use case

As a user, I would like to upgrade the devlake smoothly without losing my data.

Description

Why do we need it?

The encKey or ENCODE_KEY was introduced a long time ago to encrypt sensitive information (e.g. access token, password, etc) stored in the database.

What went wrong?

We decided to generate the encryption key and save it to the .env file automatically on the first boot if it was omitted in order to make the installation process smoother.

However, it has been proven problematic along the way:

1. bad naming
2. the devlake container must be deployed as a stateful set, since we have to write the key back to the .env file
3. users often do not aware of the key and run into encKey after upgrading if they chose to deploy a completely new instance with the existing database

What can we do about it?

Users should be properly informed about the encryption key, and how should they do about it.

Plan of attack

1. Unify encKey ENCODE_KEY to encryptionSecret / ENCRYPTION_SECRET both codebase and documentation
2. Make .env file Readonly and Optional in codebase: devlake backend could still read setting from dotenv file IFF it exists
3. Remove the auto-generate logic from our code base, devlake should fail if no ENCRYPTION_SECRET is provided
4. v0.18 Installation Guide to ask users to generate the ENCRYPTION_SECRET manually with some example command, and ask them to store it properly if they want to deploy a brand new instance using the existing database.
   • docker-compose @klesh
   • helm-chart @IronCore864
5. v0.18 Upgrade Guide should guide users to extract the existing ENCODE_KEY and set it up on the new version
   • docker-compose @klesh
   • helm-chart @IronCore864
6. v0.18 Release Note must warn users it is a BREAKING change, and ask users to follow the Upgrade Guide @abeizn
7. Proceed issue Run devlake as a deployment instead of statefulset incubator-devlake-helm-chart#130

Note

Please NOTE that we are deprecating the reading/wrting from devlake to the .env, you may still use it as env_file in the docker-compose.yml, it is not related to the topic.

Related issues

• Run devlake as a deployment instead of statefulset incubator-devlake-helm-chart#130
• Add ENCODE_KEY helm values inputs incubator-devlake-helm-chart#133
• Many others: https://github.com/apache/incubator-devlake/issues?q=is%3Aissue+enckey

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[klesh]

@IronCore864 The .env file is optional now, we can now proceed to update the helm chart to use the user-generated ENCRYPTION_SECRET from Environment Variable, check apache/incubator-devlake-website#550 for detail