Refusing to start due to insecure SECRET_KEY

[dat-linux]

Why is this startup error happening with the latest release? (Ubuntu 22.04)

/superset/2.1.0/bin/superset db upgrade
--------------------------------------------------------------------------------
                                    WARNING
--------------------------------------------------------------------------------
A Default SECRET_KEY was detected, please use superset_config.py to override it.
Use a strong complex alphanumeric string and use a tool to help you generate 
a sufficiently random sequence, ex: openssl rand -base64 42
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
Refusing to start due to insecure SECRET_KEY

[mdeshmu]

It's because you are using default secret key is your config.

You will have to rotate secret key using https://superset.apache.org/docs/installation/configuring-superset/#secret_key-rotation

Then run "superset db upgrade"

[kamon405]

I'm unable to get the key with the provided code.

[kamon405]

Providing an update.. I found constants.py and saw the default value for the secret key... so I put that in my superset_config.py file as PREVIOUS_SECRET_KEY so the value is set, but I am still receiving this error. Since I do not have docker and can't use it due to certain environment constraints.. I am unable to use the working solutions provided in this thread.

[kamon405]

Ok so I figured it out. But it isn't the best solution but it's a development environment so this is fine for testing purposes. I set the secret _key variable in config.py within superset directory directly..

[jvanheijzen]

I figured it out. All of the online documentation uses the variable "SECRET_KEY". But in the config.py file it is looking for an environment variable called "SUPERSET_SECRET_KEY." Once I changed my variable name everything has started working properly.

[mdeshmu]

@sfirke @rusackas probably we should mention the environment variable SUPERSET_SECRET_KEY in here https://superset.apache.org/docs/installation/configuring-superset#specifying-a-secret_key

[dat-linux]

After full install (adding the secret key as required before "superset db upgrade", then "superset fab create-admin", then "superset load_examples", and "superset init"), the install looks to complete, but then an error in the application (did not get this is previous version):

sqlite3.OperationalError: no such table: main.cleaned_sales_data
2023-04-07 13:40:08,172:INFO:werkzeug:127.0.0.1 - - [07/Apr/2023 13:40:08] "PUT /api/v1/dashboard/9 HTTP/1.1" 200 -
2023-04-07 13:40:08,178:INFO:werkzeug:127.0.0.1 - - [07/Apr/2023 13:40:08] "GET /static/assets/6abe473e8ad84b72d7f9.chunk.js HTTP/1.1" 200 -
2023-04-07 13:40:08,257:INFO:werkzeug:127.0.0.1 - - [07/Apr/2023 13:40:08] "POST /api/v1/chart/data?form_data=%7B%22slice_id%22%3A117%7D&dashboard_id=9&force HTTP/1.1" 400 -
2023-04-07 13:40:08,289:INFO:werkzeug:127.0.0.1 - - [07/Apr/2023 13:40:08] "POST /api/v1/chart/data?form_data=%7B%22slice_id%22%3A125%7D&dashboard_id=9&force HTTP/1.1" 400 -
2023-04-07 13:40:08,307:INFO:werkzeug:127.0.0.1 - - [07/Apr/2023 13:40:08] "GET /static/assets/images/filter.svg HTTP/1.1" 200 -
2023-04-07 13:40:08,344:INFO:werkzeug:127.0.0.1 - - [07/Apr/2023 13:40:08] "POST /api/v1/chart/data?form_data=%7B%22slice_id%22%3A74%7D&dashboard_id=9&force HTTP/1.1" 400 -
2023-04-07 13:40:08,591:INFO:werkzeug:127.0.0.1 - - [07/Apr/2023 13:40:08] "POST /api/v1/chart/data?form_data=%7B%22slice_id%22%3A82%7D&dashboard_id=9&force HTTP/1.1" 400 -
2023-04-07 13:40:08,624:INFO:werkzeug:127.0.0.1 - - [07/Apr/2023 13:40:08] "POST /api/v1/dashboard/9/filter_state?tab_id=7 HTTP/1.1" 201 -

SCREEN SHOT

[MVanDerlofske]

I had a prior installation of Superset and have recently pulled the latest. Now I received the Refusing to start message, populating a new key allows the app to start but I cannot access any of my prior database connections.
I would like to perform the secret_key-rotation but since I didn't provide a prior value in my old configuration how can I know what my prior key was?
I have tried:
PREVIOUS_SECRET_KEY = 'thisismysecretkeychangeit'
PREVIOUS_SECRET_KEY = 'thisismysecretkey'
PREVIOUS_SECRET_KEY = 'thisismyscretkey\e\y\y\h' --Value from my image prior to upgrade running from flask import current_app; print(current_app.config["SECRET_KEY"])

All result in the error when running the rotate key command:
2023-04-08 03:43:38,925:INFO:superset.utils.encrypt:Collecting info for re encryption
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 126, in decrypt
decrypted = decrypted.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc0 in position 1: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/app/superset/utils/encrypt.py", line 146, in _re_encrypt_row
unencrypted_value = previous_encrypted_type.process_result_value(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 479, in process_result_value
value = super().process_result_value(value=value, dialect=dialect)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 424, in process_result_value
decrypted_value = self.engine.decrypt(value)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 128, in decrypt
raise ValueError('Invalid decryption key')
ValueError: Invalid decryption key

[MVanDerlofske]

One more try figuring maybe I need to escape the characters but still fails with an invalid decryption key:
PREVIOUS_SECRET_KEY = 'thisismyscretkey\e\y\y\h'

[MVanDerlofske]

I was able to revert my image back to version 2.0.1, get my key, upgrade, and run the re-encrypt-secrets. (CHANGE_ME_TO_A_COMPLEX_RANDOM_SECRET)

[mdeshmu]

@MVanDerlofske

Please send step by step guide you followed so that it can be helpful for others.

[MVanDerlofske]

In retrospect it was pretty simple. Here are the steps I took.
First I stopped my docker images from the upgrade:
docker-compose -f docker-compose-non-dev.yml down
Next I restored my backup directory from prior to doing the upgrade
mv superset supersetupgraded
mv supersetold superset
Next I updated my docker-compose-non-dev.yml in the superset directory to have a version prior to the version that required a secrete key:
x-superset-image: &superset-image apache/superset:2.0.1
Additionally, since I am using a custom Flask OpenId implementation for authentication I had to update my requirements-local.txt file
pymssql
itsdangerous<2.1
flask-openid
The need for this is documented here: puiterwijk/flask-oidc#147
Next I ran a pull:
docker-compose -f docker-compose-non-dev.yml pull
Then ran an up:
docker-compose -f docker-compose-non-dev.yml up -d
At this point the image was running, however I could not access the web interface due to a database schema mismatch.
This didn't matter because I only needed to extract the old secret key.
To get the secrete key I ran:
docker exec -it superset_app superset shell
from flask import current_app; print(current_app.config["SECRET_KEY"])
Now that I had the secret key stopped the docker images, and flipped the back directories:
docker-compose -f docker-compose-non-dev.yml down
mv superset supersetold
mv supersetupgraded superset
I updated my superset_config.py in superset/docker/pythonpath_dev/ to have a new SECRET_KEY and the old default key:
SECRET_KEY = 'output of openssl rand -base64 42'
PREVIOUS_SECRET_KEY = 'CHANGE_ME_TO_A_COMPLEX_RANDOM_SECRET'
Where the PREVIOUS_SECRET_KEY was the value I found above.
Next I ran a pull and an up:
docker-compose -f docker-compose-non-dev.yml pull
docker-compose -f docker-compose-non-dev.yml up -d
Once the updated image was running I ran the migration:
docker exec -it superset_app superset re-encrypt-secrets
Tested and everything was working as expected, I did have to clear my cookies but other then that it worked like a charm.

[sfirke]

Thanks for providing this walkthrough! Folks who didn't specify their own key can also browse the Superset repo's git history to see what the default value was at the version/time they deployed. Here's the file constants.py as of version 2.0.1 which shows the value you found: https://github.com/apache/superset/blob/507a7562e099707ab1103f4173d6c3b0ade2ec2d/superset/constants.py

[danielfernandez]

Please note in 2.1.0 this is now read at config.py as a runtime environment variable, so you can simply generate the secret key with:

$ openssl rand -base64 42

And then simply use the result of that command as the value of the SUPERSET_SECRET_KEY environment variable when running your docker image:

$ docker run ... -e SUPERSET_SECRET_KEY=xxxxxxxxx ...

...or include such variable in the corresponding environment variables file for the Docker CLI or Docker Compose.

[TrabajadorPraOleo]

In my case it doesn't work nothing. I change in the file "superset/config.py" the "SECRET KEY" for one created with "openssl rand -base64 42" and below I write "PREVIOUS_SECRET_KEY" with my old "SECRET KEY".

I save and exit. Next I ran a pull and an up:

-docker-compose -f docker-compose-non-dev.yml pull
-docker-compose -f docker-compose-non-dev.yml up -d

Once the updated successfully, I ran the migration with an error:
-docker exec -it superset_app superset re-encrypt-secrets

I don't understand it and I don't know what to do.

[haodengwavely]

openssl rand -base64 42

I have to manually append:
SUPERSET_SECRET_KEY=xxxxxx

to docker/.env-non-dev file

then docker compose up

[TrabajadorPraOleo]

@haodengwavely I think your answer has helped me but I don't know for sure because I have a new problem.

This error:
-ModuleNotFoundError: No module named 'cryptography.hazmat.backends.openssl.x509'

I have searched for this error in different forums. The forums says to install the specifics versions for the frameworks (WTFForms, cryptography, pyopenssl ...). I have tried but it has not been solved

[haodengwavely]

Another way to set the new key is to append this to /docker/pythonpath_dev/superset_config.py:
SECRET_KEY=xxxxxx

I didn't get ModuleNotFoundError.

I got another issue when loading "database" from UI, and solved by this thread:
#8538

[santosh-sahoo334]

One important step - need to run superset re-encrypt-secrets post superset db upgrade

[natemaxfield]

Hopefully this helps someone. I had the same issue with the default SECRET_KEY error. I am using docker compose with a Dockerfile. I solved the issue by adding an environment variable to the Dockerfile:

FROM apache/superset

ENV SUPERSET_SECRET_KEY=xxxxxxxx

...

I generated the secret key using openssl as recommended.

[ProjectsOfMLee]

I did everything the above solutions suggested, but the "Default Key detected ... Refusing to start due to insecure SECRET_KEY" still shows up once I run docker-compose up

[ASchmidtGit]

Same here, added previous and new secret key and run "superset re-encrypt-secrets" but service still refuses to start.

Edit: i run "superset re-encrypt-secrets" with "su superset" and get this error now:

Edit2: i also tried to add this command to the docker-init and docker-bootstrap.sh but still, refusing to start...

[HJrookie]

Same here, just add SUPERSET_SECRET_KEY to your os env , rather than SECRET_KEY

export SUPERSET_SECRET_KEY=xxxxxxxxxxxxxx/xxxxxxxxxx

because in venv/lib/python3.9/site-packages/superset/config.py :195 the python code try to get SUPERSET_SECRET_KEY from environment

SECRET_KEY = os.environ.get("SUPERSET_SECRET_KEY") or CHANGE_ME_SECRET_KEY

and then do superset db upgrade

[08tjlys]

it didn' work still

[coderyeasin]

I just set it in .env file SUPERSET_SECRET_KEY
after that i run below commands:
docker exec -it superset superset db upgrade
docker exec -it superset superset init

And its work for me

[JohnDietrich-Pepper]

This was a real PITA to fix. What I did:

• sudo docker compose down
• Modify superset_config to have a PREVIOUS_SECRET_KEY = "default key from the constant file linked above"
• Modify superset_config to have a SECRET_KEY = "generate this, I used Excel but you could use OpenSSH as noted above"
• sudo docker compose up -d
• sudo docker exec -it superset-app superset db upgrade
• sudo docker exec -it superset-app superset init
• sudo docker exec -it superset-app superset re-encrypt-secrets

And that worked. Now I assume I need to delete the Previous_Secret_Key before I do a compose down and up again

[cwegener]

And that worked. Now I assume I need to delete the Previous_Secret_Key before I do a compose down and up again

Nope. That value is only ever read when superset re-encrypt-secrets is run.

[cjrodriguez98]

superset re-encrypt-secrets
Loaded your LOCAL configuration at [/app/docker/pythonpath_dev/superset_config.py]
logging was configured successfully
2023-09-01 11:10:38,182:INFO:superset.utils.logging_configurator:logging was configured successfully
2023-09-01 11:10:38,189:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
/usr/local/lib/python3.9/site-packages/flask_limiter/extension.py:293: UserWarning: Using the in-memory storage for tracking rate limits as no storage was explicitly specified. This is not recommended for production use. See: https://flask-limiter.readthedocs.io#configuring-a-storage-backend for documentation about configuring the storage backend.
warnings.warn(
Collecting info for re encryption
2023-09-01 11:10:39,412:INFO:superset.utils.encrypt:Collecting info for re encryption
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 126, in decrypt
decrypted = decrypted.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfe in position 0: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/app/superset/utils/encrypt.py", line 150, in _re_encrypt_row
unencrypted_value = previous_encrypted_type.process_result_value(
File "/usr/local/lib/python3.9/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 479, in process_result_value
value = super().process_result_value(value=value, dialect=dialect)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 424, in process_result_value
decrypted_value = self.engine.decrypt(value)
File "/usr/local/lib/python3.9/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 128, in decrypt
raise ValueError('Invalid decryption key')
ValueError: Invalid decryption key

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/bin/superset", line 33, in
sys.exit(load_entry_point('apache-superset', 'console_scripts', 'superset')())
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/flask/cli.py", line 357, in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/app/superset/cli/update.py", line 119, in re_encrypt_secrets
secrets_migrator.run()
File "/app/superset/utils/encrypt.py", line 194, in run
self._re_encrypt_row(conn, row, table_name, columns)
File "/app/superset/utils/encrypt.py", line 167, in _re_encrypt_row
raise Exception from ex # pylint: disable=broad-exception-raised
Exception

I am getting this error

[ineu]

I have superset installed in kubernetes via helm. Upgraded it via helm upgrade and faced the problem from this thread.

The steps I tried to resolve it:

1. Set a new key in values.yaml:

extraSecretEnv:
  SUPERSET_SECRET_KEY: "something-here"

2. Run helm upgrade
3. See the init-db container failing with "Invalid decryption key"
4. Connect to the container running the web app (the init-db container dies immediately because of the exception)
5. Run superset shell, run from flask import current_app; print(current_app.config["SECRET_KEY"]), ensure that it returns the new key from values.yaml
6. Exit superset shell, run in bash: superset re-encrypt-secrets -a CHANGE_ME_TO_A_COMPLEX_RANDOM_SECRET

The last command fails with the same "Invalid decryption key" as the init-db container. I'm pretty sure that CHANGE_ME_TO_A_COMPLEX_RANDOM_SECRET is the correct previous value, because when I expicitly set to the key to it, the init-db container fails with Refusing to start thing.

I don't know if changing superset_config.py would have worked, because in kubernetes it's mounted read-only. But looks like the -a key to superset re-encrypt-secrets does not work.

[3etesol]

Hello, did you solve the issue?

[ineu]

No

[ineu]

In the end I connected to the superset DB directly and reset the passwords: update dbs set password = null, encrypted_extra = null;. It allowed the web process to start, making it possible to go to the admin interface and set passwords there.