[v4] API token get invalidated seemingly at random

[ovlb]

Bug report

Describe the bug

We have a Strapi instance running on Digital Ocean. We use the API from it to deploy a site on Netlify.

When creating an API key, the first build always succeeds. After that it gets weird. We had some tokens which worked for two or three deploys, another one which worked for around 24 hours. The last two were returning 401 errors after the first deploy.

Here’s what we ruled out so far:

1. Rate limiting. I created a token last night, but the forst deploy this morning (~9 hours after last deploy and token creation was failing). And we had a token working for almost a day where we sometimes multiple deploys in ~90 seconds were working correctly.
2. [v4] npm install does not work with npm 8.3.0 #12232. I downgraded Node to v14 and NPM to v6 yesterday, and the problem is still creeping up.
3. Config errors. We use the netlify.toml file to store the API key, which does not change. And we can reproduce the error in Paw/Postman when using the key

The problem did not show up during local development but only since we use the «proper» deploy with DO/Netlify.

For now, we’ve changed the access permissions for the necessary post types to be public, as we must publish the prod site this evening. But that’s obviously not a long term solution.

Is there any config we might have missed?

Steps to reproduce the behavior

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior

An API key should work.

System

• Node.js version: v14/v16
• NPM version: v6/v8
• Strapi version: v4.0.4/v4.0.5
• Database: Postgres 12
• Operating system: Ubuntu

Additional context

Add any other context about the problem here.

[kibblerz]

Are you using a custom plugin?

[ovlb]

No. The settings are generated through Settings -> API Tokens.

[enflam3]

Hello, Iam also having sort of same situation with both Next.js frontend and Strapi backend sitting at Digital Ocean App platform.
When I make changes and push/redeploy Strapi App all my previous content + newly added stuff seems to deploy fine as I am using same managed DB. But I always have to regenerate new API token and update frontend env variables with newest token after each strapi redeploy. Otherwise old API token shows as persistent but frontend throws 401 Unauthorized when making requests.

[ovlb]

I checked our deploy logs and the failed frontend deploys relate to backend deploys on our site, too.

[MichaelLegg]

This sounds similar to what we're getting too over at #12212
Sometimes it seems almost random when they invalidate.

[derrickmehaffy]

Can't reproduce locally, need additional basic steps to reproduce 🤔

[ovlb]

Unfortunately, it’s impossible to reproduce this locally. It only seems to happen in a remote deployment environment. I’ve been working on this project since December and the issue came up only after we set up the prod env two days ago.

[MichaelLegg]

@ovlb Do you find that if you point your local version to look at the same database as your deployed version, that tokens don't work across both as shown here?:
https://www.loom.com/share/33ac3ffee9ee4371aff07627d7fbe9d2

I was finding that to be the case, in addition to the tokens becoming seemingly randomly invalidated on the hosted environment.

Are tokens somehow linked to the environment the service is deployed to? And perhaps changes in the environment, or the service perhaps being shutdown/moved and restarted on a different physical server on Digital Ocean, or in my case Heroku, is causing them to become invalid?

[ovlb]

@MichaelLegg Yup, it’s basically the same.

[MichaelLegg]

Some more testing around this, running locally.

If you create a token on a machine where the backend Strapi service is running (npm run develop), it works fine. it even continues to work if you restart the Strapi service or reboot the machine.

If you try to use that token on a different machine that's also locally running the Strapi service, you get a 403 (to be clear, both are pointing at the same, hosted database).

Are API tokens stored locally on the machine they're generated on as opposed to in the database?

[MichaelLegg]

Ah. So correct me if I'm wrong, but it looks like a Strapi instance will generate it's own API_TOKEN_SALT environment variable the first time you create an API token (or just when the env variable doesn't already exist). It then uses this salt going forward to validate and generate API tokens.

These obviously aren't committed - and since one didn't exist (for me at least) in my .env file when I first deployed to my hosting platform (because I hadn't yet created an API token), I never ended up configuring it on there.

Then - since I'm currently hosting on shared infrastructure that shuts itself down after a period of inactivity, the API_TOKEN_SALT environment variable was getting recreated every time it was spooled back up, and the environment variables loaded in.

I'm unsure if this is the same issue you're having @ovlb, but that seems likely to be the issue in my instance.

EDIT: I'm not sure what the solution here would be - it's not technically a bug, but it could perhaps be a good idea if the first time this environment variable gets created, it warns in the UI that it's been added, so that developers know to add the variable to their .env.

[ovlb]

Thanks for investigating this. I'm already in the weekend. Will verify if it's similar on our side on Monday.

[MichaelLegg]

No worries. Hopefully it's as simple as that.

Taking a looking at the docs for v4, they currently suggest that the salt should get automatically stored in /config/env/admin.js, and that you can optionally put it in your .env instead.

Looking in my admin.js however, it doesn't seem to have been added automatically - so a possible bug there. It also seems like a bit of a security issue to automatically populate a .js file with a salt, even if it was working.

[derrickmehaffy]

No worries. Hopefully it's as simple as that.

Taking a looking at the docs for v4, they currently suggest that the salt should get automatically stored in /config/env/admin.js, and that you can optionally put it in your .env instead.

Looking in my admin.js however, it doesn't seem to have been added automatically - so a possible bug there. It also seems like a bit of a security issue to automatically populate a .js file with a salt, even if it was working.

It's exactly that, and it's why I couldn't reproduce it.

[kevinvugts]

Can't reproduce locally, need additional basic steps to reproduce 🤔

Unfortunately I am also experiencing this with the basic JWT token to validate requests for strapi.
When copy and pasting the token to jwt.io the exp date is still 1 month ahead. Though, strapi returns a 401 error on this token used to request data from the server. Very weird stuf...

Any idea?

[derrickmehaffy]

Can't reproduce locally, need additional basic steps to reproduce thinking

Unfortunately I am also experiencing this with the basic JWT token to validate requests for strapi. When copy and pasting the token to jwt.io the exp date is still 1 month ahead. Though, strapi returns a 401 error on this token used to request data from the server. Very weird stuf...

Any idea?

Set the API_TOKEN_SALT variable in whatever hosting solution you use to some long random string. If you don't it's changed every time you deploy your code so all of the API Tokens are now invalidated because the salt has changed.

[kevinvugts]

Can't reproduce locally, need additional basic steps to reproduce thinking

Unfortunately I am also experiencing this with the basic JWT token to validate requests for strapi. When copy and pasting the token to jwt.io the exp date is still 1 month ahead. Though, strapi returns a 401 error on this token used to request data from the server. Very weird stuf...
Any idea?

Set the API_TOKEN_SALT variable in whatever hosting solution you use to some long random string. If you don't it's changed every time you deploy your code so all of the API Tokens are now invalidated because the salt has changed.

Hey,

i have set the API TOKEN SALT in my env’s. And also as statet per docs in admin.js in the object:

apiToken: {
salt: env(“API_TOKEN_SALT”)
}

This does not change the behaviour..

[derrickmehaffy]

Can't reproduce locally, need additional basic steps to reproduce thinking

Unfortunately I am also experiencing this with the basic JWT token to validate requests for strapi. When copy and pasting the token to jwt.io the exp date is still 1 month ahead. Though, strapi returns a 401 error on this token used to request data from the server. Very weird stuf...
Any idea?

Set the API_TOKEN_SALT variable in whatever hosting solution you use to some long random string. If you don't it's changed every time you deploy your code so all of the API Tokens are now invalidated because the salt has changed.

Hey,

i have set the API TOKEN SALT in my env’s. And also as statet per docs in admin.js in the object:

apiToken: { salt: env(“API_TOKEN_SALT”) }

This does not change the behaviour..

Are you passing a static salt key in your deployed environment? Where are you hosting?

[kevinvugts]

Can't reproduce locally, need additional basic steps to reproduce thinking

Unfortunately I am also experiencing this with the basic JWT token to validate requests for strapi. When copy and pasting the token to jwt.io the exp date is still 1 month ahead. Though, strapi returns a 401 error on this token used to request data from the server. Very weird stuf...
Any idea?

Set the API_TOKEN_SALT variable in whatever hosting solution you use to some long random string. If you don't it's changed every time you deploy your code so all of the API Tokens are now invalidated because the salt has changed.

Hey,
i have set the API TOKEN SALT in my env’s. And also as statet per docs in admin.js in the object:
apiToken: { salt: env(“API_TOKEN_SALT”) }
This does not change the behaviour..

Are you passing a static salt key in your deployed environment? Where are you hosting?

Jo Derrick,

Yes I am.
This is my admin.js file located in the config folder:

  console.log("SALT => ", env("API_TOKEN_SALT"));

  return {
    auth: {
      secret: env(
        "ADMIN_JWT_SECRET",
        "blablbabla"
      ),
    },
    apiToken: {
      salt: env(
        "API_TOKEN_SALT",
        "nanana"
      ),
    },
  };
};

And I am hosting stuff on Heroku. I see the correct salt key outputted in the console.log. So things are static over each deployment.

[derrickmehaffy]

@kevinvugts is it every deployment you see the failure?

[ovlb]

So, finally got back to the project. It seems like the issue is solved for me too by adding the Salt to my deploy environment.

@kevinvugts Have you tried deploying without setting apiToken.salt in your config? For me, it works without explicitly adding this to the config file.

@derrickmehaffy Can you explain the reasons that made you develop the feature in this way? Wouldn’t it be possible to generate a UUID once during project bootstrapping (similar to the ADMIN_JWT_SECRET or the UUID in the package.json and keep it persistent between builds? But I guess you had a reason to do it differently :)

[derrickmehaffy]

So, finally got back to the project. It seems like the issue is solved for me too by adding the Salt to my deploy environment.

@kevinvugts Have you tried deploying without setting apiToken.salt in your config? For me, it works without explicitly adding this to the config file.

@derrickmehaffy Can you explain the reasons that made you develop the feature in this way? Wouldn’t it be possible to generate a UUID once during project bootstrapping (similar to the ADMIN_JWT_SECRET or the UUID in the package.json and keep it persistent between builds? But I guess you had a reason to do it differently :)

It's set in the filesystem, not in the database. This is normal for most configuration options in Strapi such as the U&P JWT secret and Admin JWT secret.

There was some odd choices we made with how to auto-generate these secrets because we saw a lot of people in v3 not changing the defaults which is extremely insecure (even though we clearly documented how to) so a decision was made to forcefully generate new random ones if they weren't defined. The decision was slightly flawed by us doing it in a .env file (which has been changed now for production) and instead we will simply throw an error if the user doesn't set the keys via the normal means.

Our purpose is to help everyone be made aware of how to properly secure their applications and the simple answer is using environment variables properly. This will be made more clear when we completely overhaul our deployment documentation in the future.

[ovlb]

Gotcha, thanks :)

[petersg83]

Hopefully it's going to be fixed by #12287

[strapi-bot]

This issue has been mentioned on Strapi Community Forum. There might be relevant details there:

https://forum.strapi.io/t/modifying-api-token-lifetime/16053/5

[jonadeline]

@kevinvugts, did you try to generate a new API key after setting the SALT env var ? It's what i had to do to make it work. (and no need to change the config file)

[strapi-bot]

This issue has been mentioned on Strapi Community Forum. There might be relevant details there:

https://forum.strapi.io/t/api-token-salt-generation-strategy/37040/1