Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

failed to find a valid digest in the 'integrity' attribute? #297

Closed
georgesaliba opened this issue Nov 16, 2023 · 28 comments · Fixed by #304 or #311
Closed

failed to find a valid digest in the 'integrity' attribute? #297

georgesaliba opened this issue Nov 16, 2023 · 28 comments · Fixed by #304 or #311
Labels
question Further information is requested

Comments

@georgesaliba
Copy link

I recently updated to rc4 and all users got the following error when I deployed to production:

image

Is this related to the update? Is it a bug, or would there be a changelog somewhere that would indicate what changes I might need to make?

@georgesaliba georgesaliba added the question Further information is requested label Nov 16, 2023
@Baroshem
Copy link
Owner

Baroshem commented Nov 17, 2023

Hello,

Yes, I think it is caused by the latest changes in RC.4 where we have enabled strict CSP by default to enforce good security practices.

Take a look at following resources to learn more about Integrity

@vejja maybe we could add migration guide for this issue to Release Notes?

@georgesaliba
Copy link
Author

Makes sense. Yes, maybe a migration guide would be helpful - though maybe it makes sense that I should expect it to have breaking changes since I was on the RC. :)

But maybe the bigger problem is that I'm not sure that the feature is working right? That first article says "Nuxt Security automatically computes the integrity hash of each static asset (scripts, stylesheets, etc.) that are bundled in your Nuxt Application, and then inserts this value in the resulting HTML file." I'm pretty sure the failing files were all CSS & JS that are generated by my Nuxt application. Here's the screenshot with less obfuscated:

image

@Baroshem
Copy link
Owner

We are trying our best not to include any breaking changes but sometimes it is just not possible. There we also try to add migration guides to make migration easier.

At this point, if after upgrading, you encounter an issue that is blocking the applciation you can either stay at 1.0.0-rc.3 version or disable Subresource Integrity for now (security.sri: false)

Me and @vejja will try to find the solution in the meantime :)

@vejja
Copy link
Collaborator

vejja commented Nov 17, 2023

@georgesaliba Can you tell me what is the value of the integrity attribute of your /_nuxt/entry.9660c2bb.js ?

@georgesaliba
Copy link
Author

@vejja How can I find that? I think that file is dynamically constructed by Nuxt, the issue only occurred when I tried to ship the code into production (didn't repro locally), and I had to quickly roll it back because it caused the whole site to go down. So even if I knew how to calculate it, would I have to cause another live site outage to find that value, or is there a less impactful way?

@vejja
Copy link
Collaborator

vejja commented Nov 17, 2023

You're right... For now I think the safest thing to do is to set { sri: false } as @Baroshem indicated.
We might have to set this as the default for now, to avoid SRI blocking resources.

Is there a way you could build locally without deploying ?

If not : I am trying to understand why the calculated integrity hash differs from the one you deliver in production.
Are you deploying with a post-build step ? By that, I mean : is there any possibility that your js files are altered by your deployment pipeline (e.g. minification/uglification by your hosting service).
Also, are you deploying a static build (SSG) or a server (SSR) ?

@georgesaliba
Copy link
Author

Yes, I can run build locally.

But not sure how to find the integrity. I tried the fourth option command from Baroshem's second link above on my generated file, and it gave me a hash of wsVcd+MufiHE2rN83nQXLvySQzpPqUfo/pALNeuhqmHpP/M9kO0/4Yw8IVkyvW7g - but I'm not sure what the production one is (the code has changed since the outage). I'm guessing I'd have to deploy again to find it? Also assuming if I were to deploy with sri: false, I could then just download the file from my production server and run the same command to see if it's the same? There's a big launch today, so I probably won't be able to experiment with it this weekend, but maybe I can try this in the middle of the night sometime next week.

To answer your question, my stack is all based in Google Cloud, fronted by Cloudflare. So the static files are hosted by Google Firebase and dynamic bits are served by Google Cloud Functions, and Cloudflare caches various things for different periods. I'm not aware of any sort of minification or such happening post build, but I guess I wouldn't be shocked if either Google or Cloudflare was trying to optimize in some way, adding additional headers, etc.

@vejja
Copy link
Collaborator

vejja commented Nov 17, 2023

Ok thank you very much for the explanation.

You will find the integrity hash in the <head> section of the html of your file. Open your website with Chrome DevTools, and look either in the Elements tab (find the <head> in the parsed document), or in the Network tab (look for the Response sub-tab to find out the actual HTML that was delivered). In both cases your /_nuxt/entry.9660c2bb.js link in the <head> tag will have an integrity attribute - this is what I'm looking for.

One thing that you can do without risking another redeploy, is to save this /_nuxt/entry.9660c2bb.js file that Google Cloud/Cloudflare delivers to your users, and then compare it with the one that is produced by your local build.

I would want to understand if they are similar or if there are any differences. You can do a simple diff in the terminal to find out.

Totally understand that you probably don't have so much time to do that this kind of research, but if you manage to get the information it would be extremely helpful.

@georgesaliba
Copy link
Author

I assume there's no way to get the matching values on a local machine, eh? This is the local HTML I see, but it doesn't have this integrity value:
image

But it also doesn't include the URL hash, so I'm guessing it's just a dev-only version, and wouldn't match my prod env entirely. Once I'm able to redeploy to prod I'll try to follow the steps you mentioned to see if I can get the right value.

Oh, I also didn't answer you earlier question - the site is a mix of SSR and SPA.

@vejja
Copy link
Collaborator

vejja commented Nov 17, 2023

Yes this is a dev build, but weird that you don’t have integrity there
Locally you could try to set

  • rc4 for Nuxt-security
  • sri: true (under security)
  • npm run generate (assuming this is your build command)
  • npx serve .output/… (whatever the previous command says you should do)
    And look for the html on localhost

@Baroshem
Copy link
Owner

@georgesaliba could you verify if the issue still occurs?

I would like to release a new version this week and if this issue appears for more use cases, then we have to fix it before releasing the stable 1.0.0 version

@georgesaliba
Copy link
Author

So I was able to deploy to production with sri: false and it didn't bring the site down. However, I also don't see the integrity in the production HTML (maybe as a result of the sri being false?).

@Baroshem
Copy link
Owner

So if I understand correctly, when sri is set to true (which is by default) your site does not work correctly?

@georgesaliba
Copy link
Author

georgesaliba commented Nov 20, 2023 via email

@Baroshem
Copy link
Owner

I personally cannot reproduce it unfortunately and when I was testing it before releasing and after the release the default SRI was not causing any problems for me.

@georgesaliba Could you provide us with more info about your config? TBH, I have no idea what could be causing it so any information you could give us would help us solve the issue.

Also, have you tried this from @vejja ? #297 (comment)

@vejja
Copy link
Collaborator

vejja commented Nov 20, 2023

So I was able to deploy to production with sri: false and it didn't bring the site down. However, I also don't see the integrity in the production HTML (maybe as a result of the sri being false?).

Right, sri: false suppresses the integrity from the HTML, so this is normal.

One thing you could do, now that you have redeployed, is to diff the online version of your /_nuxt/entry.xxxxxxxx.js file with the one that you obtain locally by running npm run generate ?

@georgesaliba
Copy link
Author

So I can see there are differences, and it looks like something is slightly minifying the production file, removing unneeded whitespace:

image

Or this extra line feed at the end of the file:
image

Sure enough, if I go into Cloudflare, I can see that it has auto-minify turned on for CSS & JS. I'd bet that's the culprit - thanks guys!

@Baroshem
Copy link
Owner

@georgesaliba Thanks for verifying that.

Would you be able to disable this css/js minify for a while to see if it makes the Nuxt Security 1.0.0-rc.4 work?

This is currently our main blocker for releasing 1.0.0 version :(

@georgesaliba
Copy link
Author

georgesaliba commented Nov 20, 2023

I disabled the minification at Cloudflare and flushed my cached, but I'm still seeing slightly modified minified files. Not yet sure if it's just caching (files could be cached in some of the data centers even after flushing) or if there's something else in the stack that might be minifying the file. But I need to get the files matching first or redeploying with SRI enabled would guarantee an outage.

@Baroshem
Copy link
Owner

I see.

Lets reschedule the 1.0.0 stable release until I am back from vacation ~1st december.

@georgesaliba
Copy link
Author

georgesaliba commented Nov 21, 2023

So the cache appears to have flushed, and there are far fewer modifications to the files in production vs. pre-deployment. However, they're still different. It appears something is changing the space character. I'm not sure if it's part of some post-build deployment step (not intentional on my part if so), or some optimization by Google or Cloudflare, or just a result of different systems. But the space characters in the following are different, even though they look the same:

image

Digging into it, my local pre-deployed version is just a space character (20), while the post-deployed production version is a non-breaking space (c2 a0).

@vejja
Copy link
Collaborator

vejja commented Nov 21, 2023

Yes, this might be related to 2 different systems used for building.
Is there a way you can build in the production environment but without deploying? We would then be able to check manually the hashes without risking your site to go down ?

@georgesaliba
Copy link
Author

Nope, don't think I have a way to do that.

@vejja
Copy link
Collaborator

vejja commented Nov 22, 2023

Ok
How do you deploy ? Is there a CI/CD pipeline ?

@georgesaliba
Copy link
Author

@vejja No, I literally just run npm run build && firebase deploy and Firebase takes care of deploying it.

@vejja
Copy link
Collaborator

vejja commented Nov 22, 2023

@vejja No, I literally just run npm run build && firebase deploy and Firebase takes care of deploying it.

Ok I get it. But then there should be no differences between your local build and the Firebase deployed files, so that's the worse-case scenario from my perspective...

Is there anything specific about your npm run build or your nuxt.config.ts ? Do you use a Nitro preset to deploy to Firebase, or do you use some kind of firebaserc config to indicate where your functions are ?
I would have assumed that npm run build would have built an SSR application (unless you modified the package.json build script definition), and I think you said it was SSG/SPA.

I am asking because

  • If it is indeed SSG/SPA, there is something wrong but we can examine your local files in the temporary .nuxt folder
  • If it is in fact SSR, I would be ready to bet that everything will be fine with integrity hashes.

In other words, I'm trying to have a detailed picture of who is serving your files.

@vejja
Copy link
Collaborator

vejja commented Nov 22, 2023

Sorry forget my previous comment, I misread what you said which was: "a mix of SSR and SPA."

In which case I do believe you should be fine.
I think you can test this in Firebase with Previews or A/B Testing. This should allow you to verify that the hashes are fine without touching your existing deployment.

@vejja
Copy link
Collaborator

vejja commented Nov 28, 2023

PR #304 adds a section in Documentation about Post-Build Processing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants