v3.2.0 causes /public assets to return 503

[thiskevinwang]

Problem

When deploying my Next.js app with v3.2.0 my previously-working references of /public folder assets (ie. /image/kevin.webp, not to be confused with Next-Image usage) began returning a generic CloudFront 503.

Similarly, not-found routes returned the same CloudFront 503.

Steps I took to remediate

Downgrade from v3.2.0 to v3.1.0 and redeploy with AWS CDK.

✅ /public images began working again.
❌ Not-found routes began returning a generic S3 403, XML page.

Root cause

TBD, but should be within this diff: v3.1.0...v3.2.0

[ilyagru]

Hey @thiskevinwang, have you resolved this? It seems I'm getting the same:

503 ERROR
The request could not be satisfied.
The Lambda function associated with the CloudFront distribution is invalid or doesn't have the required permissions. We can't connect to the server for this app or website at this time. There might be too much traffic or a configuration error. Try again later, or contact the app or website owner.
If you provide content to customers through CloudFront, you can find steps to troubleshoot and help prevent this error by reviewing the CloudFront documentation. 

[ilyagru]

On CloudWatch, I found this, so might be the source of the issue?

2023-06-20T09:41:06.934Z	2bfecb77-795c-4069-b692-666895ee89f8	ERROR	{
  clientName: 'S3Client',
  commandName: 'ListObjectsV2Command',
  input: { Bucket: undefined, Prefix: 'UHWU2zPBkGlTVRE-lYYUO/index' },
  error: Error: No value provided for input HTTP label: Bucket.
      at _V (/var/task/cache.cjs:15:9782)
      at ise (/var/task/cache.cjs:31:100000)
      at async /var/task/cache.cjs:2:382,
  metadata: undefined
}

[thiskevinwang]

@ilyagru I haven't looked into it yet (and probably don't plan on digging in myself), but I believe I saw a slightly different 503 with no mention of "lambda".

What does your LogGroup name look like for that log?

[bestickley]

Hey @thiskevinwang, I wrote the code for 3.2 update - sorry I didn't catch the /public 503 issue. My best guess at locating the issue is when I cleaned up the "fix host header" code in assets/lambda@edge/LambdaOriginRequest.ts and put it into fixHostHeader function in inassets/lambda@edge/common.ts. Commit is here. I will try to debug later this week but wanted to provide that info in case anyone else wants to pick it up.

[kevin-mitchell]

On CloudWatch, I found this, so might be the source of the issue?

2023-06-20T09:41:06.934Z	2bfecb77-795c-4069-b692-666895ee89f8	ERROR	{
  clientName: 'S3Client',
  commandName: 'ListObjectsV2Command',
  input: { Bucket: undefined, Prefix: 'UHWU2zPBkGlTVRE-lYYUO/index' },
  error: Error: No value provided for input HTTP label: Bucket.
      at _V (/var/task/cache.cjs:15:9782)
      at ise (/var/task/cache.cjs:31:100000)
      at async /var/task/cache.cjs:2:382,
  metadata: undefined
}

I also am seeing this!

...
clientName: 'S3Client',
  commandName: 'ListObjectsV2Command',
  input: { Bucket: undefined, Prefix: 'Yl4y8zRHm1RfTpxK3eNC4/favicon.ico' },
  error: Error: No value provided for input HTTP label: Bucket.
...

I posted about this in the Discord in a discussion thread I have going, I'll report here if I hear anything.

[ilyagru]

Hey, the No value provided for input HTTP label: Bucket. error comes from ServerHandler log group.

I found another issue in default-EdgeFn-c82-DefaultOriginRequestEdge-... which might be related, not sure.

2023-06-20T09:46:32.439Z	c7acfe6b-ce93-4d5c-afed-3fa8df2400ae	ERROR	Invoke Error 	
{
    "errorType": "TypeError",
    "errorMessage": "Cannot read properties of undefined (reading 'value')",
    "stack": [
        "TypeError: Cannot read properties of undefined (reading 'value')",
        "    at Gt (/var/task/LambdaOriginRequest.js:12:3176)",
        "    at Bt (/var/task/LambdaOriginRequest.js:12:2335)",
        "    at Runtime.no [as handler] (/var/task/LambdaOriginRequest.js:12:2282)",
        "    at Runtime.handleOnceNonStreaming (file:///var/runtime/index.mjs:1086:29)"
    ]
}

[kevin-mitchell]

FYI @ilyagru I created #115 to track this specific issue. I'm going to try to see if I can figure out what is wrong, but to be honest most of the Next.js stuff I'm pretty unfamiliar with so I might not have much luck. I'll probably start an issue / discussion in Discord as well. If anybody has any understanding of what might cause this missing parameter that'd be amazing :).

[bestickley]

@thiskevinwang, I cannot reproduce this error cdk-nextjs-standalone@4.0.0-beta.0. See page here with image from public/ folder. I will take down the site once you confirm.

You can see the example code here.

I'll close this issue once you confirm.

[thiskevinwang]

Hey @bestickley, the issue occurs with references to public assets. NextImage usage is one exception where the breakage doesn't happen.

1. If you replace NextImage with a plain img tag, do you see an error?
   • https://github.com/bestickley/cdk-nextjs-standalone-issue-111/blob/main/ui/src/app/camera/page.tsx#L5
2. What if you view the asset url directly?

[bestickley]

@thiskevinwang, asset URLs viewed directly work too. See here.

[thiskevinwang]

I recently tried out 4.0.0-beta.1 and didn't see this issue anymore
Thanks @bestickley!