Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add URL escaping for custom headers #12031

Merged
merged 6 commits into from
Apr 30, 2020
Merged

Add URL escaping for custom headers #12031

merged 6 commits into from
Apr 30, 2020

Conversation

ykzts
Copy link
Contributor

@ykzts ykzts commented Apr 19, 2020

Regression from #10324

Exception throws when adding URL to HTTP header in Next.js v9.3.5 or later (e.g. Content-Security-Policy).

TypeError: Missing parameter name at 41
    at lexer (/home/ykzts/local/src/reproduce-case/node_modules/next/dist/compiled/path-to-regexp/index.js:47:23)
    at parse (/home/ykzts/local/src/reproduce-case/node_modules/next/dist/compiled/path-to-regexp/index.js:97:18)
    at Object.compile (/home/ykzts/local/src/reproduce-case/node_modules/next/dist/compiled/path-to-regexp/index.js:181:29)
    at Object.fn (/home/ykzts/local/src/reproduce-case/node_modules/next/dist/next-server/server/next-server.js:307:58)
    at Router.execute (/home/ykzts/local/src/reproduce-case/node_modules/next/dist/next-server/server/router.js:134:44)
    at DevServer.run (/home/ykzts/local/src/reproduce-case/node_modules/next/dist/next-server/server/next-server.js:506:47)
    at DevServer.run (/home/ykzts/local/src/reproduce-case/node_modules/next/dist/server/next-dev-server.js:12:724)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async DevServer.handleRequest (/home/ykzts/local/src/reproduce-case/node_modules/next/dist/next-server/server/next-server.js:150:20)

Don't change the header value if the source doesn't contain a parms to improve this bug.

@ijjk
Copy link
Member

ijjk commented Apr 19, 2020

Stats from current PR

Default Server Mode (Increase detected ⚠️)
General Overall increase ⚠️
zeit/next.js canary ykzts/next.js without-params Change
buildDuration 12s 12.1s ⚠️ +157ms
nodeModulesSize 55.6 MB 55.6 MB ⚠️ +183 B
Page Load Tests Overall increase ✓
zeit/next.js canary ykzts/next.js without-params Change
/ failed reqs 0 0
/ total time (seconds) 2.197 2.094 -0.1
/ avg req/sec 1138.1 1193.69 ⚠️ +55.59
/error-in-render failed reqs 0 0
/error-in-render total time (seconds) 1.196 1.144 -0.05
/error-in-render avg req/sec 2089.74 2186.01 ⚠️ +96.27
Client Bundles (main, webpack, commons)
zeit/next.js canary ykzts/next.js without-params Change
main-HASH.js gzip 6.25 kB 6.25 kB
webpack-HASH.js gzip 746 B 746 B
de003c3a9d30..5e5e.js gzip 10.3 kB 10.3 kB
framework.a1..NSE.txt gzip 220 B 220 B
framework.a1..NSE.txt gzip 220 B 220 B
framework.HASH.js gzip 39 kB 39 kB
Overall change 56.7 kB 56.7 kB
Client Bundles (main, webpack, commons) Modern
zeit/next.js canary ykzts/next.js without-params Change
main-HASH.module.js gzip 4.79 kB 4.79 kB
webpack-HASH..dule.js gzip 746 B 746 B
de003c3a9d30..dule.js gzip 6.85 kB 6.85 kB
framework.HA..dule.js gzip 39 kB 39 kB
Overall change 51.4 kB 51.4 kB
Legacy Client Bundles (polyfills)
zeit/next.js canary ykzts/next.js without-params Change
polyfills-HASH.js gzip 26.2 kB 26.2 kB
Overall change 26.2 kB 26.2 kB
Client Pages
zeit/next.js canary ykzts/next.js without-params Change
_app.js gzip 1.23 kB 1.23 kB
_error.js gzip 3.12 kB 3.12 kB
hooks.js gzip 663 B 663 B
index.js gzip 222 B 222 B
link.js gzip 2.06 kB 2.06 kB
routerDirect.js gzip 280 B 280 B
withRouter.js gzip 278 B 278 B
Overall change 7.85 kB 7.85 kB
Client Pages Modern
zeit/next.js canary ykzts/next.js without-params Change
_app.module.js gzip 598 B 598 B
_error.module.js gzip 2.09 kB 2.09 kB
hooks.module.js gzip 383 B 383 B
index.module.js gzip 223 B 223 B
link.module.js gzip 1.52 kB 1.52 kB
routerDirect..dule.js gzip 279 B 279 B
withRouter.m..dule.js gzip 278 B 278 B
Overall change 5.37 kB 5.37 kB
Client Build Manifests
zeit/next.js canary ykzts/next.js without-params Change
_buildManifest.js gzip 61 B 61 B
_buildManife..dule.js gzip 61 B 61 B
Overall change 122 B 122 B
Rendered Page Sizes
zeit/next.js canary ykzts/next.js without-params Change
index.html gzip 928 B 928 B
link.html gzip 938 B 938 B
withRouter.html gzip 926 B 926 B
Overall change 2.79 kB 2.79 kB

Serverless Mode
General Overall increase ⚠️
zeit/next.js canary ykzts/next.js without-params Change
buildDuration 12.6s 12.9s ⚠️ +340ms
nodeModulesSize 55.6 MB 55.6 MB ⚠️ +183 B
Client Bundles (main, webpack, commons)
zeit/next.js canary ykzts/next.js without-params Change
main-HASH.js gzip 6.25 kB 6.25 kB
webpack-HASH.js gzip 746 B 746 B
de003c3a9d30..5e5e.js gzip 10.3 kB 10.3 kB
framework.a1..NSE.txt gzip 220 B 220 B
framework.a1..NSE.txt gzip 220 B 220 B
framework.HASH.js gzip 39 kB 39 kB
Overall change 56.7 kB 56.7 kB
Client Bundles (main, webpack, commons) Modern
zeit/next.js canary ykzts/next.js without-params Change
main-HASH.module.js gzip 4.79 kB 4.79 kB
webpack-HASH..dule.js gzip 746 B 746 B
de003c3a9d30..dule.js gzip 6.85 kB 6.85 kB
framework.HA..dule.js gzip 39 kB 39 kB
Overall change 51.4 kB 51.4 kB
Legacy Client Bundles (polyfills)
zeit/next.js canary ykzts/next.js without-params Change
polyfills-HASH.js gzip 26.2 kB 26.2 kB
Overall change 26.2 kB 26.2 kB
Client Pages
zeit/next.js canary ykzts/next.js without-params Change
_app.js gzip 1.23 kB 1.23 kB
_error.js gzip 3.12 kB 3.12 kB
hooks.js gzip 663 B 663 B
index.js gzip 222 B 222 B
link.js gzip 2.06 kB 2.06 kB
routerDirect.js gzip 280 B 280 B
withRouter.js gzip 278 B 278 B
Overall change 7.85 kB 7.85 kB
Client Pages Modern
zeit/next.js canary ykzts/next.js without-params Change
_app.module.js gzip 598 B 598 B
_error.module.js gzip 2.09 kB 2.09 kB
hooks.module.js gzip 383 B 383 B
index.module.js gzip 223 B 223 B
link.module.js gzip 1.52 kB 1.52 kB
routerDirect..dule.js gzip 279 B 279 B
withRouter.m..dule.js gzip 278 B 278 B
Overall change 5.37 kB 5.37 kB
Client Build Manifests
zeit/next.js canary ykzts/next.js without-params Change
_buildManifest.js gzip 61 B 61 B
_buildManife..dule.js gzip 61 B 61 B
Overall change 122 B 122 B
Serverless bundles
zeit/next.js canary ykzts/next.js without-params Change
_error.js 558 kB 558 kB
404.html 4.18 kB 4.18 kB
hooks.html 3.81 kB 3.81 kB
index.js 558 kB 558 kB
link.js 595 kB 595 kB
routerDirect.js 587 kB 587 kB
withRouter.js 587 kB 587 kB
Overall change 2.89 MB 2.89 MB

@ijjk
Copy link
Member

ijjk commented Apr 19, 2020

Stats from current PR

Default Server Mode (Decrease detected ✓)
General Overall increase ⚠️
zeit/next.js canary ykzts/next.js without-params Change
buildDuration 11.3s 11.3s ⚠️ +38ms
nodeModulesSize 55.6 MB 55.6 MB ⚠️ +183 B
Page Load Tests Overall decrease ⚠️
zeit/next.js canary ykzts/next.js without-params Change
/ failed reqs 0 0
/ total time (seconds) 1.87 1.859 -0.01
/ avg req/sec 1337.21 1344.45 ⚠️ +7.24
/error-in-render failed reqs 0 0
/error-in-render total time (seconds) 0.938 0.962 ⚠️ +0.02
/error-in-render avg req/sec 2664.75 2597.76 -66.99
Client Bundles (main, webpack, commons)
zeit/next.js canary ykzts/next.js without-params Change
main-HASH.js gzip 6.25 kB 6.25 kB
webpack-HASH.js gzip 746 B 746 B
de003c3a9d30..5e5e.js gzip 10.3 kB 10.3 kB
framework.a1..NSE.txt gzip 220 B 220 B
framework.a1..NSE.txt gzip 220 B 220 B
framework.HASH.js gzip 39 kB 39 kB
Overall change 56.7 kB 56.7 kB
Client Bundles (main, webpack, commons) Modern
zeit/next.js canary ykzts/next.js without-params Change
main-HASH.module.js gzip 4.79 kB 4.79 kB
webpack-HASH..dule.js gzip 746 B 746 B
de003c3a9d30..dule.js gzip 6.85 kB 6.85 kB
framework.HA..dule.js gzip 39 kB 39 kB
Overall change 51.4 kB 51.4 kB
Legacy Client Bundles (polyfills)
zeit/next.js canary ykzts/next.js without-params Change
polyfills-HASH.js gzip 26.2 kB 26.2 kB
Overall change 26.2 kB 26.2 kB
Client Pages
zeit/next.js canary ykzts/next.js without-params Change
_app.js gzip 1.23 kB 1.23 kB
_error.js gzip 3.12 kB 3.12 kB
hooks.js gzip 663 B 663 B
index.js gzip 222 B 222 B
link.js gzip 2.06 kB 2.06 kB
routerDirect.js gzip 280 B 280 B
withRouter.js gzip 278 B 278 B
Overall change 7.85 kB 7.85 kB
Client Pages Modern
zeit/next.js canary ykzts/next.js without-params Change
_app.module.js gzip 598 B 598 B
_error.module.js gzip 2.09 kB 2.09 kB
hooks.module.js gzip 383 B 383 B
index.module.js gzip 223 B 223 B
link.module.js gzip 1.52 kB 1.52 kB
routerDirect..dule.js gzip 279 B 279 B
withRouter.m..dule.js gzip 278 B 278 B
Overall change 5.37 kB 5.37 kB
Client Build Manifests
zeit/next.js canary ykzts/next.js without-params Change
_buildManifest.js gzip 61 B 61 B
_buildManife..dule.js gzip 61 B 61 B
Overall change 122 B 122 B
Rendered Page Sizes
zeit/next.js canary ykzts/next.js without-params Change
index.html gzip 928 B 928 B
link.html gzip 938 B 938 B
withRouter.html gzip 926 B 926 B
Overall change 2.79 kB 2.79 kB

Serverless Mode
General Overall increase ⚠️
zeit/next.js canary ykzts/next.js without-params Change
buildDuration 11.5s 11.6s ⚠️ +124ms
nodeModulesSize 55.6 MB 55.6 MB ⚠️ +183 B
Client Bundles (main, webpack, commons)
zeit/next.js canary ykzts/next.js without-params Change
main-HASH.js gzip 6.25 kB 6.25 kB
webpack-HASH.js gzip 746 B 746 B
de003c3a9d30..5e5e.js gzip 10.3 kB 10.3 kB
framework.a1..NSE.txt gzip 220 B 220 B
framework.a1..NSE.txt gzip 220 B 220 B
framework.HASH.js gzip 39 kB 39 kB
Overall change 56.7 kB 56.7 kB
Client Bundles (main, webpack, commons) Modern
zeit/next.js canary ykzts/next.js without-params Change
main-HASH.module.js gzip 4.79 kB 4.79 kB
webpack-HASH..dule.js gzip 746 B 746 B
de003c3a9d30..dule.js gzip 6.85 kB 6.85 kB
framework.HA..dule.js gzip 39 kB 39 kB
Overall change 51.4 kB 51.4 kB
Legacy Client Bundles (polyfills)
zeit/next.js canary ykzts/next.js without-params Change
polyfills-HASH.js gzip 26.2 kB 26.2 kB
Overall change 26.2 kB 26.2 kB
Client Pages
zeit/next.js canary ykzts/next.js without-params Change
_app.js gzip 1.23 kB 1.23 kB
_error.js gzip 3.12 kB 3.12 kB
hooks.js gzip 663 B 663 B
index.js gzip 222 B 222 B
link.js gzip 2.06 kB 2.06 kB
routerDirect.js gzip 280 B 280 B
withRouter.js gzip 278 B 278 B
Overall change 7.85 kB 7.85 kB
Client Pages Modern
zeit/next.js canary ykzts/next.js without-params Change
_app.module.js gzip 598 B 598 B
_error.module.js gzip 2.09 kB 2.09 kB
hooks.module.js gzip 383 B 383 B
index.module.js gzip 223 B 223 B
link.module.js gzip 1.52 kB 1.52 kB
routerDirect..dule.js gzip 279 B 279 B
withRouter.m..dule.js gzip 278 B 278 B
Overall change 5.37 kB 5.37 kB
Client Build Manifests
zeit/next.js canary ykzts/next.js without-params Change
_buildManifest.js gzip 61 B 61 B
_buildManife..dule.js gzip 61 B 61 B
Overall change 122 B 122 B
Serverless bundles
zeit/next.js canary ykzts/next.js without-params Change
_error.js 558 kB 558 kB
404.html 4.18 kB 4.18 kB
hooks.html 3.81 kB 3.81 kB
index.js 558 kB 558 kB
link.js 595 kB 595 kB
routerDirect.js 587 kB 587 kB
withRouter.js 587 kB 587 kB
Overall change 2.89 MB 2.89 MB

4cm4k1 added a commit to 4cm4k1/personal-website that referenced this pull request Apr 23, 2020
@4cm4k1
Copy link
Contributor

4cm4k1 commented Apr 29, 2020

Hey Next.js maintainers,

This is not a blocking issue for me because I have simply commented out the header that manifests the regression, but if you need a reproduction case to validate the bug and @ykzts’ PR, uncomment this: https://github.com/4cm4k1/personal-website/blob/master/next.config.js#L69-L72

Thanks!

@ijjk
Copy link
Member

ijjk commented Apr 30, 2020

Stats from current PR

Default Server Mode (Decrease detected ✓)
General Overall increase ⚠️
zeit/next.js canary ykzts/next.js without-params Change
buildDuration 12s 11.9s -94ms
nodeModulesSize 55.2 MB 55.2 MB ⚠️ +273 B
Page Load Tests Overall decrease ⚠️
zeit/next.js canary ykzts/next.js without-params Change
/ failed reqs 0 0
/ total time (seconds) 1.938 2.016 ⚠️ +0.08
/ avg req/sec 1289.71 1239.97 -49.74
/error-in-render failed reqs 0 0
/error-in-render total time (seconds) 1.043 1.042 0
/error-in-render avg req/sec 2397.91 2398.32 ⚠️ +0.41
Client Bundles (main, webpack, commons)
zeit/next.js canary ykzts/next.js without-params Change
main-HASH.js gzip 6.28 kB 6.28 kB
webpack-HASH.js gzip 746 B 746 B
de003c3a9d30..5232.js gzip 10.3 kB 10.3 kB
framework.a1..NSE.txt gzip 220 B 220 B
framework.a1..NSE.txt gzip 220 B 220 B
framework.HASH.js gzip 39.2 kB 39.2 kB
Overall change 57 kB 57 kB
Client Bundles (main, webpack, commons) Modern
zeit/next.js canary ykzts/next.js without-params Change
main-HASH.module.js gzip 4.82 kB 4.82 kB
webpack-HASH..dule.js gzip 746 B 746 B
de003c3a9d30..dule.js gzip 6.88 kB 6.88 kB
framework.HA..dule.js gzip 39.2 kB 39.2 kB
Overall change 51.7 kB 51.7 kB
Legacy Client Bundles (polyfills)
zeit/next.js canary ykzts/next.js without-params Change
polyfills-HASH.js gzip 26.3 kB 26.3 kB
Overall change 26.3 kB 26.3 kB
Client Pages
zeit/next.js canary ykzts/next.js without-params Change
_app.js gzip 1.23 kB 1.23 kB
_error.js gzip 3.13 kB 3.13 kB
hooks.js gzip 663 B 663 B
index.js gzip 222 B 222 B
link.js gzip 2.06 kB 2.06 kB
routerDirect.js gzip 280 B 280 B
withRouter.js gzip 278 B 278 B
Overall change 7.86 kB 7.86 kB
Client Pages Modern
zeit/next.js canary ykzts/next.js without-params Change
_app.module.js gzip 598 B 598 B
_error.module.js gzip 2.09 kB 2.09 kB
hooks.module.js gzip 383 B 383 B
index.module.js gzip 223 B 223 B
link.module.js gzip 1.52 kB 1.52 kB
routerDirect..dule.js gzip 279 B 279 B
withRouter.m..dule.js gzip 278 B 278 B
Overall change 5.37 kB 5.37 kB
Client Build Manifests
zeit/next.js canary ykzts/next.js without-params Change
_buildManifest.js gzip 61 B 61 B
_buildManife..dule.js gzip 61 B 61 B
Overall change 122 B 122 B
Rendered Page Sizes
zeit/next.js canary ykzts/next.js without-params Change
index.html gzip 926 B 926 B
link.html gzip 937 B 937 B
withRouter.html gzip 925 B 925 B
Overall change 2.79 kB 2.79 kB

Serverless Mode
General Overall increase ⚠️
zeit/next.js canary ykzts/next.js without-params Change
buildDuration 12.4s 12.7s ⚠️ +219ms
nodeModulesSize 55.2 MB 55.2 MB ⚠️ +273 B
Client Bundles (main, webpack, commons)
zeit/next.js canary ykzts/next.js without-params Change
main-HASH.js gzip 6.28 kB 6.28 kB
webpack-HASH.js gzip 746 B 746 B
de003c3a9d30..5232.js gzip 10.3 kB 10.3 kB
framework.a1..NSE.txt gzip 220 B 220 B
framework.a1..NSE.txt gzip 220 B 220 B
framework.HASH.js gzip 39.2 kB 39.2 kB
Overall change 57 kB 57 kB
Client Bundles (main, webpack, commons) Modern
zeit/next.js canary ykzts/next.js without-params Change
main-HASH.module.js gzip 4.82 kB 4.82 kB
webpack-HASH..dule.js gzip 746 B 746 B
de003c3a9d30..dule.js gzip 6.88 kB 6.88 kB
framework.HA..dule.js gzip 39.2 kB 39.2 kB
Overall change 51.7 kB 51.7 kB
Legacy Client Bundles (polyfills)
zeit/next.js canary ykzts/next.js without-params Change
polyfills-HASH.js gzip 26.3 kB 26.3 kB
Overall change 26.3 kB 26.3 kB
Client Pages
zeit/next.js canary ykzts/next.js without-params Change
_app.js gzip 1.23 kB 1.23 kB
_error.js gzip 3.13 kB 3.13 kB
hooks.js gzip 663 B 663 B
index.js gzip 222 B 222 B
link.js gzip 2.06 kB 2.06 kB
routerDirect.js gzip 280 B 280 B
withRouter.js gzip 278 B 278 B
Overall change 7.86 kB 7.86 kB
Client Pages Modern
zeit/next.js canary ykzts/next.js without-params Change
_app.module.js gzip 598 B 598 B
_error.module.js gzip 2.09 kB 2.09 kB
hooks.module.js gzip 383 B 383 B
index.module.js gzip 223 B 223 B
link.module.js gzip 1.52 kB 1.52 kB
routerDirect..dule.js gzip 279 B 279 B
withRouter.m..dule.js gzip 278 B 278 B
Overall change 5.37 kB 5.37 kB
Client Build Manifests
zeit/next.js canary ykzts/next.js without-params Change
_buildManifest.js gzip 61 B 61 B
_buildManife..dule.js gzip 61 B 61 B
Overall change 122 B 122 B
Serverless bundles
zeit/next.js canary ykzts/next.js without-params Change
_error.js 557 kB 557 kB
404.html 4.18 kB 4.18 kB
hooks.html 3.81 kB 3.81 kB
index.js 557 kB 557 kB
link.js 595 kB 595 kB
routerDirect.js 587 kB 587 kB
withRouter.js 588 kB 588 kB
Overall change 2.89 MB 2.89 MB

@ijjk
Copy link
Member

ijjk commented Apr 30, 2020

Failing test suites

test/integration/custom-routes/test/index.test.js

  • Custom routes > dev mode > should apply params header key/values with URL
  • Custom routes > server mode > should apply params header key/values with URL
  • Custom routes > serverless mode > should apply params header key/values with URL
Expand output

● Custom routes › dev mode › should apply params header key/values with URL

expect(received).toBe(expected) // Object.is equality

Expected: "https://example.com/first"
Received: null

  320 |   it('should apply params header key/values with URL', async () => {
  321 |     const res = await fetchViaHTTP(appPort, '/with-params/url/first')
> 322 |     expect(res.headers.get('x-url')).toBe('https://example.com/first')
      |                                      ^
  323 |   })
  324 | 
  325 |   it('should support named pattern for header key/values', async () => {

  at Object.<anonymous> (integration/custom-routes/test/index.test.js:322:38)

● Custom routes › server mode › should apply params header key/values with URL

expect(received).toBe(expected) // Object.is equality

Expected: "https://example.com/first"
Received: null

  320 |   it('should apply params header key/values with URL', async () => {
  321 |     const res = await fetchViaHTTP(appPort, '/with-params/url/first')
> 322 |     expect(res.headers.get('x-url')).toBe('https://example.com/first')
      |                                      ^
  323 |   })
  324 | 
  325 |   it('should support named pattern for header key/values', async () => {

  at Object.<anonymous> (integration/custom-routes/test/index.test.js:322:38)

● Custom routes › serverless mode › should apply params header key/values with URL

expect(received).toBe(expected) // Object.is equality

Expected: "https://example.com/first"
Received: null

  320 |   it('should apply params header key/values with URL', async () => {
  321 |     const res = await fetchViaHTTP(appPort, '/with-params/url/first')
> 322 |     expect(res.headers.get('x-url')).toBe('https://example.com/first')
      |                                      ^
  323 |   })
  324 | 
  325 |   it('should support named pattern for header key/values', async () => {

  at Object.<anonymous> (integration/custom-routes/test/index.test.js:322:38)
      at runMicrotasks (<anonymous>)

@ykzts ykzts changed the title Do not change headers when without params Add URL escaping for custom headers Apr 30, 2020
@ijjk ijjk added this to the 9.3.7 milestone Apr 30, 2020
@ijjk ijjk merged commit 703164e into vercel:canary Apr 30, 2020
@ijjk
Copy link
Member

ijjk commented Apr 30, 2020

Thanks for the PR!

@ykzts ykzts deleted the without-params branch May 1, 2020 10:15
@Timer Timer modified the milestones: 9.3.7, 9.4 May 11, 2020
rokinsky pushed a commit to rokinsky/next.js that referenced this pull request Jul 11, 2020
* Add URL escaping for custom headers

* Update to handle more header values

Co-authored-by: JJ Kasper <jj@jjsweb.site>
@vercel vercel locked as resolved and limited conversation to collaborators Jan 30, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants