Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cy.request() errors when URL's path contains ’ character #5274

Closed
jennifer-shehane opened this issue Oct 2, 2019 · 7 comments · Fixed by #5813
Closed

cy.request() errors when URL's path contains ’ character #5274

jennifer-shehane opened this issue Oct 2, 2019 · 7 comments · Fixed by #5813

Comments

@jennifer-shehane
Copy link
Member

@jennifer-shehane jennifer-shehane commented Oct 2, 2019

Current behavior:

Using a character in the Path of the URL passed to cy.request() causes a TypeError: The header content contains invalid characters.

The apostrophe is not a ' apostrophe character, it's the character. If I paste the same url into my Chrome browser - the url redirects correctly.

I tried a lot of other characters that did not cause any issue including some emojis 😅

Why is someone even doing this? Well @oak-wildwood was dynamically pulling content from our docs to generate urls then using cy.request() to visit them - so the generated URLs had the character.

Screen Shot 2019-10-02 at 3 41 00 PM

Stack trace

RequestError: TypeError: The header content contains invalid characters
    at new RequestError (/Users/jennifer/Library/Caches/Cypress/3.4.1/Cypress.app/Contents/Resources/app/packages/server/node_modules/request-promise-core/lib/errors.js:14:15)
    at Request.plumbing.callback (/Users/jennifer/Library/Caches/Cypress/3.4.1/Cypress.app/Contents/Resources/app/packages/server/node_modules/request-promise-core/lib/plumbing.js:87:29)
    at Request.RP$callback [as _callback] (/Users/jennifer/Library/Caches/Cypress/3.4.1/Cypress.app/Contents/Resources/app/packages/server/node_modules/request-promise-core/lib/plumbing.js:46:31)
    at self.callback (/Users/jennifer/Library/Caches/Cypress/3.4.1/Cypress.app/Contents/Resources/app/packages/server/node_modules/request/request.js:185:22)
    at emitOne (events.js:116:13)
    at Request.emit (events.js:211:7)
    at Request.start (/Users/jennifer/Library/Caches/Cypress/3.4.1/Cypress.app/Contents/Resources/app/packages/server/node_modules/request/request.js:753:10)
    at Request.end (/Users/jennifer/Library/Caches/Cypress/3.4.1/Cypress.app/Contents/Resources/app/packages/server/node_modules/request/request.js:1511:10)
    at end (/Users/jennifer/Library/Caches/Cypress/3.4.1/Cypress.app/Contents/Resources/app/packages/server/node_modules/request/request.js:564:14)
    at Immediate.<anonymous> (/Users/jennifer/Library/Caches/Cypress/3.4.1/Cypress.app/Contents/Resources/app/packages/server/node_modules/request/request.js:578:7)
    at runCallback (timers.js:789:20)
    at tryOnImmediate (timers.js:751:5)
    at processImmediate [as _immediateCallback] (timers.js:722:5)

Screen Shot 2019-10-02 at 3 42 12 PM

Desired behavior:

Requesting this url should not throw an invalid characters warning and should instead behave as Chrome does.

Steps to reproduce: (app code and test code)

it('test', () => {
  cy.request('https://on.cypress.io/key-differences#What-you’ll-learn') // fails
})

it('test a fake url too', () => {
  cy.request('http://’')             // works
  cy.request('http://google.com’')   // works
  cy.request('http://google.com/’')  // fails
})

Versions

Cypress 3.4.1 & 3.5.0 prerelease

@flotwig

This comment has been minimized.

Copy link
Member

@flotwig flotwig commented Oct 2, 2019

The root of the problem here is that you can't put a Unicode character in the HTTP request path or in the HTTP request headers. So the request that is made by:

cy.request('http://google.com/’')    

Looks like this:

GET /’

Host: google.com

Which is invalid, since it contains Unicode, and shouldn't be sent anyways.

encodeURIComponent('’') returns "%E2%80%99", so maybe the proper request would be this:

cy.request('http://google.com/%E2%80%99')    

Can we automatically do this for users somehow, without breaking existing behavior? Maybe by first running decodeURI on the URL to undo any percent-encoding the user has already done, and then running encodeURI to encode any special characters?

@ayandebbarman

This comment has been minimized.

Copy link

@ayandebbarman ayandebbarman commented Oct 23, 2019

I am currently facing this same error as i am trying to verify through 1000s of links for places and many contains ' character. Any work around would be appreciated

@flotwig

This comment has been minimized.

Copy link
Member

@flotwig flotwig commented Oct 23, 2019

@ayandebbarman The issue is that Cypress does not encode the URL for you. You can use encodeURI to do this. So, instead of doing this:

cy.visit('http://google.com/’')

Do this:

cy.visit(encodeURI('http://google.com/’'))
@avallete

This comment has been minimized.

Copy link
Contributor

@avallete avallete commented Nov 27, 2019

I'm facing the same issue.

If no one already started working on this issue I would gladly submit a PR to fix it.

@flotwig

This comment has been minimized.

Copy link
Member

@flotwig flotwig commented Nov 27, 2019

@avallete Feel free to open a PR, let us know if you have any questions we can help with.

avallete added a commit to avallete/cypress that referenced this issue Nov 27, 2019
Use encodeURI inside `cy.request` to make sure the url does not contain
any unescaped Unicode character.

Fixes cypress-io#5274
avallete added a commit to avallete/cypress that referenced this issue Dec 8, 2019
Use `URL constructor` inside `cy.request` to make sure the url is well
encoded when for both domain and pathname.

Fixes cypress-io#5274
flotwig added a commit that referenced this issue Dec 11, 2019
* test(pkg/driver): url with unicode characters into .request command

* fix(pkg/driver): unicode character escaping in url for .request command

Use encodeURI inside `cy.request` to make sure the url does not contain
any unescaped Unicode character.

Fixes #5274

* fix(pkg/driver): unicode character escaping in url for .request command

Use `URL constructor` inside `cy.request` to make sure the url is well
encoded when for both domain and pathname.

Fixes #5274

* fix(tests): add some more tests for the case of a % character in url

* refactor(tests): Move the tests to request_spec test file

- Move the tests inside the request_spec test file instead of his own
test file to be sure it will be maintain.
- Rename some "it" test titles to semantically fit in the actual tests
naming convention.
- Remove the no more used 5274_spec.coffee test file.
@cypress-bot

This comment has been minimized.

Copy link

@cypress-bot cypress-bot bot commented Dec 11, 2019

The code for this is done in cypress-io/cypress#5813, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

@cypress-bot

This comment has been minimized.

Copy link

@cypress-bot cypress-bot bot commented Dec 12, 2019

Released in 3.8.0.

@cypress-io cypress-io locked as resolved and limited conversation to collaborators Dec 16, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
4 participants
You can’t perform that action at this time.