Set User-Agent #196

Closed
Georgi87 opened this Issue Apr 27, 2011 · 15 comments

Projects

None yet

10 participants

@Georgi87

I have tested jsdom and it works fine so far. There is only one thing that is missing for me. There should be an optional way to set an user-agent in jsdom.env as there are sites (e.g. wikipedia) that do not allow to be requested without an user-agent. http://meta.wikimedia.org/wiki/User-Agent_policy

While crawling a few pages with jsdom, I receive the following message after a few successful crawlings:

Scripts should use an informative User-Agent string with contact information, or they may be IP-blocked without notice.

@tmpvar
Owner

there are two issues here:

  1. https://github.com/tmpvar/jsdom/blob/master/lib/jsdom/browser/index.js#L130 needs to be moved out of the closure so we can extend it
  2. https://github.com/tmpvar/jsdom/blob/master/lib/jsdom/level2/html.js#L92-98 needs to utilize the window.userAgent
@jobowoo

Was this ever incorporated into main?

@tmpvar
Owner

the fix / tests have not yet been written. Are you up for the task?

@robinouu

+1 for this one, I have the same issue with jsdom . When will this feature be implemented ?

@tmpvar
Owner

When somebody writes the code and tests for it

@jeffstieler

I've started work on this, but I'm not sure if I'm going about it correctly.

Here's my branch: https://github.com/jeffstieler/jsdom/tree/196_set_user_agent

All of the tests still pass, but I've removed the usage of __proto__ from the DOMWindow prototype modification - I'll have to claim ignorance for not being able to still incorporate that.

What I have discovered though, is that you can specify a User Agent if you are using the jsdom.env convenience function.

Since it uses the request module rather than the core resource loader, it can accept a "headers" property on the configuration object.

@vicapow

+1

@lyetz

Was this ever implemented?

@domenic
Collaborator

@lyetz nope. Thanks for bringing it up though; we can try to make it a priority. I already saw the need to rearrange this code for #424 anyway.

@rowanmanning rowanmanning referenced this issue in springernature/truffler Jun 9, 2014
Closed

Allow setting user-agent string #1

@domenic domenic added feature and removed window bug TESTME labels Jul 10, 2014
@oveddan

@domenic Is this a feature request? I'd be glad to work on it.

@domenic
Collaborator

@oveddan go for it!

@oveddan

@domenic Is this already doable via the ability to set headers as done in #225?

@domenic
Collaborator

I doubt that would affect the client-side navigator.userAgent

@domenic
Collaborator

Wait, yes, this is definitely doable via config.headers. Closing.

@domenic domenic closed this Jun 26, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment