WebDriver lacks HTTP response header and status code methods #141

Closed
lukeis opened this Issue Mar 2, 2016 · 135 comments

Comments

Projects
None yet
1 participant
@lukeis
Member

lukeis commented Mar 2, 2016

Originally reported on Google Code with ID 141


It would be useful to have methods to read the HTTP status code and headers from an
HTTP 
response.  HtmlUnit already has this functionality, but it's not been brought through
to WebDriver.

Reported by stephendv on 2009-02-01 16:05:44

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
Issue 152 has been merged into this issue.

Reported by simon.m.stewart on 2009-09-14 17:42:11

Member

lukeis commented Mar 2, 2016

Issue 152 has been merged into this issue.

Reported by simon.m.stewart on 2009-09-14 17:42:11

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
good feature

Reported by risticnemanja on 2009-10-16 13:27:22

Member

lukeis commented Mar 2, 2016

good feature

Reported by risticnemanja on 2009-10-16 13:27:22

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
good feature

Reported by risticnemanja on 2009-10-16 13:27:31

Member

lukeis commented Mar 2, 2016

good feature

Reported by risticnemanja on 2009-10-16 13:27:31

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
We will not be adding this feature to the WebDriver API as it falls outside of our current

scope (emulating user actions).

Reported by jmleyba on 2009-12-17 17:49:33

  • Status changed: WontFix
Member

lukeis commented Mar 2, 2016

We will not be adding this feature to the WebDriver API as it falls outside of our current

scope (emulating user actions).

Reported by jmleyba on 2009-12-17 17:49:33

  • Status changed: WontFix
@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
This would make WebDriver unusable for testing.
Selenium is supporting this feature, so I guess I am stuck with selenium forever :)

Reported by ivanjanic on 2009-12-17 18:23:32

Member

lukeis commented Mar 2, 2016

This would make WebDriver unusable for testing.
Selenium is supporting this feature, so I guess I am stuck with selenium forever :)

Reported by ivanjanic on 2009-12-17 18:23:32

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
Adding access for status code and response headers would be a really good addition.


If these features will not be added, then WebDriver cannot be regarded as a full web
testing tool, but merely a wrapper for various HTTP/S interaction APIs.

Reported by timo.westkamper on 2010-02-09 10:12:25

Member

lukeis commented Mar 2, 2016

Adding access for status code and response headers would be a really good addition.


If these features will not be added, then WebDriver cannot be regarded as a full web
testing tool, but merely a wrapper for various HTTP/S interaction APIs.

Reported by timo.westkamper on 2010-02-09 10:12:25

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
"Emulating user actions" might be useful for a generic browser driver, but for a 
TESTING tool it will be necessary to read the status code, headers and cookie values.

This is needed for performing non-functional automated tests, e.g.:
- Does the logon form always result in a 302 redirect?
- Is the session cookie set before logon, or after (to identify session fixation attacks)?
- Is the page with my bank details cached or not?

Until these features are added, I too cannot use webdriver for testing.

Reported by stephendv on 2010-02-09 10:21:41

Member

lukeis commented Mar 2, 2016

"Emulating user actions" might be useful for a generic browser driver, but for a 
TESTING tool it will be necessary to read the status code, headers and cookie values.

This is needed for performing non-functional automated tests, e.g.:
- Does the logon form always result in a 302 redirect?
- Is the session cookie set before logon, or after (to identify session fixation attacks)?
- Is the page with my bank details cached or not?

Until these features are added, I too cannot use webdriver for testing.

Reported by stephendv on 2010-02-09 10:21:41

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
I appreciate having a clean API, but this is honestly something that is very
important in a lot of my test cases so I hope it gets fixed.

Reported by ponzao on 2010-02-10 08:46:24

Member

lukeis commented Mar 2, 2016

I appreciate having a clean API, but this is honestly something that is very
important in a lot of my test cases so I hope it gets fixed.

Reported by ponzao on 2010-02-10 08:46:24

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
yes agree with above commnets. For a web testing tool its is imperative to be able to
read HTTP headers.


Reported by pnakhat on 2010-03-05 09:16:04

Member

lukeis commented Mar 2, 2016

yes agree with above commnets. For a web testing tool its is imperative to be able to
read HTTP headers.


Reported by pnakhat on 2010-03-05 09:16:04

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
Issue 404 has been merged into this issue.

Reported by dawagner on 2010-03-05 11:41:07

Member

lukeis commented Mar 2, 2016

Issue 404 has been merged into this issue.

Reported by dawagner on 2010-03-05 11:41:07

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
For the time being we are standing by our decision not to add this feature.  If we attempt
to make WebDriver the silver bullet of web testing, then it will suffer in overall
quality.

A few points to consider:

1. driver.get(url) blocks until the browser has loaded the page.  In the case of a
login redirect, what status/headers would you want returned?  By the semantics of the
API, we would 
return the response for the final loaded page.  You would end up with a 200 instead
of the 302 you're looking for.

2. WebDriver does support checking cookies, just not at the HTTP protocol level.  Take
a look at the WebDriver.Options interface:
http://selenium.googlecode.com/svn/trunk/docs/api/java/org/openqa/selenium/WebDriver.Options.html

3. If you want to test HTTP request/response headers, then write an appropriately sized
test and test those directly.  All you need is a simple HTTP client library (and as
you mentioned, 
HtmlUnit already offers some of these capabilities).  If you really want to combine
the two levels of testing, setup a proxy for your browsers so you can capture net traffic
and verify 
headers there.


Reported by ashley.leyba on 2010-03-10 12:41:45

Member

lukeis commented Mar 2, 2016

For the time being we are standing by our decision not to add this feature.  If we attempt
to make WebDriver the silver bullet of web testing, then it will suffer in overall
quality.

A few points to consider:

1. driver.get(url) blocks until the browser has loaded the page.  In the case of a
login redirect, what status/headers would you want returned?  By the semantics of the
API, we would 
return the response for the final loaded page.  You would end up with a 200 instead
of the 302 you're looking for.

2. WebDriver does support checking cookies, just not at the HTTP protocol level.  Take
a look at the WebDriver.Options interface:
http://selenium.googlecode.com/svn/trunk/docs/api/java/org/openqa/selenium/WebDriver.Options.html

3. If you want to test HTTP request/response headers, then write an appropriately sized
test and test those directly.  All you need is a simple HTTP client library (and as
you mentioned, 
HtmlUnit already offers some of these capabilities).  If you really want to combine
the two levels of testing, setup a proxy for your browsers so you can capture net traffic
and verify 
headers there.


Reported by ashley.leyba on 2010-03-10 12:41:45

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
This means that the only reason is "it falls outside of our current scope (emulating
user actions)".

I find it a bit strange. Why doesn't this apply to managing cookies?

Additionally if in the "GettingStarted" Wiki page you write:
"WebDriver is a tool for automating testing web applications, and in particular to
verify that they work as expected". The possibility to access response code is very
useful for this purpose.

Reported by mguillemot@yahoo.fr on 2010-03-10 12:53:43

Member

lukeis commented Mar 2, 2016

This means that the only reason is "it falls outside of our current scope (emulating
user actions)".

I find it a bit strange. Why doesn't this apply to managing cookies?

Additionally if in the "GettingStarted" Wiki page you write:
"WebDriver is a tool for automating testing web applications, and in particular to
verify that they work as expected". The possibility to access response code is very
useful for this purpose.

Reported by mguillemot@yahoo.fr on 2010-03-10 12:53:43

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
In my case I'm trying to test that the last step in a flow, and that is supposed to
generate a PDF. I won't try to validate the contents, but I do need to check that the
page isn't an error. The PDF generation I can test at a different level.

Now, I could implement this with HtmlUnit or something, but that basically means that
my nicely reusable Page objects (which all use WebDriver) are worhthless. I'd be
re-implementing a lot of the steps that WebDriver already fixed for me in such a nice
way.

Hope you'll reconsider.

Reported by anders.r.sveen on 2010-03-17 12:59:22

Member

lukeis commented Mar 2, 2016

In my case I'm trying to test that the last step in a flow, and that is supposed to
generate a PDF. I won't try to validate the contents, but I do need to check that the
page isn't an error. The PDF generation I can test at a different level.

Now, I could implement this with HtmlUnit or something, but that basically means that
my nicely reusable Page objects (which all use WebDriver) are worhthless. I'd be
re-implementing a lot of the steps that WebDriver already fixed for me in such a nice
way.

Hope you'll reconsider.

Reported by anders.r.sveen on 2010-03-17 12:59:22

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
This feature is highly needed. When I call get() on a URL, I would like to know if the
correct page was loaded or if it was a 404. Currently I can only get page source and
look for the 404 string but it is highly dependent of underlying server (because I
don't have a custom 404 page in my webapp).

Reported by glaurung.aubrane on 2010-08-25 20:53:34

Member

lukeis commented Mar 2, 2016

This feature is highly needed. When I call get() on a URL, I would like to know if the
correct page was loaded or if it was a 404. Currently I can only get page source and
look for the 404 string but it is highly dependent of underlying server (because I
don't have a custom 404 page in my webapp).

Reported by glaurung.aubrane on 2010-08-25 20:53:34

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
Please reconsider this issue.  
@jmleyba/ashley :-)

In case #1 it is reasonable for Webdriver to not be very verbose with redirects, etc.
 I would expect to be able to see the HTTP headers / status code of the rendered page
that was returned by driver.get(url)

Testers frequently use tools like Firebug, HttpFox, Web Developer Toolbar (Firefox),
iehttpheaders (IE), the built in developer tools in Chrome, and Safari Web Inspector
to inspect headers from the browser.

I don't think that looking at headers of the rendered page is outside the scope of
a browser driver used as a testing tool.

Reported by colin.harrington on 2010-08-26 14:22:14

Member

lukeis commented Mar 2, 2016

Please reconsider this issue.  
@jmleyba/ashley :-)

In case #1 it is reasonable for Webdriver to not be very verbose with redirects, etc.
 I would expect to be able to see the HTTP headers / status code of the rendered page
that was returned by driver.get(url)

Testers frequently use tools like Firebug, HttpFox, Web Developer Toolbar (Firefox),
iehttpheaders (IE), the built in developer tools in Chrome, and Safari Web Inspector
to inspect headers from the browser.

I don't think that looking at headers of the rendered page is outside the scope of
a browser driver used as a testing tool.

Reported by colin.harrington on 2010-08-26 14:22:14

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
I also find it crucial that WebDriver exposes the HTTP status code of the rendered page.
The same way you can get the raw, HTML by calling getPageSource() on the driver, or
getCurrentUrl(), you should be able to call getResponseCode(). How are the first two
methods so different to the last one here? 

If we apply your logic for not adding a method to get the HTTP status code, then you
also shouldn't add getPageSource(), since any data on the page should be extracted
from the DOM via the driver.

Reported by npellow on 2010-09-07 05:14:46

Member

lukeis commented Mar 2, 2016

I also find it crucial that WebDriver exposes the HTTP status code of the rendered page.
The same way you can get the raw, HTML by calling getPageSource() on the driver, or
getCurrentUrl(), you should be able to call getResponseCode(). How are the first two
methods so different to the last one here? 

If we apply your logic for not adding a method to get the HTTP status code, then you
also shouldn't add getPageSource(), since any data on the page should be extracted
from the DOM via the driver.

Reported by npellow on 2010-09-07 05:14:46

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
This is a crucial feature amiss in the WebDriver API today and doing it with a light-weight
HTTP-only library doesn't cut it as what WebDriver does through its browser and HtmlUnit
integration is so much more than what happens at the HTTP level and thus impossible
(or at least, very, very hard) to replicate in a "lightweight" HTTP library.

For instance, when you submit a form, you would like to know if the resulting page
returns HTTP Status Code 400 because of bad content within the posted content or 200
if everything went okay. Testing this when the form itself relies on JavaScript that
might alter the form values before they are posted makes it virtually impossible to
test this in an HTTP-only library.

The reason we use WebDriver is be able to test the functionality as close to the user
experience as possible, and whether you want to admit it or not, HTTP status codes
are so close to the user experience that it makes no sense to hide them in the API.

Most users know what "404 Not Found" means and this is the response they get when browsing
to a page that doesn't exist in most cases. While the user doesn't look at the actual
HTTP status code, but an HTML representation of it, he is exposed to the status code
while WebDriver itself doesn't expose it to the test that is supposed to carry out
the same action as a regular user.

If you don't want to do anything complicated here, then please at least just expose
the Request and Reponse as strings so we can parse and make sense of it ourselves.
Hiding this information is limiting the ways WebDriver can be used, especially when
it's testing through HtmlUnit where everything happens programmatically anyway.

Reported by asbjornu on 2010-10-20 11:07:02

Member

lukeis commented Mar 2, 2016

This is a crucial feature amiss in the WebDriver API today and doing it with a light-weight
HTTP-only library doesn't cut it as what WebDriver does through its browser and HtmlUnit
integration is so much more than what happens at the HTTP level and thus impossible
(or at least, very, very hard) to replicate in a "lightweight" HTTP library.

For instance, when you submit a form, you would like to know if the resulting page
returns HTTP Status Code 400 because of bad content within the posted content or 200
if everything went okay. Testing this when the form itself relies on JavaScript that
might alter the form values before they are posted makes it virtually impossible to
test this in an HTTP-only library.

The reason we use WebDriver is be able to test the functionality as close to the user
experience as possible, and whether you want to admit it or not, HTTP status codes
are so close to the user experience that it makes no sense to hide them in the API.

Most users know what "404 Not Found" means and this is the response they get when browsing
to a page that doesn't exist in most cases. While the user doesn't look at the actual
HTTP status code, but an HTML representation of it, he is exposed to the status code
while WebDriver itself doesn't expose it to the test that is supposed to carry out
the same action as a regular user.

If you don't want to do anything complicated here, then please at least just expose
the Request and Reponse as strings so we can parse and make sense of it ourselves.
Hiding this information is limiting the ways WebDriver can be used, especially when
it's testing through HtmlUnit where everything happens programmatically anyway.

Reported by asbjornu on 2010-10-20 11:07:02

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
This feature isn't going to happen. The recommended approach is to either extend the
HtmlUnitDriver to access the information you require or to make use of an external
proxy that exposes this information such as the BrowserMob Proxy (http://proxy.browsermob.com)

Reported by simon.m.stewart on 2010-10-20 11:37:49

Member

lukeis commented Mar 2, 2016

This feature isn't going to happen. The recommended approach is to either extend the
HtmlUnitDriver to access the information you require or to make use of an external
proxy that exposes this information such as the BrowserMob Proxy (http://proxy.browsermob.com)

Reported by simon.m.stewart on 2010-10-20 11:37:49

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
It's impossible to extend the HtmlUnitDriver when you're writing your tests in C#, since
there is no HtmlUnitDriver in C#. Or do you expect me to write a new "selenium-server-standalone-2.0a6.jar"
that exposes HTTP information and then write a new version of WebDriver that exposes
HTTP information to be able to get access to it?

If you enable people to get the source of the web page, you should enable people to
dig even further down into the protocol to detect whether the web application behaves
as it should or not. Just asserting on "visual" artifacts of a web page is simply just
not enough to get a full view of how your web application is behaving. It only tells
a part of the story and there's really no reason WebDriver can't tell the full story
(even though limited to how it would be seen through a web browser).

Reported by asbjornu on 2010-10-22 06:30:50

Member

lukeis commented Mar 2, 2016

It's impossible to extend the HtmlUnitDriver when you're writing your tests in C#, since
there is no HtmlUnitDriver in C#. Or do you expect me to write a new "selenium-server-standalone-2.0a6.jar"
that exposes HTTP information and then write a new version of WebDriver that exposes
HTTP information to be able to get access to it?

If you enable people to get the source of the web page, you should enable people to
dig even further down into the protocol to detect whether the web application behaves
as it should or not. Just asserting on "visual" artifacts of a web page is simply just
not enough to get a full view of how your web application is behaving. It only tells
a part of the story and there's really no reason WebDriver can't tell the full story
(even though limited to how it would be seen through a web browser).

Reported by asbjornu on 2010-10-22 06:30:50

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
Issue 919 has been merged into this issue.

Reported by dawagner on 2010-10-22 13:32:49

Member

lukeis commented Mar 2, 2016

Issue 919 has been merged into this issue.

Reported by dawagner on 2010-10-22 13:32:49

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
WebDriver is concerned with driving the browser. Where more sophisticated information
is required it is designed to integrate effectively with other tools which can provide
this deeper level of information. In the case of inspecting HTTP traffic, the only
way to do this in a cross-browser way is via a proxy. WebDriver exposes APIs to make
it easy to set the proxy to use for a browser instance, so you can hook in whichever
is the most suitable tool for your project.

As a starting point, we recommend looking at the BrowserMob Proxy (http://proxy.browsermob.com
)

Reported by simonstewart@google.com on 2010-10-25 11:21:16

Member

lukeis commented Mar 2, 2016

WebDriver is concerned with driving the browser. Where more sophisticated information
is required it is designed to integrate effectively with other tools which can provide
this deeper level of information. In the case of inspecting HTTP traffic, the only
way to do this in a cross-browser way is via a proxy. WebDriver exposes APIs to make
it easy to set the proxy to use for a browser instance, so you can hook in whichever
is the most suitable tool for your project.

As a starting point, we recommend looking at the BrowserMob Proxy (http://proxy.browsermob.com
)

Reported by simonstewart@google.com on 2010-10-25 11:21:16

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
Issue 950 has been merged into this issue.

Reported by dawagner on 2010-11-05 16:53:24

Member

lukeis commented Mar 2, 2016

Issue 950 has been merged into this issue.

Reported by dawagner on 2010-11-05 16:53:24

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
How can I control browser locale without this? 

Reported by serge.shikov on 2010-12-16 12:43:17

Member

lukeis commented Mar 2, 2016

How can I control browser locale without this? 

Reported by serge.shikov on 2010-12-16 12:43:17

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
for testing mobile site is absolutely necessary for us to be able to pass headers with
user-agent and phone num.
I appreciate you don't want to pollute the api, but we need so we're going to write
it.
Is there a way to make it available to others as well as plugin or some more nice method
than a diff? 

Reported by uberto.gama on 2011-02-02 12:25:59

Member

lukeis commented Mar 2, 2016

for testing mobile site is absolutely necessary for us to be able to pass headers with
user-agent and phone num.
I appreciate you don't want to pollute the api, but we need so we're going to write
it.
Is there a way to make it available to others as well as plugin or some more nice method
than a diff? 

Reported by uberto.gama on 2011-02-02 12:25:59

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
The best way to contribute would be to contact Patrick Lightbody and see what can be
added to the BrowserMob Proxy. That's a project based on the proxy used by selenium,
but the focus there is on adding the functionality that projects, such as yours, require.

Reported by simon.m.stewart on 2011-02-02 13:30:08

Member

lukeis commented Mar 2, 2016

The best way to contribute would be to contact Patrick Lightbody and see what can be
added to the BrowserMob Proxy. That's a project based on the proxy used by selenium,
but the focus there is on adding the functionality that projects, such as yours, require.

Reported by simon.m.stewart on 2011-02-02 13:30:08

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
mmmh the point is we need headers control in writing our cucumber tests.
Writing stuff like
Given I'm a iphone user with msisidn "12345678"
etc.etc.

and we translate that using user-agent and msisdn headers.
From what I understood from their site BrowserMob has no integration with cucumber.
Of course the ideal setup would be for webdriver to drive the iphone browser itself,
but for the moment we're happy with fake user-agent. Currectly DevQA is using modify-headers
plugin to test them.

For the same reason we need to test also result codes and redirects, from inside cucumber
tests.

Reported by uberto.gama on 2011-02-02 14:23:04

Member

lukeis commented Mar 2, 2016

mmmh the point is we need headers control in writing our cucumber tests.
Writing stuff like
Given I'm a iphone user with msisidn "12345678"
etc.etc.

and we translate that using user-agent and msisdn headers.
From what I understood from their site BrowserMob has no integration with cucumber.
Of course the ideal setup would be for webdriver to drive the iphone browser itself,
but for the moment we're happy with fake user-agent. Currectly DevQA is using modify-headers
plugin to test them.

For the same reason we need to test also result codes and redirects, from inside cucumber
tests.

Reported by uberto.gama on 2011-02-02 14:23:04

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
An issue tracker isn't really the right place for this conversation, but I understand
that the plan is to give the browsermob proxy an API.

Reported by simon.m.stewart on 2011-02-02 14:27:30

Member

lukeis commented Mar 2, 2016

An issue tracker isn't really the right place for this conversation, but I understand
that the plan is to give the browsermob proxy an API.

Reported by simon.m.stewart on 2011-02-02 14:27:30

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
Sad to see this issue marked as "Won't Fix", as it's an important one for me as the
others have noted above. An important part of our criteria for a web testing framework
is a test development environment that doesn't take too much knowhow, setup and maintenance
in and of itself. Since you're recommending an extension of HTMLUnitDriver, do you
have an example somewhere of how this is done?

Another criterion that's important for us is being able to keep our tests integrated.
If I can test a bunch of web app behavior in WebDriver but have to switch to BrowserMob
to check http responses, I apparently lose the ability to have a single test script.
Instead I have to split my test scripts, and fork my testing processes, based on what
WebDriver will and won't do. (It wouldn't be so bad splitting up testing based on meaningful
distinctions like functional vs. unit tests. But this would require splitting up at
a low level that runs across meaningful boundaries.) Wait, BrowserMob doesn't even
have an API? How then can it be recommended instead of WebDriver for use in automated
tests?

I can understand the desire to resist API bloat, but refusing to expose a widely requested
piece of information like this seems questionable.

Reported by huttarl on 2011-05-06 21:56:41

Member

lukeis commented Mar 2, 2016

Sad to see this issue marked as "Won't Fix", as it's an important one for me as the
others have noted above. An important part of our criteria for a web testing framework
is a test development environment that doesn't take too much knowhow, setup and maintenance
in and of itself. Since you're recommending an extension of HTMLUnitDriver, do you
have an example somewhere of how this is done?

Another criterion that's important for us is being able to keep our tests integrated.
If I can test a bunch of web app behavior in WebDriver but have to switch to BrowserMob
to check http responses, I apparently lose the ability to have a single test script.
Instead I have to split my test scripts, and fork my testing processes, based on what
WebDriver will and won't do. (It wouldn't be so bad splitting up testing based on meaningful
distinctions like functional vs. unit tests. But this would require splitting up at
a low level that runs across meaningful boundaries.) Wait, BrowserMob doesn't even
have an API? How then can it be recommended instead of WebDriver for use in automated
tests?

I can understand the desire to resist API bloat, but refusing to expose a widely requested
piece of information like this seems questionable.

Reported by huttarl on 2011-05-06 21:56:41

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
Though this issue is marked as will not fix, can the experts provide samples on integrating
broowsermob or htmlunit with webdriver?

Reported by rparampa on 2011-05-19 01:29:24

Member

lukeis commented Mar 2, 2016

Though this issue is marked as will not fix, can the experts provide samples on integrating
broowsermob or htmlunit with webdriver?

Reported by rparampa on 2011-05-19 01:29:24

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
I second comment #31 -- are there any samples available for how we can work around this
deficiency in Selenium/WebDriver? It seems to be quite a common issue.

Reported by neuronexmachina on 2011-06-01 00:16:24

Member

lukeis commented Mar 2, 2016

I second comment #31 -- are there any samples available for how we can work around this
deficiency in Selenium/WebDriver? It seems to be quite a common issue.

Reported by neuronexmachina on 2011-06-01 00:16:24

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
I blogged about a workaround for this using an embedded proxy. 

See my HOWTO here: http://www.supermind.org/blog/968/howto-collect-webdriver-http-request-and-response-headers

Reported by superkelvint on 2011-06-22 16:53:26

Member

lukeis commented Mar 2, 2016

I blogged about a workaround for this using an embedded proxy. 

See my HOWTO here: http://www.supermind.org/blog/968/howto-collect-webdriver-http-request-and-response-headers

Reported by superkelvint on 2011-06-22 16:53:26

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
Issue 2047 has been merged into this issue.

Reported by dawagner on 2011-07-13 15:25:39

Member

lukeis commented Mar 2, 2016

Issue 2047 has been merged into this issue.

Reported by dawagner on 2011-07-13 15:25:39

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
In http://code.google.com/p/selenium/issues/detail?id=2047, dawagner suggests using
BrowserMob's proxy.

Three issues with that:
- it doesn't support header manipulation via the REST API
- it only supports java natively, so no use to us python programmers
- The WebDriver python bindings don't appear to support proxies, or if it does, it's
not obvious how.

Isn't this design choice negating one of the main benefits of using WebDriver over
the original Selenium, that of removing the need to run a separate server?

Reported by rwillmer on 2011-07-14 13:34:15

Member

lukeis commented Mar 2, 2016

In http://code.google.com/p/selenium/issues/detail?id=2047, dawagner suggests using
BrowserMob's proxy.

Three issues with that:
- it doesn't support header manipulation via the REST API
- it only supports java natively, so no use to us python programmers
- The WebDriver python bindings don't appear to support proxies, or if it does, it's
not obvious how.

Isn't this design choice negating one of the main benefits of using WebDriver over
the original Selenium, that of removing the need to run a separate server?

Reported by rwillmer on 2011-07-14 13:34:15

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
We use the BrowserMob proxy as an example. You could use Fiddler, a commercial tool
or something you whip together yourself. Selenium is a browser automation framework,
not an HTTP recording framework, and as such HTTP recording and manipulation is out
of scope for the project.

In the common case, where you don't care about http headers, you can still run without
an additional server. In the case where that data is important to you, you will, almost
by definition, need a separate server.

Reported by simon.m.stewart on 2011-07-14 16:22:10

Member

lukeis commented Mar 2, 2016

We use the BrowserMob proxy as an example. You could use Fiddler, a commercial tool
or something you whip together yourself. Selenium is a browser automation framework,
not an HTTP recording framework, and as such HTTP recording and manipulation is out
of scope for the project.

In the common case, where you don't care about http headers, you can still run without
an additional server. In the case where that data is important to you, you will, almost
by definition, need a separate server.

Reported by simon.m.stewart on 2011-07-14 16:22:10

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
Is it possible for you to give an example of how you configure the python bindings to
use a proxy?

Reported by rwillmer on 2011-07-14 16:24:34

Member

lukeis commented Mar 2, 2016

Is it possible for you to give an example of how you configure the python bindings to
use a proxy?

Reported by rwillmer on 2011-07-14 16:24:34

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
Just to clarify some misunderstanding, it's explicitly nobody's job to handle this or
any other issue.  That's not to be trite.  Certainly the dev team is going to be the
most qualified to handle any of them.  But, each member on the team became qualified
by starting off as community members and contributing.  There seems to be a large misunderstanding
around the project that comes up time and again.  This is a volunteer-driven project.
 Employees of other companies that happen to work on Selenium are not being paid to
work solely on or provide support for Selenium.  Selenium is not special in this regard.

#108 I tend to agree with you.  Having said that, I've yet to see a case where a clean
solution that works in all drivers (or many of them anyway) hasn't managed to win the
argument.  Fortunately, now that the project is git-based, forking and keeping that
fork up to date is substantially easier than when this was a subversion project.  We
even review and accept pull requests through the GitHub mirror.  Which I guess is to
say if you do implement it, you have three valid paths, each of which sound valuable
to you: 1) the implementation is merged into Selenium; 2) you have a private fork with
the exact behavior you want and it's trivially kept up-to-date via git; or 3) you have
a public fork that everyone loves and you've de facto one the debate when people start
using your fork instead.

Reported by nirvdrum on 2013-06-22 16:18:00

Member

lukeis commented Mar 2, 2016

Just to clarify some misunderstanding, it's explicitly nobody's job to handle this or
any other issue.  That's not to be trite.  Certainly the dev team is going to be the
most qualified to handle any of them.  But, each member on the team became qualified
by starting off as community members and contributing.  There seems to be a large misunderstanding
around the project that comes up time and again.  This is a volunteer-driven project.
 Employees of other companies that happen to work on Selenium are not being paid to
work solely on or provide support for Selenium.  Selenium is not special in this regard.

#108 I tend to agree with you.  Having said that, I've yet to see a case where a clean
solution that works in all drivers (or many of them anyway) hasn't managed to win the
argument.  Fortunately, now that the project is git-based, forking and keeping that
fork up to date is substantially easier than when this was a subversion project.  We
even review and accept pull requests through the GitHub mirror.  Which I guess is to
say if you do implement it, you have three valid paths, each of which sound valuable
to you: 1) the implementation is merged into Selenium; 2) you have a private fork with
the exact behavior you want and it's trivially kept up-to-date via git; or 3) you have
a public fork that everyone loves and you've de facto one the debate when people start
using your fork instead.

Reported by nirvdrum on 2013-06-22 16:18:00

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
another blog post on the matter (by Simon):

http://blog.rocketpoweredjetpants.com/2013/06/the-unix-philosophy-webdriver-and-http.html

Reported by luke.semerau on 2013-06-24 20:36:53

Member

lukeis commented Mar 2, 2016

another blog post on the matter (by Simon):

http://blog.rocketpoweredjetpants.com/2013/06/the-unix-philosophy-webdriver-and-http.html

Reported by luke.semerau on 2013-06-24 20:36:53

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
required reading for anyone wanting to post a future comment:

http://jimevansmusic.blogspot.com/2013/08/implementing-webdriver-http-status.html
http://jimevansmusic.blogspot.com/2013/08/implementing-http-status-codes-in.html
http://jimevansmusic.blogspot.com/2013/08/implementing-http-status-codes-in_17.html

Reported by luke.semerau on 2013-08-26 18:13:37

Member

lukeis commented Mar 2, 2016

required reading for anyone wanting to post a future comment:

http://jimevansmusic.blogspot.com/2013/08/implementing-webdriver-http-status.html
http://jimevansmusic.blogspot.com/2013/08/implementing-http-status-codes-in.html
http://jimevansmusic.blogspot.com/2013/08/implementing-http-status-codes-in_17.html

Reported by luke.semerau on 2013-08-26 18:13:37

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
#109: I suppose (hopefully) what #107 meant by "it's your job" is that it *is* within
Selenium's scope.
See #85 explaining why developers who use Selenium feel that way.

Reported by huttarl on 2013-08-26 19:05:00

Member

lukeis commented Mar 2, 2016

#109: I suppose (hopefully) what #107 meant by "it's your job" is that it *is* within
Selenium's scope.
See #85 explaining why developers who use Selenium feel that way.

Reported by huttarl on 2013-08-26 19:05:00

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
At least there now seems to be clear instructions about how to get at an HTTP status
that goes beyond hand-waving. I'm not impressed that it takes three blog articles to
explain how to accomplish what takes a single method call with Twill, but I may have
to use that approach at some point, so the instructions are appreciated. The fact that
getting an HTTP status is possible using this convoluted method is not very satisfactory,
but if technical limitations mean it's the best we're likely to get, it's better than
nothing.

Reported by JonathanRRogers on 2013-08-27 00:23:19

Member

lukeis commented Mar 2, 2016

At least there now seems to be clear instructions about how to get at an HTTP status
that goes beyond hand-waving. I'm not impressed that it takes three blog articles to
explain how to accomplish what takes a single method call with Twill, but I may have
to use that approach at some point, so the instructions are appreciated. The fact that
getting an HTTP status is possible using this convoluted method is not very satisfactory,
but if technical limitations mean it's the best we're likely to get, it's better than
nothing.

Reported by JonathanRRogers on 2013-08-27 00:23:19

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
The blog posts are very appreciated. However, can I recommend to remove the pictures
at the top of each post? The overall experience of this issue (as you may have notices)
has been a bad one. Frustrated developers everywhere, almost begging for a feature
to be added and always having to hear "no". So, the pictures don't do any good in this
case I believe.

Reported by stijnherreman on 2013-08-29 14:21:19

Member

lukeis commented Mar 2, 2016

The blog posts are very appreciated. However, can I recommend to remove the pictures
at the top of each post? The overall experience of this issue (as you may have notices)
has been a bad one. Frustrated developers everywhere, almost begging for a feature
to be added and always having to hear "no". So, the pictures don't do any good in this
case I believe.

Reported by stijnherreman on 2013-08-29 14:21:19

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
I'm glad to see motion on this issue as comment #111 reads.  An additional use case
for this relates to SEO...

From the Bing "Note that in order for Bing to detect that the page has in fact been
removed from your site and is now returning a 404 or 410 HTTP status code, Bingbot
needs to be able to access the URL...".

The approved way to get a page out of a search engine is to return code 404.  Selenium
and Selenium IDE are the ideal tools to verify the site is returning a search engine
compatible return code.  Bing will not read the result page like a human.

Reported by digitalbitstream on 2013-09-10 17:45:43

Member

lukeis commented Mar 2, 2016

I'm glad to see motion on this issue as comment #111 reads.  An additional use case
for this relates to SEO...

From the Bing "Note that in order for Bing to detect that the page has in fact been
removed from your site and is now returning a 404 or 410 HTTP status code, Bingbot
needs to be able to access the URL...".

The approved way to get a page out of a search engine is to return code 404.  Selenium
and Selenium IDE are the ideal tools to verify the site is returning a search engine
compatible return code.  Bing will not read the result page like a human.

Reported by digitalbitstream on 2013-09-10 17:45:43

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
I've ported Jim's example into Ruby if anyone is interested. You can find it at http://elementalselenium.com/tips/17-retrieve-http-status-codes


Cheers,
Dave H
@TourDeDave

Reported by dave@arrgyle.com on 2013-09-17 16:01:00

Member

lukeis commented Mar 2, 2016

I've ported Jim's example into Ruby if anyone is interested. You can find it at http://elementalselenium.com/tips/17-retrieve-http-status-codes


Cheers,
Dave H
@TourDeDave

Reported by dave@arrgyle.com on 2013-09-17 16:01:00

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
I agree completely with 107. I have only just come across this issue and I have read
about the proxy solution which I guess I will have to implement.  

Really do not need another dependency for such a basic thing as retrieving a status
code but it looks like the only solution at this point.

I understand the point about being for "emulating user action" .. user action when
faced with a 404 or 500 etc is to stop what they were doing .. how are we supposed
to do the same when we cannot even determine if that was the case when we have no idea
how these pages will be rendered on different services.

As 107 said it sounds like the project team are very close minded about the use cases
for this software.

Would love to see this added at some point.




Reported by pg1671 on 2013-10-26 00:24:38

Member

lukeis commented Mar 2, 2016

I agree completely with 107. I have only just come across this issue and I have read
about the proxy solution which I guess I will have to implement.  

Really do not need another dependency for such a basic thing as retrieving a status
code but it looks like the only solution at this point.

I understand the point about being for "emulating user action" .. user action when
faced with a 404 or 500 etc is to stop what they were doing .. how are we supposed
to do the same when we cannot even determine if that was the case when we have no idea
how these pages will be rendered on different services.

As 107 said it sounds like the project team are very close minded about the use cases
for this software.

Would love to see this added at some point.




Reported by pg1671 on 2013-10-26 00:24:38

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
Four years after the start of this conversation it's clear many people want this feature.
I have to admit I can't understand why reading HTTP headers falls outside the scope
of WebDriver.

Reported by mikosullivan on 2013-10-29 03:05:53

Member

lukeis commented Mar 2, 2016

Four years after the start of this conversation it's clear many people want this feature.
I have to admit I can't understand why reading HTTP headers falls outside the scope
of WebDriver.

Reported by mikosullivan on 2013-10-29 03:05:53

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
Not found the required reading.I also want this feature for this software.Thanks!

Reported by luoxuwy on 2013-11-04 03:05:39

Member

lukeis commented Mar 2, 2016

Not found the required reading.I also want this feature for this software.Thanks!

Reported by luoxuwy on 2013-11-04 03:05:39

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
I can understand the "out of scope" thing. But, i need that feature too. I am attending
to test our users's users. And so it is a NxN diagram to test. They can change the
theme. But they can not change the request headers. And that is the only way for me
to test that issue. For this reason, it is not out of scope, i think.

I am not capable of building my own solution, except using a ProxyServer.  Make him
log. Read logs then solve the issue.


Reported by macitemre on 2013-11-19 16:46:16

Member

lukeis commented Mar 2, 2016

I can understand the "out of scope" thing. But, i need that feature too. I am attending
to test our users's users. And so it is a NxN diagram to test. They can change the
theme. But they can not change the request headers. And that is the only way for me
to test that issue. For this reason, it is not out of scope, i think.

I am not capable of building my own solution, except using a ProxyServer.  Make him
log. Read logs then solve the issue.


Reported by macitemre on 2013-11-19 16:46:16

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
This feature would be very useful to me.  I understand the desire to keep a clean API,
but ultimately the end-user of the API are the developers and if many developers want
a feature than a patch should be accepted at the minimum.

Reported by dan.sandberg on 2013-12-07 07:16:53

Member

lukeis commented Mar 2, 2016

This feature would be very useful to me.  I understand the desire to keep a clean API,
but ultimately the end-user of the API are the developers and if many developers want
a feature than a patch should be accepted at the minimum.

Reported by dan.sandberg on 2013-12-07 07:16:53

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
Issue 2047 has been merged into this issue.

Reported by nirvdrum on 2013-12-16 14:13:00

Member

lukeis commented Mar 2, 2016

Issue 2047 has been merged into this issue.

Reported by nirvdrum on 2013-12-16 14:13:00

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
For our website mobile testing, we'd like to be able to set the user-agent.

(This echoes comment #103 from the Wikipedia team above).

It seems a bit strange that you can't do something as simple as setting the user-agent
in Selenium.

Is that also out of scope, like getting back the HTTP Status codes?


Reported by victorhooi on 2014-01-28 01:17:00

Member

lukeis commented Mar 2, 2016

For our website mobile testing, we'd like to be able to set the user-agent.

(This echoes comment #103 from the Wikipedia team above).

It seems a bit strange that you can't do something as simple as setting the user-agent
in Selenium.

Is that also out of scope, like getting back the HTTP Status codes?


Reported by victorhooi on 2014-01-28 01:17:00

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
Issue 2047 has been merged into this issue.

Reported by nirvdrum on 2014-01-28 04:26:32

Member

lukeis commented Mar 2, 2016

Issue 2047 has been merged into this issue.

Reported by nirvdrum on 2014-01-28 04:26:32

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
#123 - Please see the discussion on issue 2047 for the matter of request headers.

Reported by nirvdrum on 2014-01-28 04:28:17

Member

lukeis commented Mar 2, 2016

#123 - Please see the discussion on issue 2047 for the matter of request headers.

Reported by nirvdrum on 2014-01-28 04:28:17

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
Came upon this discussion because I am dealing with an issue right now where having
access to the underlying http headers would be extremely convenient. As much as I would
like to weigh in on the side of adding it, I understand the team's position. It's a
simple fact that http protocol information is per-request, and the output of webdriver
is a fully-rendered page which will have made a number of requests since the initial
action to request the resource. Should they really try to populate an array with the
results of every HTTP request made on the page? Doesn't make sense to me. So, bottom
line: it's a one-to-many problem, and I can understand them ruling it out of scope.

Reported by betz.mark on 2014-04-10 21:12:42

Member

lukeis commented Mar 2, 2016

Came upon this discussion because I am dealing with an issue right now where having
access to the underlying http headers would be extremely convenient. As much as I would
like to weigh in on the side of adding it, I understand the team's position. It's a
simple fact that http protocol information is per-request, and the output of webdriver
is a fully-rendered page which will have made a number of requests since the initial
action to request the resource. Should they really try to populate an array with the
results of every HTTP request made on the page? Doesn't make sense to me. So, bottom
line: it's a one-to-many problem, and I can understand them ruling it out of scope.

Reported by betz.mark on 2014-04-10 21:12:42

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
While I can definitely understand the one to many aspect of it, at the very least I
would see it being a useful thing to add for the context of the page you loaded directly
(eg the URL you navigated to). Sure, there may have been many requests as part of that,
but I think you would cover a good chunk of what people want (80/20 rule) with a fairly
minor (arbitrary guess) amount of overall effort.

My 2c

Reported by glenn@ensquared.net on 2014-07-28 01:16:17

Member

lukeis commented Mar 2, 2016

While I can definitely understand the one to many aspect of it, at the very least I
would see it being a useful thing to add for the context of the page you loaded directly
(eg the URL you navigated to). Sure, there may have been many requests as part of that,
but I think you would cover a good chunk of what people want (80/20 rule) with a fairly
minor (arbitrary guess) amount of overall effort.

My 2c

Reported by glenn@ensquared.net on 2014-07-28 01:16:17

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
When I'm testing or debugging a web application, the very first thing I do is open up
the web developer's console. Regardless of which browser I'm using, I'll see timing
information, headers, cookies, post data, and RESPONSE CODES.

Maybe back in 2009 it wasn't easy to see all this data in a browser, but 5 years later,
response codes are routinely examined in a browser when you're looking at a 302 redirection
chain.

Browsers have gotten better since this issue was first closed, why shouldn't WebDriver
keep up?

Reported by ironchefpython on 2014-09-15 19:00:04

Member

lukeis commented Mar 2, 2016

When I'm testing or debugging a web application, the very first thing I do is open up
the web developer's console. Regardless of which browser I'm using, I'll see timing
information, headers, cookies, post data, and RESPONSE CODES.

Maybe back in 2009 it wasn't easy to see all this data in a browser, but 5 years later,
response codes are routinely examined in a browser when you're looking at a 302 redirection
chain.

Browsers have gotten better since this issue was first closed, why shouldn't WebDriver
keep up?

Reported by ironchefpython on 2014-09-15 19:00:04

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
Up ! It should be really useful...

Reported by lucas@cherifi.info on 2014-09-29 16:04:40

Member

lukeis commented Mar 2, 2016

Up ! It should be really useful...

Reported by lucas@cherifi.info on 2014-09-29 16:04:40

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
I was kind of shocked when I realized response codes were not accessible via WebDriver.
 I'm integrating HTMLUnitDriver and WebDriver with JMeter, and it would be super-useful
to be able to check the response codes, rather than having to parse the response data.

Reported by mark.bonassera on 2014-10-02 18:00:36

Member

lukeis commented Mar 2, 2016

I was kind of shocked when I realized response codes were not accessible via WebDriver.
 I'm integrating HTMLUnitDriver and WebDriver with JMeter, and it would be super-useful
to be able to check the response codes, rather than having to parse the response data.

Reported by mark.bonassera on 2014-10-02 18:00:36

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
i do not understand why you do not offer the HTTP error codes via the webdriver. i am
testing a webpage and occassionaly get something else than HTTP 200. should i now parse
the html and define every possible HTTP error that might come? What if somebody changes
the webserver or webserver version? are you expecting me to rewrite the html parsing
then again? Wouldn't it be easier just to check for the error code in the browser?

Reported by erik.dobak on 2014-10-06 08:22:50

Member

lukeis commented Mar 2, 2016

i do not understand why you do not offer the HTTP error codes via the webdriver. i am
testing a webpage and occassionaly get something else than HTTP 200. should i now parse
the html and define every possible HTTP error that might come? What if somebody changes
the webserver or webserver version? are you expecting me to rewrite the html parsing
then again? Wouldn't it be easier just to check for the error code in the browser?

Reported by erik.dobak on 2014-10-06 08:22:50

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
If people want to implement their own workaround, see my solution: https://github.com/MachinePublishers/ScreenSlicer/blob/master/core/firefox-addon/lib/main.js

It's a firefox addon that calls 127.0.0.1:8888/httpstatus/{statuscode}. You'd need
a webservice listening at that extension. With some work you might be able to write
that status into the html page itself. You could write something similar for if you're
using other browser drivers. The executable addon that you can drop into the extensions
dir of your firefox profile is: https://github.com/MachinePublishers/ScreenSlicer/raw/master/core/firefox-addon/build/jid1-QcfAcjQBdRo0lA%40jetpack.xpi

Reported by dan2k7 on 2014-10-17 16:40:16

Member

lukeis commented Mar 2, 2016

If people want to implement their own workaround, see my solution: https://github.com/MachinePublishers/ScreenSlicer/blob/master/core/firefox-addon/lib/main.js

It's a firefox addon that calls 127.0.0.1:8888/httpstatus/{statuscode}. You'd need
a webservice listening at that extension. With some work you might be able to write
that status into the html page itself. You could write something similar for if you're
using other browser drivers. The executable addon that you can drop into the extensions
dir of your firefox profile is: https://github.com/MachinePublishers/ScreenSlicer/raw/master/core/firefox-addon/build/jid1-QcfAcjQBdRo0lA%40jetpack.xpi

Reported by dan2k7 on 2014-10-17 16:40:16

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
I was shocked when I learned that it could not implement already for 4 years, even if
only for readonly mode.

Reported by june.universum on 2014-10-21 10:30:18

Member

lukeis commented Mar 2, 2016

I was shocked when I learned that it could not implement already for 4 years, even if
only for readonly mode.

Reported by june.universum on 2014-10-21 10:30:18

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
You can just ignore another users request, aka "please make headers accessible", but
you just have another user less. This is the open source version of https://www.youtube.com/watch?v=6wS5xOZ7Rq8
 I will neither use selenium+webdriver nor advertise it somewhere (because I wont use
it). I guess  you guys dont mind, but I now regret to have taken the 2 hours to get
everything set up with it and blogged it. There are alternatives and I am gonna use
them... for any other reference see #108/#109 

Creating fans the selenium way ... ;)

Reported by sebs@2xs.org on 2014-10-23 13:49:55

Member

lukeis commented Mar 2, 2016

You can just ignore another users request, aka "please make headers accessible", but
you just have another user less. This is the open source version of https://www.youtube.com/watch?v=6wS5xOZ7Rq8
 I will neither use selenium+webdriver nor advertise it somewhere (because I wont use
it). I guess  you guys dont mind, but I now regret to have taken the 2 hours to get
everything set up with it and blogged it. There are alternatives and I am gonna use
them... for any other reference see #108/#109 

Creating fans the selenium way ... ;)

Reported by sebs@2xs.org on 2014-10-23 13:49:55

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
I want to be able to set HTTP headers before clicking buttons in the interface.

Reported by asx34rtg.klagagagagag656 on 2014-10-26 12:37:55

Member

lukeis commented Mar 2, 2016

I want to be able to set HTTP headers before clicking buttons in the interface.

Reported by asx34rtg.klagagagagag656 on 2014-10-26 12:37:55

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
Another vote to expose the response code

Reported by Einevea on 2014-10-27 13:32:20

Member

lukeis commented Mar 2, 2016

Another vote to expose the response code

Reported by Einevea on 2014-10-27 13:32:20

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
I need to set headers to access our staging area, through firewalls, when using cloud
browsers.

Also our "users", when testing from outside firewall change their headers accordingly.
So as WebDriver should mimic users, I'd like it to mimic our test users.

Reported by mikael.x.wallin on 2014-10-28 13:48:01

Member

lukeis commented Mar 2, 2016

I need to set headers to access our staging area, through firewalls, when using cloud
browsers.

Also our "users", when testing from outside firewall change their headers accordingly.
So as WebDriver should mimic users, I'd like it to mimic our test users.

Reported by mikael.x.wallin on 2014-10-28 13:48:01

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
+1 for response codes etc, things have change dramatically since 2009, websites are
now webapps with a ton of functionality and browsers have changed dramatically why
not keep up with the changing times?

Reported by casey@lucidchart.com on 2014-10-29 15:04:36

Member

lukeis commented Mar 2, 2016

+1 for response codes etc, things have change dramatically since 2009, websites are
now webapps with a ton of functionality and browsers have changed dramatically why
not keep up with the changing times?

Reported by casey@lucidchart.com on 2014-10-29 15:04:36

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
vote to have the response code

Reported by salustsai on 2014-10-30 18:21:32

Member

lukeis commented Mar 2, 2016

vote to have the response code

Reported by salustsai on 2014-10-30 18:21:32

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member
One more vote to have http status and headers introspection.

Rationale:
1. Browsers can check that data via js code, so I do not see why the webdriver should
be more limited / not expose it
2. in todays restful world, http internals are much more important than they were 5
years ago
2. having to set up a tool such as a proxy or http client is quite a chore, whereas
selenium is today integrated transparently into testing suites such as behat, which
make writing tests much nicer and faster
3. it seems that user have been consistently requesting this feature for 5 years, with
no-one opposing it except core devs...

Reported by ggiunta@kaliop.com on 2014-11-18 11:42:01

Member

lukeis commented Mar 2, 2016

One more vote to have http status and headers introspection.

Rationale:
1. Browsers can check that data via js code, so I do not see why the webdriver should
be more limited / not expose it
2. in todays restful world, http internals are much more important than they were 5
years ago
2. having to set up a tool such as a proxy or http client is quite a chore, whereas
selenium is today integrated transparently into testing suites such as behat, which
make writing tests much nicer and faster
3. it seems that user have been consistently requesting this feature for 5 years, with
no-one opposing it except core devs...

Reported by ggiunta@kaliop.com on 2014-11-18 11:42:01

@lukeis

This comment has been minimized.

Show comment
Hide comment
@lukeis

lukeis Mar 2, 2016

Member

Reported by luke.semerau on 2014-11-18 19:27:09

  • Labels added: Restrict-AddIssueComment-Commit
Member

lukeis commented Mar 2, 2016

Reported by luke.semerau on 2014-11-18 19:27:09

  • Labels added: Restrict-AddIssueComment-Commit
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.