No way to setup proxy for HtmlUnitDriver #80

Closed
sarxos opened this Issue May 9, 2012 · 4 comments

Projects

None yet

3 participants

@sarxos
sarxos commented May 9, 2012

When I setup and point them to some URLs, eg:

<configuration>
    <preloadSources>
        <source>http://ajax.googleapis.com/ajax/libs/jquery/1/jquery.min.js</source>
        <source>https://raw.github.com/documentcloud/underscore/master/underscore.js</source>
        <source>https://raw.github.com/documentcloud/backbone/master/backbone.js</source>
    </preloadSources>
</configuration>

There is no way to run mvn test when you are behind proxy. Proxy settings are being passed by:

$ mvn clean test -Dhttp.proxyHost=myproxyhost.com -Dhttp.proxyPort=8000

In such a case you will get:

[INFO] Executing Jasmine Specs
2012-05-09 15:54:24 com.gargoylesoftware.htmlunit.html.HtmlPage loadExternalJavaScriptFile
SEVERE: Error loading JavaScript from [http://ajax.googleapis.com/ajax/libs/jquery/1/jquery.min.js].
org.apache.http.conn.HttpHostConnectException: Connection to http://ajax.googleapis.com refused
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:158)
        at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:149)
        at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:573)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:425)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:776)
        at com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:152)
        at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1439)
        at com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1358)
        at com.gargoylesoftware.htmlunit.html.HtmlPage.loadJavaScriptFromUrl(HtmlPage.java:1008)
        at com.gargoylesoftware.htmlunit.html.HtmlPage.loadExternalJavaScriptFile(HtmlPage.java:950)
        at com.gargoylesoftware.htmlunit.html.HtmlScript.executeScriptIfNeeded(HtmlScript.java:349)
        at com.gargoylesoftware.htmlunit.html.HtmlScript$1.execute(HtmlScript.java:230)
        at com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:240)
        at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:598)
        at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
        at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:556)
        at org.cyberneko.html.HTMLTagBalancer.callEndElement(HTMLTagBalancer.java:1142)
        at org.cyberneko.html.HTMLTagBalancer.endElement(HTMLTagBalancer.java:1044)
        at org.cyberneko.html.filters.DefaultFilter.endElement(DefaultFilter.java:206)
        at org.cyberneko.html.filters.NamespaceBinder.endElement(NamespaceBinder.java:329)
        at org.cyberneko.html.HTMLScanner$ContentScanner.scanEndElement(HTMLScanner.java:3018)
        at org.cyberneko.html.HTMLScanner$ContentScanner.scan(HTMLScanner.java:2005)
        at org.cyberneko.html.HTMLScanner.scanDocument(HTMLScanner.java:908)
        at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:499)
        at org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:452)
        at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
        at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.parse(HTMLParser.java:789)
        at com.gargoylesoftware.htmlunit.html.HTMLParser.parse(HTMLParser.java:225)
        at com.gargoylesoftware.htmlunit.html.HTMLParser.parseHtml(HTMLParser.java:179)
        at com.gargoylesoftware.htmlunit.DefaultPageCreator.createHtmlPage(DefaultPageCreator.java:221)
        at com.gargoylesoftware.htmlunit.DefaultPageCreator.createPage(DefaultPageCreator.java:106)
        at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:433)
        at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:311)
        at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:373)
        at org.openqa.selenium.htmlunit.HtmlUnitDriver.get(HtmlUnitDriver.java:345)
        at org.openqa.selenium.htmlunit.HtmlUnitDriver.get(HtmlUnitDriver.java:332)
        at com.github.searls.jasmine.runner.SpecRunnerExecutor.execute(SpecRunnerExecutor.java:31)
        at com.github.searls.jasmine.TestMojo.executeSpecs(TestMojo.java:57)
        at com.github.searls.jasmine.TestMojo.run(TestMojo.java:35)
        at com.github.searls.jasmine.AbstractJasmineMojo.execute(AbstractJasmineMojo.java:233)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:319)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
Caused by: java.net.ConnectException: Connection timed out: connect
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
        at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
        at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
        at java.net.Socket.connect(Socket.java:529)
        at com.gargoylesoftware.htmlunit.SocksSocketFactory.connectSocket(SocksSocketFactory.java:89)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)
        ... 62 more

I believe this could be fixed in TestMojo by checking if proxy has been set:

private WebDriver createDriver() {
    ...
    String proxyHost = System.setProperty("http.proxyHost");
    String proxyPort = System.setProperty("http.proxyPort");
    ...
    // set proxy in web driver

HtmlUnitDriver::setProxy(String, int) would have to be used:

http://www.jarvana.com/jarvana/view/org/seleniumhq/selenium/selenium-htmlunit-driver/2.0a1/selenium-htmlunit-driver-2.0a1-javadoc.jar!/org/openqa/selenium/htmlunit/HtmlUnitDriver.html#setProxy%28java.lang.String,%20int%29

@searls
Owner
searls commented May 9, 2012

I'd consider this a low priority since it can almost always be easily
worked around by not relying on remote/CDN'd JavaScript files (it seems
like the venn diagram of "users who are interested in using public CDNs"
and "users who are behind an HTTP proxy" would have a very narrow
intersection.

If you're interested in opening a pull request, I'll consider it if you add
a Cucumber test demonstrating it works and update the unit tests.

Thanks,

Justin

On Wed, May 9, 2012 at 10:01 AM, Bartosz Firyn <
reply@reply.github.com

wrote:

When I setup and point them to some URLs, eg:

<configuration>
       <preloadSources>
               <source>
http://ajax.googleapis.com/ajax/libs/jquery/1/jquery.min.js</source>
               <source>
https://raw.github.com/documentcloud/underscore/master/underscore.js
</source>
               <source>
https://raw.github.com/documentcloud/backbone/master/backbone.js</source>
       </preloadSources>
</configuration>

There is no way to run mvn test when you are behind proxy. Proxy settings
are being passed by:

$ mvn clean test -Dhttp.proxyHost=myproxyhost.com -Dhttp.proxyPort=8000

In such a case you will get:

[INFO] Executing Jasmine Specs
2012-05-09 15:54:24 com.gargoylesoftware.htmlunit.html.HtmlPage
loadExternalJavaScriptFile
SEVERE: Error loading JavaScript from [
http://ajax.googleapis.com/ajax/libs/jquery/1/jquery.min.js].
org.apache.http.conn.HttpHostConnectException: Connection to
http://ajax.googleapis.com refused
       at
org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:158)
       at
org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:149)
       at
org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
       at
org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:573)
       at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:425)
       at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
       at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:776)
       at
com.gargoylesoftware.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:152)
       at
com.gargoylesoftware.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1439)
       at
com.gargoylesoftware.htmlunit.WebClient.loadWebResponse(WebClient.java:1358)
       at
com.gargoylesoftware.htmlunit.html.HtmlPage.loadJavaScriptFromUrl(HtmlPage.java:1008)
       at
com.gargoylesoftware.htmlunit.html.HtmlPage.loadExternalJavaScriptFile(HtmlPage.java:950)
       at
com.gargoylesoftware.htmlunit.html.HtmlScript.executeScriptIfNeeded(HtmlScript.java:349)
       at
com.gargoylesoftware.htmlunit.html.HtmlScript$1.execute(HtmlScript.java:230)
       at
com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:240)
       at
com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:598)
       at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown
Source)
       at
com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:556)
       at
org.cyberneko.html.HTMLTagBalancer.callEndElement(HTMLTagBalancer.java:1142)
       at
org.cyberneko.html.HTMLTagBalancer.endElement(HTMLTagBalancer.java:1044)
       at
org.cyberneko.html.filters.DefaultFilter.endElement(DefaultFilter.java:206)
       at
org.cyberneko.html.filters.NamespaceBinder.endElement(NamespaceBinder.java:329)
       at
org.cyberneko.html.HTMLScanner$ContentScanner.scanEndElement(HTMLScanner.java:3018)
       at
org.cyberneko.html.HTMLScanner$ContentScanner.scan(HTMLScanner.java:2005)
       at org.cyberneko.html.HTMLScanner.scanDocument(HTMLScanner.java:908)
       at
org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:499)
       at
org.cyberneko.html.HTMLConfiguration.parse(HTMLConfiguration.java:452)
       at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
       at
com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.parse(HTMLParser.java:789)
       at
com.gargoylesoftware.htmlunit.html.HTMLParser.parse(HTMLParser.java:225)
       at
com.gargoylesoftware.htmlunit.html.HTMLParser.parseHtml(HTMLParser.java:179)
       at
com.gargoylesoftware.htmlunit.DefaultPageCreator.createHtmlPage(DefaultPageCreator.java:221)
       at
com.gargoylesoftware.htmlunit.DefaultPageCreator.createPage(DefaultPageCreator.java:106)
       at
com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:433)
       at
com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:311)
       at
com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:373)
       at
org.openqa.selenium.htmlunit.HtmlUnitDriver.get(HtmlUnitDriver.java:345)
       at
org.openqa.selenium.htmlunit.HtmlUnitDriver.get(HtmlUnitDriver.java:332)
       at
com.github.searls.jasmine.runner.SpecRunnerExecutor.execute(SpecRunnerExecutor.java:31)
       at com.github.searls.jasmine.TestMojo.executeSpecs(TestMojo.java:57)
       at com.github.searls.jasmine.TestMojo.run(TestMojo.java:35)
       at
com.github.searls.jasmine.AbstractJasmineMojo.execute(AbstractJasmineMojo.java:233)
       at
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
       at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
       at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
       at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
       at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
       at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
       at
org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
       at
org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
       at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:319)
       at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
       at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
       at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
       at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
       at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:597)
       at
org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
       at
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
       at
org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
       at
org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
Caused by: java.net.ConnectException: Connection timed out: connect
       at java.net.PlainSocketImpl.socketConnect(Native Method)
       at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
       at
java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
       at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
       at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
       at java.net.Socket.connect(Socket.java:529)
       at
com.gargoylesoftware.htmlunit.SocksSocketFactory.connectSocket(SocksSocketFactory.java:89)
       at
org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)
       ... 62 more

I believe this could be fixed in TestMojo by checking if proxy has been
set:

private WebDriver createDriver() {
   ...
   String proxyHost = System.setProperty("http.proxyHost");
   String proxyPort = System.setProperty("http.proxyPort");
   ...
   // set proxy in web driver

HtmlUnitDriver::setProxy(String, int) would have to be used:

http://www.jarvana.com/jarvana/view/org/seleniumhq/selenium/selenium-htmlunit-driver/2.0a1/selenium-htmlunit-driver-2.0a1-javadoc.jar!/org/openqa/selenium/htmlunit/HtmlUnitDriver.html#setProxy%28java.lang.String,%20int%29


Reply to this email directly or view it on GitHub:
#80

@sarxos sarxos added a commit to sarxos/jasmine-maven-plugin that referenced this issue May 9, 2012
@sarxos sarxos Add possibility to specify proxy
closes #80
b9a0249
@sarxos
sarxos commented May 9, 2012

I never worked with cucumber, but I can assure you it works well. I tested it on my project and I'm behind proxy all the time. Also there is a problem with unit testing proxy settings since I would have to somehow mock proxy which is not trivial task. See the pull request - code is really simple. You can consider adding it to master, but I will understand if you won't do that - in such a case I will simply create new mvn profile so modified version of jasmine-maven-plugin will be used.

Debug output:

$ mvn test -Dhttp.proxyHost=myproxyserver.com -Dhttp.proxyPort=8000 -X
... specs runner html here ...
</html> to directory d:\usr\workspace\backbone.modelbinding\target\jasmine
[DEBUG] Setting proxy myproxyserver.com:8000
[INFO] 
-------------------------------------------------------
 J A S M I N E   S P E C S
-------------------------------------------------------
[INFO] 
checkbox convention bindings
...
@klieber klieber was assigned Jan 9, 2014
@klieber
Collaborator
klieber commented Jan 9, 2014

After pull request #200 is merged in and released you will be able to specify proxy settings using the webDriverCapabilities parameter.

For example:

<plugin>
  <groupId>com.github.searls</groupId>
  <artifactId>jasmine-maven-plugin</artifactId>
  <version>${jasmine-maven-plugin.version}</version>
  <executions>
    <execution>
      <goals>
        <goal>test</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <webDriverCapabilities>
      <capability>
        <name>proxy</name>
        <map>
          <httpProxy>myproxyserver.com:8000</httpProxy>
        </map>
      </capability>
    </webDriverCapabilities>
  </configuration>
</plugin>

If you want to be able to specify the parameters at the command line then just configure it like this:

<plugin>
  <groupId>com.github.searls</groupId>
  <artifactId>jasmine-maven-plugin</artifactId>
  <version>${jasmine-maven-plugin.version}</version>
  <executions>
    <execution>
      <goals>
        <goal>test</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <webDriverCapabilities>
      <capability>
        <name>proxy</name>
        <map>
          <httpProxy>${http.proxyHost}:${http.proxyPort}</httpProxy>
        </map>
      </capability>
    </webDriverCapabilities>
  </configuration>
</plugin>
@klieber klieber closed this in 5425526 Jan 9, 2014
@sarxos
sarxos commented Jan 9, 2014

Cool. This for sure will help users who are using proxy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment