Implement #196 Add file downloading mechanism without "href" #267

Closed
wants to merge 2 commits into
from

Projects

None yet

3 participants

@dimand58
Contributor
dimand58 commented Jan 9, 2016

@asolntsev please review first version of implementation #196 issue.

How to use in end-user projects:

        Selenide.prepareDownload("C:\\SomeFolder", "", "application/x-sdlc");
        WebDriverRunner.getWebDriver().get("http://javadl.sun.com/webapps/download/AutoDL?BundleId=113227");
        // Downloaded file with name chromeinstall-8u66.exe
        File tmp = new File(Selenide.$(By.tagName("body")).getText());

or

        Selenide.prepareDownload("C:\\SomeFolder","SomeFile.exe","application/x-sdlc");
        WebDriverRunner.getWebDriver().get("http://javadl.sun.com/webapps/download/AutoDL?BundleId=113227");
        // Downloaded file with name SomeFile.exe
        File tmp = new File(Selenide.$(By.tagName("body")).getText());

or

        Selenide.$(By.id("button_for_file_download_after_click"))
                .prepareDownload("C:\\SomeFolder", "", "application/pdf")
                .click();
        File tmp = new File(Selenide.$(By.tagName("body")).getText());

TODO:

  1. Review
  2. Some tests
  3. Merge conflicts
dimand58 Fix PR #267 4de91f1
@dimand58
Contributor

Fix

@asolntsev
Member

@dimand58 Thank you! Great! It's almost what we need.

The only problem is the API. The new downloading mechanism is particularly needed in situation when we don't know exact URL. You typically fill some form with 100500 fields, click "submit", and file starts downloading.

Try for example:

  • http://idemo.bspb.ru/#corporate
  • Cards -> Acquiring -> "View terminals"
  • Put some checkboxes and click "Report"
  • Expected result: an XLS file starts downloading

I suggest to make existing method $.download() more powerful without adding any new methods.

This method:

  File report = $("button").download();

currently only works with elements <a href=***>.
What I suggest is to make it work with any downloads. It could work like this:

  1. Test calls $("button").download();
  2. Selenide sets some ThreadLocal flag like Selenide.browserMobProxyActive.set(true);
  3. Selenide clicks the button
  4. Because of flag, BMP stores all received files to some folder.
  5. Selenide method download returns first of these files.
  6. Selenide method download resets flag back: Selenide.browserMobProxyActive.set(false);
  7. While this flag is false, BMP does not store any files (to avoid degrading performance).
@dimand58
Contributor

@asolntsev 2 problem exists:

  1. BrowserMob must be activated BEFORE of WebDriver instance
  2. Web must know ALL mime types for files

Because of this problems my implementation looks more appropriative.

TODO:

  • add support of BMob configuration before initiating (chained proxy and etc)
@asolntsev
Member

@dimand58

  1. BrowserMob must be activated BEFORE of WebDriver instance
    Yes, but it's not a problem. Selenide will activate BrowserMob in the very beginning (before creating webdriver). But this BrowserMob instance will not save any files. It will just proxy requests from webdriver to server. Then, method $.download() will set some internal flag to TRUE, and only then BrowserMob will save all responses with header like "attachment" to local files.

  2. Web must know ALL mime types for files
    I suggest that Selenide can return the first saved file with header like "attachment".

With your implementation, use needs to know the exact URL for downloading file. But user cannot always know it. For example, file can be downloaded as a result of form submission.

@dimand58
Contributor

OK.
Response filter will be save file if HTTP response header have "Content-Disposition: attachment;". In this case no need to know MIME type of downloaded file.

@asolntsev But what about redirections?
Test case:

  1. Go to http://www.java.com/ru/download/chrome.jsp
  2. Click on BIG RED BUTTON (java dowload)

Actual:

  1. No Content-Disposition in response header
  2. 1 redirect occured (to http://sdlc-esd.oracle.com/ESD6/JSCDL/jdk/8u66-b18/chromeinstall-8u66.exe)
  3. AGAIN - no Content-Disposition in response header

I think Selenide must have option to catch files by Content-Type in response header too.

@mplushnikov

+1 for downloading files from elements without href attribute, like buttons.

@asolntsev
Member

@mplushnikov agree. We need this. I will work on it soon.

@asolntsev asolntsev self-assigned this Apr 3, 2016
@asolntsev asolntsev added this to the 3.6 milestone Apr 3, 2016
@asolntsev
Member
asolntsev commented Apr 20, 2016 edited

@dimand58 Sorry for the long delay. I am almost ready to merge the pull request, but I have one problem.

BMP doesn't work for me when I run tests locally (against http://127.0.0.1:9000). But it works for remote sites like google.com. See some comments here: http://automated-testing.info/t/browsermob-proxy-java-webdriver-pomogite-zapustit-prostejshij-test/4531/32

Don't you know a solution?

@dimand58
Contributor

@asolntsev , hi!
Give me a few days, I will try to solve this problem.

@dimand58
Contributor

@asolntsev
Are you tried to use DNS name of your PC instead of 127.0.0.1 ? I suppose that this trick can help

@asolntsev
Member

How can I use name of computer? I cannot hardcode it in tests - tests
should be easily runnable on any machine.
On Apr 21, 2016 1:25 AM, "Demin Dmitrii" notifications@github.com wrote:

@asolntsev https://github.com/asolntsev
Are you tried to use DNS name of your PC instead of 127.0.0.1 ? I suppose
that this trick can help


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#267 (comment)

@dimand58
Contributor

You can get current pc name in code, via some method maybe

@asolntsev
Member

Yes, something like InternetAddress.getLocal().getHost() should return name
of current computer. But it can be very slow, depending on network
configuration.

Why couldn't BMP just work with IP? Both Jetty an Netty can.
On Apr 22, 2016 12:03 PM, "Demin Dmitrii" notifications@github.com wrote:

You can get current pc name in code, via some method maybe


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#267 (comment)

@dimand58
Contributor
dimand58 commented Apr 22, 2016 edited

@asolntsev What does happen when your test opens http://127.0.0.1:9000 via proxy?
I tried on my PC (opened http://127.0.0.1/ Denwer URL) and all works great.

@asolntsev
Member

This is my case that does not work:

  1. open http://127.0.0.1:9000 via proxy
  2. download a file

Result: proxy intercepts all requests, but does not intercept server
response. So I cannot get the downloaded file.
Expected result: I want to get server response.

Andrei Solntsev

2016-04-22 19:03 GMT+03:00 Demin Dmitrii notifications@github.com:

@asolntsev https://github.com/asolntsev What does happen when your test
opens http://127.0.0.1:9000 via proxy?
I tried on my PC and all works great.


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#267 (comment)

@dimand58
Contributor
dimand58 commented Apr 22, 2016 edited

@asolntsev it is strange...
My steps:

  1. Init BMP before first WD creation
BrowserMobProxy proxy = new BrowserMobProxyServer();
 {
     proxy.start(1234);
     Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);
     WebDriverRunner.setProxy(seleniumProxy);
 }
  1. Add filter before URL opening:
    proxy.addResponseFilter(new ResponseFilterImpl());
  2. Open URL:
    Selenide.open("http://127.0.0.1");
  3. In debug mode I can see call of filterResponse method in ResponseFilterImpl
@dimand58
Contributor
dimand58 commented Apr 22, 2016 edited

Browser - Google Chrome
BMP:

<dependency>
        <groupId>net.lightbody.bmp</groupId>
        <artifactId>browsermob-core-littleproxy</artifactId>
        <version>2.1.0-beta-5</version>
</dependency>

Selenide 3.4

@asolntsev
Member

@dimand58 Please review my failing test. In Selenide master branch, there are 2 tests:

  • integration.proxy.BrowserMobProxyTest - it uses deprecated net.lightbody.bmp.proxy.ProxyServer - and it works
  • integration.proxy.BrowserMobProxyServerTest - it uses newer net.lightbody.bmp.BrowserMobProxyServer - and it doesn't work for localhost. Please take a look at test local_https() that is currently ignored.

@dimand58 Can you say why this test fails?

@dimand58
Contributor

@asolntsev , got it. I will investigate this test at evening today

@dimand58
Contributor
dimand58 commented Apr 25, 2016 edited

@asolntsev can you update your certificate? It can be one of reason of current problem.
image

I had similar problem on one of my project and it was solved via fixing of SSL certificate

@asolntsev
Member

Do you mean the SSL certificate of https://localhost:xxxx?

I deliberately use self-signed (untrusted) SSL certificate there because I
want to test how Selenide works with untrusted certs. It's because many
companies use untrusted certs on their test-servers.

Typically webdrivers have method like "setTrustCerts(true)", I guess BMP
also has such a method.

Andrei Solntsev

2016-04-25 17:57 GMT+03:00 Dmitriy Demin notifications@github.com:

@asolntsev https://github.com/asolntsev can you update your
certificate? It can be one of reason of current problem.
[image: image]
https://cloud.githubusercontent.com/assets/5477035/14788108/284bdfaa-0b0f-11e6-9f10-61bd22aa66a3.png


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#267 (comment)

@dimand58
Contributor

I created new self-signed cert (with valid expiration date and from/to fields 127.0.0.1) but problem still exists.
However, in IE11 page opens via proxy! I'm totaly confused

@dimand58
Contributor

@asolntsev
I've found and checked the solution - we should use latest version of BPM (beta-7-SNAPSHOT which not realised yet). With that BMP version local web works good.
I guess we should to wait a little bit

@asolntsev asolntsev modified the milestone: 3.6, 3.9 Sep 19, 2016
@asolntsev
Member

@dimand58 I implemented this idea a little bit differently in Selenide 3.9

Huge thanks for your ideas and support!

@asolntsev asolntsev closed this Sep 19, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment