Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Published History Grid tests #4502

Merged
merged 10 commits into from Aug 30, 2017

Conversation

Projects
None yet
5 participants
@anatskiy
Copy link
Contributor

commented Aug 28, 2017

As part of #4425, these Selenium tests check the following:

  • List of published histories
  • Standard search
  • Advanced search (by name, annotation, owner, tags)
  • Sorting by name and owner
  • Clicking on tags in the grid
@bgruening

This comment has been minimized.

Copy link
Member

commented Aug 28, 2017

xref: #4425

@mvdbeek mvdbeek referenced this pull request Aug 28, 2017

Closed

Prototype published history grid using Vue.js #4425

0 of 7 tasks complete
@bgruening

This comment has been minimized.

Copy link
Member

commented Aug 28, 2017

You can run the tests locally with: ./run_tests.sh -selenium test/selenium_tests/test_published_histories_grid.py. @jmchilton selenium tests are not yet running right? How should we proceed here?

@galaxybot galaxybot added this to the 17.09 milestone Aug 28, 2017

@dannon

This comment has been minimized.

Copy link
Member

commented Aug 28, 2017

@galaxybot test this

@dannon

This comment has been minimized.

Copy link
Member

commented Aug 28, 2017

This failed a few tests when I ran them locally, but that may be my selenium setup, I'm looking into it.

@bgruening

This comment has been minimized.

Copy link
Member

commented Aug 28, 2017

@dannon these new tests or the selenium tests in general. There are a few other history tests that are known to fail imho.

@dannon

This comment has been minimized.

Copy link
Member

commented Aug 28, 2017

@bgruening The new ones, I'm running just the 6 included here. They pass for you when running the command you list above, exactly, right?

@bgruening

This comment has been minimized.

Copy link
Member

commented Aug 28, 2017

Yes, with the chrome driver :(

@dannon

This comment has been minimized.

Copy link
Member

commented Aug 28, 2017

Ok, problem might be on my end, let me investigate more.

@jmchilton

This comment has been minimized.

Copy link
Member

commented Aug 28, 2017

Totally awesome - thanks a ton @anatskiy! They work for me locally:

----------------------------------------------------------------------
Ran 6 tests in 79.544s

OK

@dannon let me know if I can help track down your local problems - if yet do you have stack traces and/or which tests failed?

Update: I've done more testing and these tests do fail sometimes for me with that command. Some of it will be solved by using Postgres the way Jenkins does - I'm working on finding other sources of issues though. I already have some changes I'll push or open a PR for tomorrow.

@bgruening

This comment has been minimized.

Copy link
Member

commented Aug 29, 2017

@jmchilton @dannon it could be the timeouts/sleep? We saw some problems with other tests, where we needed to increase the sleep-time. Maybe this is the problem here as well? I'm not sure how we can get rid of these, it feels error prone in general.

@jmchilton

This comment has been minimized.

Copy link
Member

commented Aug 29, 2017

I think the sleeps are a bit aggressive in general here - we should wait for elements to be clickable using Selenium abstractions rather then waiting for them to be visible then pausing a bit before clicking. The following change does that and few other things... jmchilton@08bb683. This test is tricky in that it relies on global Galaxy state - so it couldn't be applied against an external Galaxy server more than once - the rest of the Selenium tests can. That is fine I think - it is kind of the nature of "published".

@anatskiy

This comment has been minimized.

Copy link
Contributor Author

commented Aug 29, 2017

@jmchilton thank you very much for the enhancements!

@dannon

This comment has been minimized.

Copy link
Member

commented Aug 29, 2017

@jmchilton I've tried a bunch of stuff so far. The most promising were chromedriver non-headless via GALAXY_TEST_SELENIUM_BROWSER=CHROME GALAXY_TEST_SELENIUM_HEADLESS=0 ./run_tests.sh -selenium test/selenium_tests/test_published_histories_grid.py > selenium_log 2>&1. I could actually see that chrome said "Chrome is being controlled by automated test software", and it was attempting to run the tests. Log from that is here: https://gist.github.com/1349ce69ed1dbce49fb0c120b0463eec

How are you running these on your macbook? I tried the docker container but kept getting timeouts, I'm guessing due to wonky osx docker network handling (the ip being that of the virtualbox, instead of localhost, etc). (happy to recreate a log if this, too, if you want to see it)

@jmchilton

This comment has been minimized.

Copy link
Member

commented Aug 29, 2017

@anatskiy Awesome - can you cherry pick that commit into this branch?

@dannon Yeah - I've never been able to get the networking to work on a Macbook so I am using chromedriver - Galaxy just finds that automatically if it is on the path and you haven't specified anything else. My current version is ChromeDriver 2.29.461585. I don't run it in headless mode generally and I haven't tested this in headless mode.

@dannon

This comment has been minimized.

Copy link
Member

commented Aug 29, 2017

Hrmm. I'm using ChromeDriver 2.31.488774. Does anything in that log look suspicious to you?

@jmchilton

This comment has been minimized.

Copy link
Member

commented Aug 29, 2017

@dannon Thanks for the log - it does look like at least some part of the Selenium interaction is working since you are able to register and such - it is failing to create histories in the first step so all future steps are broken also. I'll think about this some more - nothing is obvious to me. Is it possible you have slightly different JS being served because of uncommitted files present in your Galaxy root from you ES6 branch?

Update: I'll also try updating my chromedriver and see if that breaks my build.

@dannon

This comment has been minimized.

Copy link
Member

commented Aug 29, 2017

@jmchilton Shouldn't be, but that's a good idea, I'll double-check.

Update: I checked on a completely fresh, separate clone. Same error(s) for me :/

Rework published histories selenium test a bit.
- Wait for elements to be clickable before clicking them.
- Allow individual tests to be run on their own and out of order.
- Use randomly generated emails - the tests still can't really be applied to an external Galaxy test server but this gets us closer.
@anatskiy

This comment has been minimized.

Copy link
Contributor Author

commented Aug 29, 2017

@jmchilton I cherry-picked the changes, thanks a lot!

@jmchilton

This comment has been minimized.

Copy link
Member

commented Aug 29, 2017

@dannon I just upgraded chromedriver and it still worked - so it isn't that. Let me think about this...

@jmchilton

This comment has been minimized.

Copy link
Member

commented Aug 29, 2017

@dannon Want to wipe out database/test_errors, rerun just the first test, and post the screenshot that will deposited into a sub-directory of that directory? That should yield some clues maybe...

./run_tests.sh -selenium test/selenium_tests/test_published_histories_grid.py:HistoryGridTestCase.test_history_grid_histories
@dannon

This comment has been minimized.

Copy link
Member

commented Aug 30, 2017

@jmchilton: went down the rabbithole, ended up being several things to fix on my end, last of which was sqlite locking during testing (my postgres dburi env var had been dropped out of my profile...). This works for me now when testing locally, thanks for all the patience here, and for the new tests @anatskiy!

@dannon dannon merged commit f3b8a12 into galaxyproject:dev Aug 30, 2017

1 of 2 checks passed

continuous-integration/travis-ci/pr The Travis CI build failed
Details
lgtm analysis: JavaScript No alert changes
Details

@anatskiy anatskiy deleted the anatskiy:published_history_test branch Oct 27, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.