Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes for core.galaxy_ui.yaml. #3206

Merged
merged 1 commit into from Nov 21, 2016

Conversation

Projects
None yet
2 participants
@jmchilton
Copy link
Member

commented Nov 21, 2016

Neither of these prevent the tour from working but I think they are bugs. The extra click isn't an elemment that is clickable at that point in the tour and the referenced id isn't available so nothing in particular is tagged in the UI at that point.

Fixes for core.galaxy_ui.yaml.
Neither of these prevent the tour from working but I think they are bugs. The extra click isn't an elemment that is clickable at that point in the tour and the referenced id isn't available so nothing in particular is tagged in the UI at that point.

@jmchilton jmchilton added this to the 17.01 milestone Nov 21, 2016

@martenson martenson merged commit 097300e into galaxyproject:dev Nov 21, 2016

1 of 4 checks passed

api test Test started.
Details
continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
framework test Test started.
Details
toolshed test Build finished. 580 tests run, 0 skipped, 0 failed.
Details

jmchilton added a commit to jmchilton/galaxy that referenced this pull request Nov 22, 2016

A second round of selenium tests.
- Replace broken anonymous history CasperJS tests with working Selenium tests.
- Replace broken upload tests with working Selenium tests.
- Implement tour testing via Selenium (walk the two working stock tours and verify elements avaiable and clicks are valid). This is how I discovered galaxyproject#3206.

jmchilton added a commit to jmchilton/galaxy that referenced this pull request Nov 23, 2016

A second round of selenium tests.
- Add completely new workflow GUI test (basic creation from URL and in editor).
- Replace broken anonymous history CasperJS tests with working Selenium tests.
- Replace broken upload tests with working Selenium tests.
- Implement tour testing via Selenium (walk the two working stock tours and verify elements avaiable and clicks are valid). This is how I discovered galaxyproject#3206.

jmchilton added a commit to jmchilton/galaxy that referenced this pull request Nov 28, 2016

A second round of selenium tests.
- Add completely new workflow GUI test (basic creation from URL and in editor).
- Replace broken anonymous history CasperJS tests with working Selenium tests.
- Replace broken upload tests with working Selenium tests.
- Implement tour testing via Selenium (walk the two working stock tours and verify elements avaiable and clicks are valid). This is how I discovered galaxyproject#3206.

jmchilton added a commit to jmchilton/galaxy that referenced this pull request Nov 29, 2016

A second round of selenium tests.
- Add completely new workflow GUI test (basic creation from URL and in editor).
- Replace broken anonymous history CasperJS tests with working Selenium tests.
- Replace broken upload CasperJS tests with working Selenium tests.
- Replace history options CasperJS tests with Selenium tests.
- Implement tour testing via Selenium (walk the two working stock tours and verify elements avaiable and clicks are valid). This is how I discovered galaxyproject#3206.

jmchilton added a commit to jmchilton/galaxy that referenced this pull request Nov 30, 2016

Replace CasperJS browser tests with Selenium tests.
See documentation added to run_tests.sh. This can be configured to target local web browsers or Selenium remote services as detailed in that script.

Individual tests can be executed with nosetests directly or groups of tests can share a common test Galaxy spin up when using ./run_tests.sh. If GALAXY_TEST_EXTERNAL=<url> is set - it will be respected and Galaxy will not be spun up (remember the URL needs to be reachable from the Selenium server inside the Docker container).

Every test failure writes a current screenshot of Galaxy to database/test_errors.

I will be honest that the desire to move toward selenium is based solely on failing to get the Casper tests to pass consistently due to bugs causing segfaults in the underlying tools (probably casperjs or phantomjs?). Over the past year I have tried multiple versions of dependencies, etc... and it never works out for me. Likewise Martin has never been able to get the tests to run consistently under Jenkins. I don't think Python or JavaScript is inherently better, this wasn't based on a personal preference about what kind of test I want to write.

Despite this being the primary reason, there are clear benefits to Selenium. It tests the actual web browsers we support, generates screenshots, has better documented and more robust tooling, can scale across clusters. Less clear, but certainly still a benefit of being Python based is that it fits with the rest of the test framework more cleanly than CasperJS.

Finally Carl's last words on CasperJS were "Ditch it".

The following specific tests were added or replaced:

- Implemented tour testing via Selenium (walk the two working stock tours and verify elements avaiable and clicks are valid). This is how I discovered galaxyproject#3206.
- Added completely new workflow GUI test (basic creation from URL and in editor).
- Replaced broken CasperJS registeration tests (text and expectations now wrong) with a Selenium variant that works against dev.
- Replaced broken anonymous history CasperJS tests with working Selenium tests.
- Replaced broken upload CasperJS tests with working Selenium tests.
- Replaced history options CasperJS tests with Selenium tests.
- Replaced login CasperJS tests with Selenium tests.
- Replaced history-share-tests.js with test_history_options.py
- Replaced history-panel-tests.js with test_history_panel.py
- Replaced a big part of hda-state-tests.js with test_history_dataset_state.py

I'm confident these utilities represent a sharable and higher-level abstraction around functional testing of Galaxy that can be used outside of Galaxy's testing framework. So a subset of the functional test stuff is in a separate module with minimal dependencies that I intend to make stand alone and pip installable like galaxy-lib.

This module consists of:

- Sizzle stuff in its own package. This code allows Selenium to reason with jQuery selectors instead of vanilla CSS selectors.
- Functionality for creating a Selenium driver and virtual display.
- The ``HasDriver`` mixin - this provides higher level navigation utilities not specifically tied to Galaxy.
- A package with abstractions for navigating Galaxy. This provides a higher-level interfactor for things such as logging in and out, registering a user, navigating menus, fetching Galaxy style tooltips and error messages, and walking Galaxy tours.

In order to demonstrate this new module is useful outside of Galaxy tests - I've included a CLI package to ease building argparse utilities around these abstractions and implemented a simple demonstration script that walks a Galaxy tour and dumps screenshots of Galaxy at each step to a folder. This is more of a demonstration of the utilities than an actual polished end user tool - though it might be helpful in linting and debugging tours.

In the future I hope to extend this scripting to implement

- Periodic deployment testing to ensure things like Jupyter work in production.
- A best practice recipe for Galaxy QA testing by deployers.
- An utility to generate dozens of screenshots of various aspects of Galaxy so we can quickly visually inspect ever part of the GUI before big releases.

xref galaxyproject/starforge#115 xref galaxyproject#1419

jmchilton added a commit to jmchilton/galaxy that referenced this pull request Nov 30, 2016

Replace CasperJS browser tests with Selenium tests.
See documentation added to run_tests.sh. This can be configured to target local web browsers or Selenium remote services as detailed in that script.

Individual tests can be executed with nosetests directly or groups of tests can share a common test Galaxy spin up when using ./run_tests.sh. If GALAXY_TEST_EXTERNAL=<url> is set - it will be respected and Galaxy will not be spun up (remember the URL needs to be reachable from the Selenium server inside the Docker container).

Every test failure writes a current screenshot of Galaxy to database/test_errors.

I will be honest that the desire to move toward selenium is based solely on failing to get the Casper tests to pass consistently due to bugs causing segfaults in the underlying tools (probably casperjs or phantomjs?). Over the past year I have tried multiple versions of dependencies, etc... and it never works out for me. Likewise Martin has never been able to get the tests to run consistently under Jenkins. I don't think Python or JavaScript is inherently better, this wasn't based on a personal preference about what kind of test I want to write.

Despite this being the primary reason, there are clear benefits to Selenium. It tests the actual web browsers we support, generates screenshots, has better documented and more robust tooling, can scale across clusters. Less clear, but certainly still a benefit of being Python based is that it fits with the rest of the test framework more cleanly than CasperJS.

Finally Carl's last words on CasperJS were "Ditch it".

The following specific tests were added or replaced:

- Implemented tour testing via Selenium (walk the two working stock tours and verify elements avaiable and clicks are valid). This is how I discovered galaxyproject#3206.
- Added completely new workflow GUI test (basic creation from URL and in editor).
- Replaced broken CasperJS registeration tests (text and expectations now wrong) with a Selenium variant that works against dev.
- Replaced broken anonymous history CasperJS tests with working Selenium tests.
- Replaced broken upload CasperJS tests with working Selenium tests.
- Replaced history options CasperJS tests with Selenium tests.
- Replaced login CasperJS tests with Selenium tests.
- Replaced history-share-tests.js with test_history_options.py
- Replaced history-panel-tests.js with test_history_panel.py
- Replaced a big part of hda-state-tests.js with test_history_dataset_state.py

I'm confident these utilities represent a sharable and higher-level abstraction around functional testing of Galaxy that can be used outside of Galaxy's testing framework. So a subset of the functional test stuff is in a separate module with minimal dependencies that I intend to make stand alone and pip installable like galaxy-lib.

This module consists of:

- Sizzle stuff in its own package. This code allows Selenium to reason with jQuery selectors instead of vanilla CSS selectors.
- Functionality for creating a Selenium driver and virtual display.
- The ``HasDriver`` mixin - this provides higher level navigation utilities not specifically tied to Galaxy.
- A package with abstractions for navigating Galaxy. This provides a higher-level interfactor for things such as logging in and out, registering a user, navigating menus, fetching Galaxy style tooltips and error messages, and walking Galaxy tours.

In order to demonstrate this new module is useful outside of Galaxy tests - I've included a CLI package to ease building argparse utilities around these abstractions and implemented a simple demonstration script that walks a Galaxy tour and dumps screenshots of Galaxy at each step to a folder. This is more of a demonstration of the utilities than an actual polished end user tool - though it might be helpful in linting and debugging tours.

In the future I hope to extend this scripting to implement

- Periodic deployment testing to ensure things like Jupyter work in production.
- A best practice recipe for Galaxy QA testing by deployers.
- An utility to generate dozens of screenshots of various aspects of Galaxy so we can quickly visually inspect ever part of the GUI before big releases.

xref galaxyproject/starforge#115 xref galaxyproject#1419

jmchilton added a commit to jmchilton/galaxy that referenced this pull request Dec 2, 2016

Replace CasperJS browser tests with Selenium tests.
See documentation added to run_tests.sh. This can be configured to target local web browsers or Selenium remote services as detailed in that script.

Individual tests can be executed with nosetests directly or groups of tests can share a common test Galaxy spin up when using ./run_tests.sh. If GALAXY_TEST_EXTERNAL=<url> is set - it will be respected and Galaxy will not be spun up (remember the URL needs to be reachable from the Selenium server inside the Docker container).

Every test failure writes a current screenshot of Galaxy to database/test_errors.

I will be honest that the desire to move toward selenium is based solely on failing to get the Casper tests to pass consistently due to bugs causing segfaults in the underlying tools (probably casperjs or phantomjs?). Over the past year I have tried multiple versions of dependencies, etc... and it never works out for me. Likewise Martin has never been able to get the tests to run consistently under Jenkins. I don't think Python or JavaScript is inherently better, this wasn't based on a personal preference about what kind of test I want to write.

Despite this being the primary reason, there are clear benefits to Selenium. It tests the actual web browsers we support, generates screenshots, has better documented and more robust tooling, can scale across clusters. Less clear, but certainly still a benefit of being Python based is that it fits with the rest of the test framework more cleanly than CasperJS.

Finally Carl's last words on CasperJS were "Ditch it".

The following specific tests were added or replaced:

- Implemented tour testing via Selenium (walk the two working stock tours and verify elements avaiable and clicks are valid). This is how I discovered galaxyproject#3206.
- Added completely new workflow GUI test (basic creation from URL and in editor).
- Replaced broken CasperJS registeration tests (text and expectations now wrong) with a Selenium variant that works against dev.
- Replaced broken anonymous history CasperJS tests with working Selenium tests.
- Replaced broken upload CasperJS tests with working Selenium tests.
- Replaced history options CasperJS tests with Selenium tests.
- Replaced login CasperJS tests with Selenium tests.
- Replaced history-share-tests.js with test_history_options.py
- Replaced history-panel-tests.js with test_history_panel.py
- Replaced a big part of hda-state-tests.js with test_history_dataset_state.py

I'm confident these utilities represent a sharable and higher-level abstraction around functional testing of Galaxy that can be used outside of Galaxy's testing framework. So a subset of the functional test stuff is in a separate module with minimal dependencies that I intend to make stand alone and pip installable like galaxy-lib.

This module consists of:

- Sizzle stuff in its own package. This code allows Selenium to reason with jQuery selectors instead of vanilla CSS selectors.
- Functionality for creating a Selenium driver and virtual display.
- The ``HasDriver`` mixin - this provides higher level navigation utilities not specifically tied to Galaxy.
- A package with abstractions for navigating Galaxy. This provides a higher-level interfactor for things such as logging in and out, registering a user, navigating menus, fetching Galaxy style tooltips and error messages, and walking Galaxy tours.

In order to demonstrate this new module is useful outside of Galaxy tests - I've included a CLI package to ease building argparse utilities around these abstractions and implemented a simple demonstration script that walks a Galaxy tour and dumps screenshots of Galaxy at each step to a folder. This is more of a demonstration of the utilities than an actual polished end user tool - though it might be helpful in linting and debugging tours.

In the future I hope to extend this scripting to implement

- Periodic deployment testing to ensure things like Jupyter work in production.
- A best practice recipe for Galaxy QA testing by deployers.
- An utility to generate dozens of screenshots of various aspects of Galaxy so we can quickly visually inspect ever part of the GUI before big releases.

xref galaxyproject/starforge#115 xref galaxyproject#1419
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.