Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"core" and "data_set_manager" tests can't run separately #1353

Closed
mccalluc opened this issue Aug 22, 2016 · 7 comments
Closed

"core" and "data_set_manager" tests can't run separately #1353

mccalluc opened this issue Aug 22, 2016 · 7 comments

Comments

@mccalluc
Copy link
Member

Commit: f6b1867

Steps to reproduce

$ for TEST in `find . | grep tests.py\$`; do ./manage.py test $TEST && echo "PASS $TEST" >> /tmp/tests.txt || echo "FAIL $TEST" >> /tmp/tests.txt; done
$ cat /tmp/tests.txt

Observed behavior

PASS ./analysis_manager/tests.py
PASS ./annotation_server/tests.py
FAIL ./core/tests.py
FAIL ./data_set_manager/tests.py
PASS ./file_server/tests.py
PASS ./file_store/tests.py
PASS ./galaxy_connector/tests.py
PASS ./visualization_manager/tests.py
PASS ./workflow_manager/tests.py

Looking at the fuller output, both are failing with django.contrib.contenttypes.models.DoesNotExist: ContentType matching query does not exist

Expected behavior

Tests pass, even when run one-by-one.

@mccalluc
Copy link
Member Author

Looking more closely at the stack traces for both failures, they both end with:

  File "/vagrant/refinery/core/models.py", line 2184, in _add_user_to_neo4j
    'core.read_dataset'
  File "/home/vagrant/.virtualenvs/refinery-platform/local/lib/python2.7/site-packages/guardian/shortcuts.py", line 513, in get_objects_for_group
    permission__codename=codename)
  File "/home/vagrant/.virtualenvs/refinery-platform/local/lib/python2.7/site-packages/django/db/models/manager.py", line 151, in get
    return self.get_queryset().get(*args, **kwargs)
  File "/home/vagrant/.virtualenvs/refinery-platform/local/lib/python2.7/site-packages/django/db/models/query.py", line 310, in get
    self.model._meta.object_name)
django.contrib.contenttypes.models.DoesNotExist: ContentType matching query does not exist.

The reference to models.py is the only part of the stack which is not in site-packages.

(Found some advice at SO that this is usually cause by fixtures referencing models which don't exist, but deleting them didn't change the behavior here, for better or worse.)

@mccalluc
Copy link
Member Author

If I change the code to

            lambda ds: ds.uuid, get_objects_for_group(
                ExtendedGroup.objects.public_group(),
                'core.read_dataset_CLEARLY_WRONG'
            )

It fails in the same way... So it's having trouble mapping the permissions recorded in the DB to a Model class?

@scottx611x
Copy link
Member

scottx611x commented Aug 23, 2016

@mccalluc are you running these tests on the vm or locally?

@mccalluc
Copy link
Member Author

@scottx611x : This is in the VM... I think that's the right place to run?

@scottx611x
Copy link
Member

@mccalluc yep you're correct. Maybe we can try to iron this out when we talk about IGV stuff later?

@mccalluc
Copy link
Member Author

Scott was able to reproduce the bug, and you can also see the problem in the IDE's test runner, if you try to run just core or data_set_manager. Tests are dependent on the order in which they are run. We believe the underlying problem is the same as #1146, which isn't easy.

@scottx611x scottx611x self-assigned this Oct 19, 2016
scottx611x added a commit that referenced this issue Feb 28, 2017
* Testing the feasibility of running multiple travis builds over different browsers/platforms

* Wrap arg in quotes as to not evaluate spaces in env var

* Try out an extra wait

* I think testing Safari on anything other than OS X is a stretch

* See if bumping OS X v. down helps

* See if some extra waiting helps (I think Sauce lab's OS X machines are especially slow)

* Move helper functions to utils file

* Switch back o El Capitan because i believe timing was the only issue

* move helper functions

* Test adding some longer waits

* Fix whitespace

* fix imports

* Comment out file upload test for now

* Add `cleanup_on_error()` that we can decorate test methods with

* Tidy up tests

* Remove windows from acceptance testing

* Move acceptance test running into own shell script and call from within travis

* `cd` into the proper directory

* `cd` back to `refinery` directory

* Cleanup some more tests

* Add id to aid testing

* Add id to aid testing

* Cleanup login method

* Abstract out reused login moethod

* Utilize relative import

* PEP8

* only run acceptance tests if we detect a build for a PR

* Always run our acceptance tests

* Add acceptance tests for user registration and login

* Don't use separate shell script (for now)

* Remove `cd` comands

* Refactor selenium utility functions

* Add script to run acceptance tests on vagrant or travis

* Lower `DEFAULT_WAIT` Now that we aren't relying on SauceLabs VMs

* Cleanup setting of breakpoints, and refactor `cleanup_on_error` decorator

* Remove unnecessary env var, and re-add `global` as to ensure a single build

* Sauce_connect isn't needed

* pip caching seems stable

* PEP8

* Fail shell script if any step doesn't work properly

* GeckoDriver expects a more recent version of Firefox

* Remove Comments

* Update Default wait time

* Bump pytest-selenium version

* Update Comment

* Utilize `wait_until_id_clickable`

* Add tests for global analysis popover

* Add tests for Datasets panel

* Update comment

* Add a status to all generated Analyses

* Update stats test to check for actual object counts

* include gecko driver in our build process for ease of local selenium-based testing

* Refactor old acceptance test implementation in favor of simplicity and uniformness with our current Django unit tests

* Make virtual display bigger

* Update selenium testing utils

* Update assert body text method

* Update selenium tests

* Update `wait_until_id_visible()`

* bump Factory-Boy requirement

* Update travis as mentioned issue has been closed

* Clean up gecko driver Puppet install

* Update docstrings

* Remove unnecessary tests

* more geckodriver install cleanup

* Change Default wait

* update wait_until_id_visible()

* pull out duplicated code into ui-deletion helper method

* Update acceptance tests

* Minor changes due to abstraction of deletion code

* Fix typo

* guest user creation isn't needed here anymore

* Merge branch scottx611x/django_1.7_upgrade into scottx611x/acceptance_testing

* Flip the order of some installed apps due to Django 1.7 ContentType loading

* Decorate function as to not test 3rd party code's functionality

* Utilize StaticLiveServerTestCase

* Update delete_from_ui() helper

* Refactor decorator name to be more verbose

* Don't need to take a screenshot here

* Add xvfb and python3 to travis

* add pyvirtualdisplay req

* Rename geckodriver class to selenium

* Update order

* Update order of INSTALLED_APPS for testing purposes

* Update logging levels

* Update TEST_RUNNER per Django system-check suggestion

* Update tests

* Fix `get_or_create()`

* Access actual object returned from `get_or_create()` tuple

* Update logger statement

* Do not need to specify `live server` here since it has a default value

* Update string formatting to work properly

* Update isa/pre-isa archive deletion to produce less unnecessary errors

* Update Tests

* Manually save public group to sync db between threads (Specific to use of StaticLiveServerTestCase)

* Update `DEFAULT_WAIT`

* Share Datasets upon factory_boy creation

* Create public group for APIV2 test case

* Added group creation to the wrong TestCase

* Remove `django-nose` in favor of Django's new: `DiscoverTestRunner`

* This file shouldn't exist anymore. Probably wasn't caught in the merge here: 0c727cc

* Include ontology fixture content inside of a Data migration (#1581)

* Update puppet-archive version

* Utilize `puppet-archive` and clean up neo4j.pp

* Cleanup selenium manifest

* Factor out solr class from init.pp

* don't need to chmod here

* We are now using StaticLiveServerTestCase

* "Users of the `puppet-archive`module are responsible for archive package dependencies"

* Remove unused var

* Point to new archive download location

* Fix string formatting error

* Add `TEST_RUNNER` setting to base.py so that someone testing on `prod` locally could also benefeit

* update order of `INSTALLED_APPS` addresses: #1353

* Refactor all tests that interact with the ORM to inherit from `TransactionTestCase` (removes need for crazy tearDown()'s)

* DEFAULT_WAIT Shouldn't be that long

* Manually save `public group` as to persist data between test and web driver threads

* Update comment

* Don't need to optimize sole's index in tests

* Inherit from TransactionTestCase here

* `20` was actually a good fail-safe if things get slow for one reason or another

* switch order of INSTALLED APPS

* Cleanup selenium test cases

* Remove other `unittest` usages and unnecessary tests

* Override setUp() in a cleaner manner

* Add a travis fold

* Clean up FileStoreitem and InvestigationLink error handling

* Clean up error handling for isa & pre-isa archives

* Update comments

* Cleanup factory_boy factories and utils

* This is already a default value

* DRY

* Update existing DataMigrations to a more easily manageable format

* More robust Data migrations

* Remove `pk`

* RunPython is already encapsulated in a transaction

* refactor `get_isa_archive()` & `get_pre_isa_archive()`

* Analysis.status isn't necessary for testing

* Add Error Handling to `SeleniumTestBase.setUp()`

* Add `TEST_NON_SERIALIZED_APPS` setting due to pre Django 1.9 bug

* Don't need to share datasets pubilcly in acceptance tests

* Utilize `serialized_rollback` to ensure migration data persists in StaticLiveServerTestCases

* Refactor isa/pre_isa archive fetching into one method

* Remove other unnecessary instances of `tearDown()`

* Update method signature

* Update comment

* Fix Test

* Revert "Fix Test"

This reverts commit 673827a.

* Don't bundle (pre)isa_archive fetching functionality & Utilize existing `get_investigation()`

* Handle potential AttributeErrors that could occcur

* Old form was a better representation of our overall usecase

* Reorder import
scottx611x added a commit that referenced this issue Mar 21, 2017
* Testing the feasibility of running multiple travis builds over different browsers/platforms

* Wrap arg in quotes as to not evaluate spaces in env var

* Try out an extra wait

* I think testing Safari on anything other than OS X is a stretch

* See if bumping OS X v. down helps

* See if some extra waiting helps (I think Sauce lab's OS X machines are especially slow)

* Move helper functions to utils file

* Switch back o El Capitan because i believe timing was the only issue

* move helper functions

* Test adding some longer waits

* Fix whitespace

* fix imports

* Comment out file upload test for now

* Add `cleanup_on_error()` that we can decorate test methods with

* Tidy up tests

* Remove windows from acceptance testing

* Move acceptance test running into own shell script and call from within travis

* `cd` into the proper directory

* `cd` back to `refinery` directory

* Cleanup some more tests

* Add id to aid testing

* Add id to aid testing

* Cleanup login method

* Abstract out reused login moethod

* Utilize relative import

* PEP8

* only run acceptance tests if we detect a build for a PR

* Always run our acceptance tests

* Add acceptance tests for user registration and login

* Don't use separate shell script (for now)

* Remove `cd` comands

* Refactor selenium utility functions

* Add script to run acceptance tests on vagrant or travis

* Lower `DEFAULT_WAIT` Now that we aren't relying on SauceLabs VMs

* Cleanup setting of breakpoints, and refactor `cleanup_on_error` decorator

* Remove unnecessary env var, and re-add `global` as to ensure a single build

* Sauce_connect isn't needed

* pip caching seems stable

* PEP8

* Fail shell script if any step doesn't work properly

* GeckoDriver expects a more recent version of Firefox

* Remove Comments

* Update Default wait time

* Bump pytest-selenium version

* Update Comment

* Utilize `wait_until_id_clickable`

* Add tests for global analysis popover

* Add tests for Datasets panel

* Update comment

* Add a status to all generated Analyses

* Update stats test to check for actual object counts

* include gecko driver in our build process for ease of local selenium-based testing

* Refactor old acceptance test implementation in favor of simplicity and uniformness with our current Django unit tests

* Make virtual display bigger

* Update selenium testing utils

* Update assert body text method

* Update selenium tests

* Update `wait_until_id_visible()`

* bump Factory-Boy requirement

* Update travis as mentioned issue has been closed

* Clean up gecko driver Puppet install

* Update docstrings

* Remove unnecessary tests

* more geckodriver install cleanup

* Change Default wait

* update wait_until_id_visible()

* pull out duplicated code into ui-deletion helper method

* Update acceptance tests

* Minor changes due to abstraction of deletion code

* Fix typo

* guest user creation isn't needed here anymore

* Merge branch scottx611x/django_1.7_upgrade into scottx611x/acceptance_testing

* Flip the order of some installed apps due to Django 1.7 ContentType loading

* Decorate function as to not test 3rd party code's functionality

* Utilize StaticLiveServerTestCase

* Update delete_from_ui() helper

* Refactor decorator name to be more verbose

* Don't need to take a screenshot here

* Add xvfb and python3 to travis

* add pyvirtualdisplay req

* Rename geckodriver class to selenium

* Update order

* Update order of INSTALLED_APPS for testing purposes

* Update logging levels

* Update TEST_RUNNER per Django system-check suggestion

* Update tests

* Fix `get_or_create()`

* Access actual object returned from `get_or_create()` tuple

* Update logger statement

* Do not need to specify `live server` here since it has a default value

* Update string formatting to work properly

* Update isa/pre-isa archive deletion to produce less unnecessary errors

* Update Tests

* Manually save public group to sync db between threads (Specific to use of StaticLiveServerTestCase)

* Update `DEFAULT_WAIT`

* Share Datasets upon factory_boy creation

* Create public group for APIV2 test case

* Added group creation to the wrong TestCase

* Remove `django-nose` in favor of Django's new: `DiscoverTestRunner`

* This file shouldn't exist anymore. Probably wasn't caught in the merge here: 0c727cc

* Include ontology fixture content inside of a Data migration (#1581)

* Update puppet-archive version

* Utilize `puppet-archive` and clean up neo4j.pp

* Cleanup selenium manifest

* Factor out solr class from init.pp

* don't need to chmod here

* We are now using StaticLiveServerTestCase

* "Users of the `puppet-archive`module are responsible for archive package dependencies"

* Remove unused var

* Point to new archive download location

* Fix string formatting error

* Add `TEST_RUNNER` setting to base.py so that someone testing on `prod` locally could also benefeit

* update order of `INSTALLED_APPS` addresses: #1353

* Refactor all tests that interact with the ORM to inherit from `TransactionTestCase` (removes need for crazy tearDown()'s)

* DEFAULT_WAIT Shouldn't be that long

* Manually save `public group` as to persist data between test and web driver threads

* Update comment

* Don't need to optimize sole's index in tests

* Inherit from TransactionTestCase here

* `20` was actually a good fail-safe if things get slow for one reason or another

* switch order of INSTALLED APPS

* Cleanup selenium test cases

* Remove other `unittest` usages and unnecessary tests

* Override setUp() in a cleaner manner

* Add a travis fold

* Clean up FileStoreitem and InvestigationLink error handling

* Clean up error handling for isa & pre-isa archives

* Update comments

* Cleanup factory_boy factories and utils

* This is already a default value

* DRY

* Update existing DataMigrations to a more easily manageable format

* More robust Data migrations

* Remove `pk`

* Bump DRF version to latest that supports Django 1.7

* Add `tools` app

* Add models

* Configure admin ui for new models

* Add migration

* Add `tools/definitions/` endpoint

* Wrap deletion operations in transactions

* Add tools app

* Add factories and utils method to create dummy ToolDefinitions for Testing

* We don't need `django.setup()` here unless script is being run as a standalone

* RunPython is already encapsulated in a transaction

* refactor `get_isa_archive()` & `get_pre_isa_archive()`

* Analysis.status isn't necessary for testing

* Add Error Handling to `SeleniumTestBase.setUp()`

* Allow for proper rendering of self-referential Many-to-Many rels.

* Update models & serializers

* Add basic ToolDefinition API test case

* Update Models and Migrations file

* Add creation of LIST:LIST:PAIR ToolDef to utils script

* Update Serializer for more easily readble API Responses

* Update Tests

* Add MGMT command to genereate Smaple ToolDefinitions

* Demand users be authenticated to access DRF api endpoints

* Add more tests for ToolDef API

* Refactor DRF API related tests due to newly required Authentication

* Remove unused `nesting_type` field

* Add comment

* Add AdminFieldPopulator class

* Update MGMT command

* Add `TEST_NON_SERIALIZED_APPS` setting due to pre Django 1.9 bug

* Don't need to share datasets pubilcly in acceptance tests

* Utilize `serialized_rollback` to ensure migration data persists in StaticLiveServerTestCases

* Refactor isa/pre_isa archive fetching into one method

* Remove other unnecessary instances of `tearDown()`

* Update method signature

* Update comment

* Fix Test

* Revert "Fix Test"

This reverts commit 673827a.

* Don't bundle (pre)isa_archive fetching functionality & Utilize existing `get_investigation()`

* Handle potential AttributeErrors that could occcur

* Rename `tools` to `tool_manager`

* Favor APITestCase to ensure initial DB state persists throughout tests

* Provide request auth in test for newly secured endpoints

* Remove transaction context manager

* Refactor method name

* Keeping Python 3 in mind

* Rename field for clarity

* Add comments

* Fix serializer for newly added fieldname

* django.setup() not necessary (unless running as a standalone script)

* Change `nested_elements` -> `file_relationship` per Model field rename

* Reflect lack of POSTing in comment

* Fix import ordering

* Remove MGMT command since its only needed for a short period of time

* Migrations due to GalaxyParameter child class

* Add useful comment

* Remove already default values

* Revert from usage of Generic DRF api perm classes

* Change sample data generation to reflect new schema

* Ensure that Users accessing ToolDefinitions are authenticated

* Rename GalaxyToolParameter to GalaxyParameter

* Squash migrations

* Old isa_archive deletion tests were of better form

* Quoted references only necessary for manual avoidance of circ. imports

* We could have a tool without any Parameters

* Cleanup imports

* Remove unutilized APIClient

* Fix ordering of imports

* Fix migration

* Remove unnesscessary user creation

* Remove TearDown

* Remove constraint on ToolDefinition description & rename is_editable field

* Add `galaxy_workflow_step` field and squash migrations

* Scottx611x/td api url routing (#1622)

* Add RouterCombiner class to allow for url definitions to be app specific

* Don't really need `import as`

* Relative import

* Fix import ordering

* Place hard-coded DRF api urls into their own apps

* Fix Import ordering

* Add the ability to generate basic ToolDefinitons from properly annota… (#1624)

* Add the ability to generate basic ToolDefinitons from properly annotated Galaxy WFs

* Add `galaxy_workflow_step` field

* Validate incoming workflow annotation data

* Add Factory for GalaxyParameter creation

* Add tests to ensure proper creation of ToolDefinitions from Galaxy workflow annotations

* Handle ConnectionError if we can't reach Galaxy

* Rename `create_nesting()`

* Add custom Exception

* Refactor `validate_workflow_annotation()`

* Refactor ToolDefinition generation MGMT command

* Update tests

* Simplfy validation logic in mgmt command

* Utilize jsonschema to validate incoming ToolDefinitions

* Update mgmt command

* Update tests and test data

* Add `jsonschema` requirement

* Rename some things for clarity

* Add & Rename test data

* Properly handle invalid FileTypes in workflow annotations

* Fix Typo

* Add helpful comment

* jsonschema.validate is None upon successful validation, otherwise a ValidaationError is raised

* Cleanup generate_tool_definitions mgmt command

* Remove old clunky method to create ToolDefs in favor of new one

* Fix tests

* Clean up utils.py

* Update docstring

* Changes suggested from CR

* Directly access key rather that iterating through

* Split large ToolDefinition schema into individual files

* We don't need to use `hyper-schema`

* Let an empty object denote that we're at the bottom-most FileRelationship

* Travis didn't seem to like `os.path.abspath` so lets try `TRAVIS_BUILD_DIR` env var

* Try to fix Travis

* Trying out paths again

* Git doesn't automatically recognize case-sensitive only filename changes

* Remove blank lines and "..."

* Styling fixes
scottx611x added a commit that referenced this issue Mar 28, 2017
* Cleanup some more tests

* Add id to aid testing

* Add id to aid testing

* Cleanup login method

* Abstract out reused login moethod

* Utilize relative import

* PEP8

* only run acceptance tests if we detect a build for a PR

* Always run our acceptance tests

* Add acceptance tests for user registration and login

* Don't use separate shell script (for now)

* Remove `cd` comands

* Refactor selenium utility functions

* Add script to run acceptance tests on vagrant or travis

* Lower `DEFAULT_WAIT` Now that we aren't relying on SauceLabs VMs

* Cleanup setting of breakpoints, and refactor `cleanup_on_error` decorator

* Remove unnecessary env var, and re-add `global` as to ensure a single build

* Sauce_connect isn't needed

* pip caching seems stable

* PEP8

* Fail shell script if any step doesn't work properly

* GeckoDriver expects a more recent version of Firefox

* Remove Comments

* Update Default wait time

* Bump pytest-selenium version

* Update Comment

* Utilize `wait_until_id_clickable`

* Add tests for global analysis popover

* Add tests for Datasets panel

* Update comment

* Add a status to all generated Analyses

* Update stats test to check for actual object counts

* include gecko driver in our build process for ease of local selenium-based testing

* Refactor old acceptance test implementation in favor of simplicity and uniformness with our current Django unit tests

* Make virtual display bigger

* Update selenium testing utils

* Update assert body text method

* Update selenium tests

* Update `wait_until_id_visible()`

* bump Factory-Boy requirement

* Update travis as mentioned issue has been closed

* Clean up gecko driver Puppet install

* Update docstrings

* Remove unnecessary tests

* more geckodriver install cleanup

* Change Default wait

* update wait_until_id_visible()

* pull out duplicated code into ui-deletion helper method

* Update acceptance tests

* Minor changes due to abstraction of deletion code

* Fix typo

* guest user creation isn't needed here anymore

* Merge branch scottx611x/django_1.7_upgrade into scottx611x/acceptance_testing

* Flip the order of some installed apps due to Django 1.7 ContentType loading

* Decorate function as to not test 3rd party code's functionality

* Utilize StaticLiveServerTestCase

* Update delete_from_ui() helper

* Refactor decorator name to be more verbose

* Don't need to take a screenshot here

* Add xvfb and python3 to travis

* add pyvirtualdisplay req

* Rename geckodriver class to selenium

* Update order

* Update order of INSTALLED_APPS for testing purposes

* Update logging levels

* Update TEST_RUNNER per Django system-check suggestion

* Update tests

* Fix `get_or_create()`

* Access actual object returned from `get_or_create()` tuple

* Update logger statement

* Do not need to specify `live server` here since it has a default value

* Update string formatting to work properly

* Update isa/pre-isa archive deletion to produce less unnecessary errors

* Update Tests

* Manually save public group to sync db between threads (Specific to use of StaticLiveServerTestCase)

* Update `DEFAULT_WAIT`

* Share Datasets upon factory_boy creation

* Create public group for APIV2 test case

* Added group creation to the wrong TestCase

* Remove `django-nose` in favor of Django's new: `DiscoverTestRunner`

* This file shouldn't exist anymore. Probably wasn't caught in the merge here: 0c727cc

* Include ontology fixture content inside of a Data migration (#1581)

* Update puppet-archive version

* Utilize `puppet-archive` and clean up neo4j.pp

* Cleanup selenium manifest

* Factor out solr class from init.pp

* don't need to chmod here

* We are now using StaticLiveServerTestCase

* "Users of the `puppet-archive`module are responsible for archive package dependencies"

* Remove unused var

* Point to new archive download location

* Fix string formatting error

* Add `TEST_RUNNER` setting to base.py so that someone testing on `prod` locally could also benefeit

* update order of `INSTALLED_APPS` addresses: #1353

* Refactor all tests that interact with the ORM to inherit from `TransactionTestCase` (removes need for crazy tearDown()'s)

* DEFAULT_WAIT Shouldn't be that long

* Manually save `public group` as to persist data between test and web driver threads

* Update comment

* Don't need to optimize sole's index in tests

* Inherit from TransactionTestCase here

* `20` was actually a good fail-safe if things get slow for one reason or another

* switch order of INSTALLED APPS

* Cleanup selenium test cases

* Remove other `unittest` usages and unnecessary tests

* Override setUp() in a cleaner manner

* Add a travis fold

* Clean up FileStoreitem and InvestigationLink error handling

* Clean up error handling for isa & pre-isa archives

* Update comments

* Cleanup factory_boy factories and utils

* This is already a default value

* DRY

* Update existing DataMigrations to a more easily manageable format

* More robust Data migrations

* Remove `pk`

* Bump DRF version to latest that supports Django 1.7

* Add `tools` app

* Add models

* Configure admin ui for new models

* Add migration

* Add `tools/definitions/` endpoint

* Wrap deletion operations in transactions

* Add tools app

* Add factories and utils method to create dummy ToolDefinitions for Testing

* We don't need `django.setup()` here unless script is being run as a standalone

* RunPython is already encapsulated in a transaction

* refactor `get_isa_archive()` & `get_pre_isa_archive()`

* Analysis.status isn't necessary for testing

* Add Error Handling to `SeleniumTestBase.setUp()`

* Allow for proper rendering of self-referential Many-to-Many rels.

* Update models & serializers

* Add basic ToolDefinition API test case

* Update Models and Migrations file

* Add creation of LIST:LIST:PAIR ToolDef to utils script

* Update Serializer for more easily readble API Responses

* Update Tests

* Add MGMT command to genereate Smaple ToolDefinitions

* Demand users be authenticated to access DRF api endpoints

* Add more tests for ToolDef API

* Refactor DRF API related tests due to newly required Authentication

* Remove unused `nesting_type` field

* Add comment

* Add AdminFieldPopulator class

* Update MGMT command

* Add `TEST_NON_SERIALIZED_APPS` setting due to pre Django 1.9 bug

* Don't need to share datasets pubilcly in acceptance tests

* Utilize `serialized_rollback` to ensure migration data persists in StaticLiveServerTestCases

* Refactor isa/pre_isa archive fetching into one method

* Remove other unnecessary instances of `tearDown()`

* Update method signature

* Update comment

* Fix Test

* Revert "Fix Test"

This reverts commit 673827a.

* Don't bundle (pre)isa_archive fetching functionality & Utilize existing `get_investigation()`

* Handle potential AttributeErrors that could occcur

* Rename `tools` to `tool_manager`

* Favor APITestCase to ensure initial DB state persists throughout tests

* Provide request auth in test for newly secured endpoints

* Remove transaction context manager

* Refactor method name

* Keeping Python 3 in mind

* Rename field for clarity

* Add comments

* Fix serializer for newly added fieldname

* django.setup() not necessary (unless running as a standalone script)

* Change `nested_elements` -> `file_relationship` per Model field rename

* Reflect lack of POSTing in comment

* Fix import ordering

* Remove MGMT command since its only needed for a short period of time

* Migrations due to GalaxyParameter child class

* Add useful comment

* Remove already default values

* Revert from usage of Generic DRF api perm classes

* Change sample data generation to reflect new schema

* Ensure that Users accessing ToolDefinitions are authenticated

* Rename GalaxyToolParameter to GalaxyParameter

* Squash migrations

* Old isa_archive deletion tests were of better form

* Quoted references only necessary for manual avoidance of circ. imports

* We could have a tool without any Parameters

* Cleanup imports

* Remove unutilized APIClient

* Fix ordering of imports

* Fix migration

* Remove unnesscessary user creation

* Remove TearDown

* Remove constraint on ToolDefinition description & rename is_editable field

* Add `galaxy_workflow_step` field and squash migrations

* Scottx611x/td api url routing (#1622)

* Add RouterCombiner class to allow for url definitions to be app specific

* Don't really need `import as`

* Relative import

* Fix import ordering

* Place hard-coded DRF api urls into their own apps

* Fix Import ordering

* Add the ability to generate basic ToolDefinitons from properly annota… (#1624)

* Add the ability to generate basic ToolDefinitons from properly annotated Galaxy WFs

* Add `galaxy_workflow_step` field

* Validate incoming workflow annotation data

* Add Factory for GalaxyParameter creation

* Add tests to ensure proper creation of ToolDefinitions from Galaxy workflow annotations

* Handle ConnectionError if we can't reach Galaxy

* Rename `create_nesting()`

* Add custom Exception

* Refactor `validate_workflow_annotation()`

* Refactor ToolDefinition generation MGMT command

* Update tests

* Simplfy validation logic in mgmt command

* Utilize jsonschema to validate incoming ToolDefinitions

* Update mgmt command

* Update tests and test data

* Add `jsonschema` requirement

* Rename some things for clarity

* Add & Rename test data

* Properly handle invalid FileTypes in workflow annotations

* Fix Typo

* Add helpful comment

* jsonschema.validate is None upon successful validation, otherwise a ValidaationError is raised

* Cleanup generate_tool_definitions mgmt command

* Remove old clunky method to create ToolDefs in favor of new one

* Fix tests

* Clean up utils.py

* Update docstring

* Changes suggested from CR

* Directly access key rather that iterating through

* Split large ToolDefinition schema into individual files

* We don't need to use `hyper-schema`

* Let an empty object denote that we're at the bottom-most FileRelationship

* Travis didn't seem to like `os.path.abspath` so lets try `TRAVIS_BUILD_DIR` env var

* Try to fix Travis

* Trying out paths again

* Git doesn't automatically recognize case-sensitive only filename changes

* Remove blank lines and "..."

* Styling fixes

* Update Galaxy Parameter Model to include a single field

* Update ToolDefinition schema

* Separate Parameter and GalaxyParameter schemas

* Add new sample data utilizing Parameters

* Add tests to check for good and bad parameters in Annotation data

* Avoid code reuse by extending from other schema

* Update test-data and tests to utilize Parameters

* Add proper deletion of a ToolDefinitions related objects + tests

* Update Parameter schema

* Update GalaxyParameter schema

* Add schema to aid in validating Workflows individual step's annotations

* Add test-data and tests to validate Workflows step's annotations

* Rename `validate_workflow_annotation` to `validate_tool_annotation`

* `clear()` is redundant if we are also performing `delete()` here

* Include parameter and output_file parsing from step annotations in MGMT command

* Update ParameterSerializer to include/disclude `galaxy_workflow_step` based on its existance

* Add sample data and tests for a basic Visualization-based ToolDefinition

* Fix typo and update comment

* Update comments

* Remove extra space

* RuntimeError is actually being raised here

* Rename method to create_tool_definition

* Add API test to check for different Parameter Types

* Styling fixes

* Fix comment

* Rename pre/post delete handlers

* Fix styling

* Fix more styling

* Remove newline characters

* Factor out workflow fetching,  and cleanup indentation.

* Add a sample galaxy `get_workflows()` response for testing purposes

* Test management command functionality with proper `mock` usage

* Remove comment explaining "what"

* Revert "Remove comment explaining "what""

This reverts commit 69d5c65.

* Rename `test-data` to `test_data`

* Utilize logger statements since not in a MGMT command anymore

* Handle RuntimeError
@scottx611x
Copy link
Member

This has been addressed as of commit: 1449939

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants