New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refs #25251 -- Filtered out skipped tests when processing the test suite to set _next_serialized_rollback. #10667

Merged
merged 1 commit into from Nov 29, 2018

Conversation

Projects
None yet
2 participants
@romgar
Copy link
Contributor

romgar commented Nov 19, 2018

Reported at #7528 (comment).

@timgraham

This comment has been minimized.

Copy link
Member

timgraham commented Nov 20, 2018

Could you add a regression test? Currently we have a non-deterministic failure when running the entire test suite which would be quite difficult to debug if this regresses.

@timgraham timgraham force-pushed the romgar:ticket_25251_4 branch from 58bd91b to e58fea2 Nov 29, 2018

@timgraham

This comment has been minimized.

Copy link
Member

timgraham commented Nov 29, 2018

I tried adding a test but it doesn't pass. I may have very well made a mistake somewhere as I didn't get very far in my investigation but I added the test here in case you look at the issue before I get back to it.

@timgraham timgraham force-pushed the romgar:ticket_25251_4 branch 2 times, most recently from ccbbaad to 92a8b0b Nov 29, 2018

@timgraham timgraham changed the title Refs #25251 -- Consider skipped tests when preparing data in tests with serialized_rollback option Refs #25251 -- Filtered out skipped tests when processing the test suite to set _next_serialized_rollback. Nov 29, 2018

@timgraham timgraham force-pushed the romgar:ticket_25251_4 branch from 92a8b0b to 9fa0d37 Nov 29, 2018

@timgraham

This comment has been minimized.

Copy link
Member

timgraham commented Nov 29, 2018

The patch is working now. The issue was that the test needed to skipUnlessDBFeature instead of skip. With skip, the test was already filtered out somewhere before reorder_postprocess().

@timgraham timgraham merged commit 9fa0d37 into django:master Nov 29, 2018

19 checks passed

docs Build #17716 ended
Details
flake8 Build #17825 ended
Details
isort Build #17860 succeeded in 17 sec
Details
pr-mariadb/database=mysql,label=mariadb,python=python3.7 Build #2306 ended
Details
pr-mariadb/database=mysql_gis,label=mariadb,python=python3.7 Build #2306 ended
Details
pull-requests-bionic/database=mysql,label=bionic-pr,python=python3.5 Build #2789 ended
Details
pull-requests-bionic/database=mysql,label=bionic-pr,python=python3.7 Build #2789 ended
Details
pull-requests-bionic/database=mysql_gis,label=bionic-pr,python=python3.5 Build #2789 ended
Details
pull-requests-bionic/database=mysql_gis,label=bionic-pr,python=python3.7 Build #2789 ended
Details
pull-requests-bionic/database=postgis,label=bionic-pr,python=python3.5 Build #2789 ended
Details
pull-requests-bionic/database=postgis,label=bionic-pr,python=python3.7 Build #2789 ended
Details
pull-requests-bionic/database=postgres,label=bionic-pr,python=python3.5 Build #2789 ended
Details
pull-requests-bionic/database=postgres,label=bionic-pr,python=python3.7 Build #2789 ended
Details
pull-requests-bionic/database=spatialite,label=bionic-pr,python=python3.5 Build #2789 ended
Details
pull-requests-bionic/database=spatialite,label=bionic-pr,python=python3.7 Build #2789 ended
Details
pull-requests-bionic/database=sqlite3,label=bionic-pr,python=python3.5 Build #2789 ended
Details
pull-requests-bionic/database=sqlite3,label=bionic-pr,python=python3.7 Build #2789 ended
Details
pull-requests-javascript Build #14219 ended
Details
pull-requests-windows/database=sqlite3,label=windows,python=Python35 Build #9829 ended
Details
@romgar

This comment has been minimized.

Copy link
Contributor Author

romgar commented Nov 29, 2018

Thanks @timgraham. I'm quite busy these days, I really appreciate your help.

@timgraham

This comment has been minimized.

Copy link
Member

timgraham commented Nov 30, 2018

Unfortunately, there's still the failure of migration_test_data_persistence.tests.MigrationDataPersistenceTestCase.test_persistence on the Jenkins build with --reverse. I haven't yet been able to pin down a more minimal way to reproduce it.

@timgraham

This comment has been minimized.

Copy link
Member

timgraham commented Nov 30, 2018

Can reproduce with ./tests/runtests.py --parallel=1 -v 2 --reverse migration_test_data_persistence inspectdb and even comment out tests in inspectdb until you have these minimal tests:

test_persistence (migration_test_data_persistence.tests.MigrationDataNormalPersistenceTestCase) ...
ok
test_include_views (inspectdb.tests.InspectDBTransactionalTests) ... ok
test_foreign_data_wrapper (inspectdb.tests.InspectDBTransactionalTests) ... skipped 
test_persistence (migration_test_data_persistence.tests.MigrationDataPersistenceTestCase) ... failed

The issue is that a test will only have __unittest_skip__ set if a skip decorator is applied to the test class, not if a test method is decorated. I'm not sure of a solution so I'm afraid it might be time to revert the original patch until someone has time to dive in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment