Join GitHub today
Fix remaining failing test in 5.2 #2195
Plone 5.2 - Python 2.7 is finally green but we still have 9 failling tests in http://jenkins.plone.org/job/plone-5.2-python-2.7-at/.
Here are these tests and links to issue/pr.
Details: The (swallowed) error is
Doing exactly the same in Plone 5.1 in
referenced this issue
Oct 27, 2017
Just a note: reverting this zopefoundation/Products.CMFCore@05b3968#diff-4e6966d1d7ef0aaed5a8c2322c887fb1
plone/Products.Marshall@23a7752 fixed failures I had when running
The remaining failure in
added a commit
Nov 9, 2017
So here is small story about the failing tests in
The two breaking tests
Here is the relevant part of the log:
What? Why is does
First I reproduced the error by testing all packages in the test-group Archetypes (where the error occurs):
Then I only use the test-layers that are actually set up according to the output:
That works, I see the error. But I will not try to read 733 tests and wait for more than 3 minutes each time I think I may have fixed something!
Thus I use divide and conquer to figure out which combination producedes the failing tests: remove half of the packages layers and see if it still fails. If they pass try the other half. Do the same with the layers.
Remember to keep
It turned out that the following combination exposed the issue:
Still way to many tests to have a look. Most of them in
Consequently I kept only one doctest that still kept the issue by commenting out the others in
Now I needed to find a combination of three tests from these layers that still expose the issue. To to that I added the option -vv to the testrunner to see the names and python path of all tests I still run.
24 seconds? I can work with that.
Still I remove tests from each layer until I only have three tests remaining and revert my changes to .
The result is the following:
Since more than one test still exposed the issue I kept only very simple ones because I guessed that the issue is actually in the setup or teardown.
So next I change the test
At this point I can skip constraining the layers since I only run three tests from two packages.
Time to actually read what the remaining tests are doing. I strip down all tests and their setup to the base minimum that still breaks the tests and cannot find anything. I turn
So the test-layers for the two tests that do not fail or error by themselves but trigger the issue in the failing tests (
This is the moment when I decide to migrate them to proper
I hope that monster will never rear its ugly head again until
I fear it will be quite some time until all
pushed a commit
Nov 9, 2017
@pbauer you should turn that into a blog post!
According to http://jenkins.plone.org/py3/deps-reverse/Products.CMFFormController.svg only archetypes, ATCT and CMFPlone use CMFFormController, and on CMFPlone is almost gone, is still that much work to do?
Lastly, regarding migration to p.a.testing, sadly is the way to go, we need to have all tests over it to avoid isolation tests.
The final issue was fixed with plone/plone.app.folder#14 by @davisagli. Many thanks!