Fix flaky specs using CKEditor #2711
Conversation
javierv
changed the title
There was a problem hiding this comment.
This is great, @javierv ! 馃帀
I've been testing this locally (as well as restarting this PR's Travis' builds several times) and so far, so good!
Just one small thing: Can you squash these commits? Being such a small patch, there's no need to clutter Git history with multiple commits
Let me know what you think or if you need any help! 馃樃
javierv
force-pushed
the
fix_flaky_ckeditor_specs
branch
from
a3b847e
to
5a43035
Compare
As stated in consuldemocracy#1196, compiling everything related to CKEditor made compilations slower. However, not compiling any plugins meant Travis had to compile them while running a test. It often resulted in a test failing because the time Travis took to compile the plugins the application uses exceeded Capybara's wait time.
javierv
force-pushed
the
fix_flaky_ckeditor_specs
branch
from
5a43035
to
54c82a5
Compare
|
馃憣 Done! |
|
Great, thanks a lot @javierv ! 馃帀 I'll be testing this patch thoroughly later today and if all goes well, I'll let you know 馃樃 |
|
@aitbw I've added a Failed Tags Category with category tags spec to the list of flakies fixed by this pull request, with a screenshot showing the failure. If there's anything I can help with, let me know! |
|
Hi @javierv ! First of all: Sorry for the late reply. We've been very busy lately. As I told you a few days ago, I've been testing this thoroughly locally and on Travis and seems like all the scenarios you mentioned on the PR description (thanks a lot for keeping it updated) are completely free of flakiness 馃帀 I'll let the team know so we can get this merged ASAP 馃槍 Thanks a lot once again 鉂わ笍 |
|
This is Fantastic! 馃帀 |
This change adresses ithe problem with assets pipeline fallback on testing environment described here: consuldemocracy#2711. Now no extra precompilation is done when running ckeditor feature specs that uses javascript.
This change adresses ithe problem with assets pipeline fallback on testing environment described here: consuldemocracy#2711. Now no extra precompilation is done when running ckeditor feature specs that uses javascript.
This change adresses ithe problem with assets pipeline fallback on testing environment described here: #2711. Now no extra precompilation is done when running ckeditor feature specs that uses javascript.
All of these plugins are not used anywhere. Change introduced at ckeditor initializer will ommit unneeded precompilation of plugins assets on production environments. Change introduced at ckeditor config file adresses the problem with assets pipeline fallback on testing environments described here: consuldemocracy#2711. Now plugins that are explicitly disabled will not be precomiled when running ckeditor javascript enabled feature specs.
References
Failed nested documentable proposals specFailure log not available anymore.Objectives
Fix the flaky specs that appeared in
spec/features/ckeditor_spec.rb:14("CKEditor is present before & after turbolinks update page"),spec/features/debates_spec.rb:998("Debates Suggesting debates Shows up to 5 suggestions"),spec/features/debates_spec.rb:1019("Debates Suggesting debates No found suggestions"),spec/features/proposals_spec.rb:1597("Proposals Suggesting proposals No found suggestions"),spec/features/budgets/investments_spec.rb:728("Budget Investments Phase I - Accepting Suggest Show up to 5 suggestions"),spec/features/budgets/investments_spec.rb:1061("Budget Investments behaves like nested documentable at new budget investment path Should not update document cached attachment field after unvalid file upload"),spec/features/proposals_spec.rb:1440("Proposals behaves like nested imageable at new_proposal_path Should show new image after successful creation with one uploaded file"),spec/features/tags/budget_investments_spec.rb:120("Tags Turbolinks sanity check from budget's show"),spec/features/proposals_spec:1467("Proposals behaves like nested documentable at edit proposal path Should update loading bar style after valid file upload") andspec/features/tags/budget_investments_spec.rb:82("Tags Category with category tags").Explain why the test is flaky, or under which conditions/scenario it fails randomly
Most assets are precompiled before running a Travis build. However, as mentioned in pull request #1196, precompiling all CKEditor plugins would take too long, and so they aren't included.
On the other hand, when no plugins are precompiled, the first time a page loading CKEditor is rendered, the application takes a few seconds to compile the plugins it uses. If there's an AJAX request during that time, the test will probably fail because the compilation will take longer than Capybara's wait time.
In a similar way, tests using
fill_in_ckeditormay fail because this method is executed while the plugins are still being compiled. It results in the editor area not being filled properly, as shown in this screenshot taken by Capybara:To reproduce one of these failures locally, run:
If it still passes, maybe your machine is too fast 馃槂. Try setting
Capybara.default_max_wait_time = 1, deleting the sprockets cache again, and running the spec again.Explain why your PR fixes it
By precompiling only the CKEditor plugins the application uses, they don't need to be compiled in the middle of a test, AJAX calls don't exceed Capybara's wait time, and compiling CKEditor assets doesn't take too long.