Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky test on update_assembly_member_spec.rb #8627

Closed
andreslucena opened this issue Dec 17, 2021 · 3 comments · Fixed by #8739
Closed

Flaky test on update_assembly_member_spec.rb #8627

andreslucena opened this issue Dec 17, 2021 · 3 comments · Fixed by #8739
Labels
flaky spec module: assemblies type: bug Issues that describe a bug

Comments

@andreslucena
Copy link
Member

Describe the bug

There's a flaky test in ./spec/commands/update_assembly_member_spec.rb
First detected in CI, to reproduce (sometimes):

To Reproduce

cd decidim-assemblies
bundle exec rspec ./spec/commands/update_assembly_member_spec.rb[1:1:1,1:1:2:1,1:2:2,1:2:3] --seed 6698

Stacktrace

Run options: include {:ids=>{"./spec/commands/update_assembly_member_spec.rb"=>["1:1:1", "1:1:2:1", "1:2:2", "1:2:3"]}}

Randomized with seed 6698
.F..

Failures:

  1) Decidim::Assemblies::Admin::UpdateAssemblyMember when everything is ok broadcasts  ok
     Failure/Error: expect { subject.call }.to broadcast(:ok)
       expected publisher to broadcast ok event (actual events broadcast: invalid)
     # ./spec/commands/update_assembly_member_spec.rb:81:in `block (3 levels) in <module:Assemblies>'

Finished in 3.39 seconds (files took 4.66 seconds to load)
4 examples, 1 failure

Failed examples:

rspec ./spec/commands/update_assembly_member_spec.rb:80 # Decidim::Assemblies::Admin::UpdateAssemblyMember when everything is ok broadcasts  ok

Randomized with seed 6698

Expected behavior
To not have it 😝

  • Decidim Version: v0.26

Additional context

Sometimes it fails in the traceability block, so my intuition says it could be related with #8608 - although I don't have any proof actually (nor time to see if I can reproduce it with the commit before that one).
@andreslucena andreslucena mentioned this issue Dec 17, 2021
Merged
12 tasks
@armandfardeau
Copy link
Contributor

armandfardeau commented Dec 23, 2021 •
edited

Hi @andreslucena, I've done a bit of research on this.

BISECT=true bundle exec rspec ./spec/commands/update_assembly_member_spec.rb --bisect

Bisect started using options: "./spec/commands/update_assembly_member_spec.rb"
Running suite to find failures... (20.36 seconds)
Starting bisect with 2 failing examples and 4 non-failing examples.
Checking that failure(s) are order-dependent... failure appears to be order-dependent

Round 1: bisecting over non-failing examples 1-4 .. multiple culprits detected - splitting candidates (27.84 seconds)
Round 2: bisecting over non-failing examples 1-2 .. multiple culprits detected - splitting candidates (29.37 seconds)
Round 3: bisecting over non-failing examples 3-4 .. multiple culprits detected - splitting candidates (36.87 seconds)
Bisect complete! Reduced necessary non-failing examples from 4 to 4 in 1 minute 50.28 seconds.

The minimal reproduction command is:
  rspec './spec/commands/update_assembly_member_spec.rb[1:1:1,1:1:2:1,1:2:1,1:2:2,1:2:3,1:2:4:1]'

After reverting #8608 the result is a bit different

BISECT=true bundle exec rspec ./spec/commands/update_assembly_member_spec.rb --bisect 

Bisect started using options: "./spec/commands/update_assembly_member_spec.rb"
Running suite to find failures... (23.44 seconds)
Starting bisect with 1 failing example and 5 non-failing examples.
Checking that failure(s) are order-dependent... failure appears to be order-dependent

Round 1: bisecting over non-failing examples 1-5 .. multiple culprits detected - splitting candidates (30.79 seconds)
Round 2: bisecting over non-failing examples 1-3 . ignoring examples 1-2 (12.71 seconds)
Round 3: bisecting over non-failing examples 4-5 .. multiple culprits detected - splitting candidates (30.54 seconds)
Bisect complete! Reduced necessary non-failing examples from 5 to 3 in 1 minute 28.51 seconds.

The minimal reproduction command is:
  rspec './spec/commands/update_assembly_member_spec.rb[1:2:1,1:2:2,1:2:3,1:2:4:1]'

@andreslucena
Copy link
Member Author

After reverting #8608 the result is a bit different

Ah ok, so my hypothesis that this flaky could be related to the paper_trail upgrade is wrong, because we still have this failing test, right?

@armandfardeau
Copy link
Contributor

After reverting #8608 the result is a bit different

Ah ok, so my hypothesis that this flaky could be related to the paper_trail upgrade is wrong, because we still have this failing test, right?

Unfortunately, I think so and I don't have any other insights.

@andreslucena andreslucena mentioned this issue Jan 19, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
flaky spec module: assemblies type: bug Issues that describe a bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix flaky test in UpdateAssemblyMember decidim/decidim
2 participants
@andreslucena @armandfardeau