Fewer flakes with max-cases 1?

[taylordowns2000]

Checking CI's response here:

1 for 1 baby! here we go!

2 for 2!!!

(Watch this space)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.47%. Comparing base (7a57071) to head (8c916b7).
Report is 26 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3150      +/-   ##
==========================================
+ Coverage   91.44%   91.47%   +0.02%     
==========================================
  Files         357      357              
  Lines       13462    13462              
==========================================
+ Hits        12310    12314       +4     
+ Misses       1152     1148       -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[stuartc]

@taylordowns2000 this would make the tests much slower, and this isn't how we run tests locally. If anything, we can try changing the ERL_FLAGS to have "less cpu". Right now it's 4:4 I think?

[taylordowns2000]

Understood. I'm up for whatever. If this reduces flakes by 5% though, I'd be happy increasing test runtime by 10x. My workflow is: 1. write code 2. run tests locally 3. push to github 4. request review 5. do entirely unrelated thing (wait however long, doesn't matter as i'm already _long_ gone) 6. retry 7. do entirely unrelated thing (wait however long, doesn't matter as i'm already _long_ gone) 8. repeat... So the duration doesn't impact me, but each flake costs me about 10 minutes of switching cost/distraction impact.

…

On Wed, May 7, 2025 at 1:54 PM Stuart Corbishley ***@***.***> wrote: *stuartc* left a comment (OpenFn/lightning#3150) <#3150 (comment)> @taylordowns2000 <https://github.com/taylordowns2000> this would make the tests much slower, and this isn't how we run tests locally. If anything, we can try changing the ERL_FLAGS to have "less cpu". Right now it's 4:4 I think? — Reply to this email directly, view it on GitHub <#3150 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACCUBLNZJGR7HLFYHOSC2HL25HYADAVCNFSM6AAAAAB33OWWL6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQNJYGI4TMMZRGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Taylor Downs CEO, OpenFn <https://www.openfn.org>

[stuartc]

I hear you. I'd like to do better.

When the elixir upgrade branch lands I'm going to change the test command to:

mix test || mix test --failed

That will rerun just the failed tests, and assuming this doesn't cause some problems with codecov (it might) we will had that "second go around" automated.

[taylordowns2000]

oooooh wow! that's such a cool idea. like... it's an automatic "try just once more before bugging me" line?!

[taylordowns2000]

Hey should I close this PR @stuartc ? You're welcome to if no longer useful