Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix sporadic failure for “firstTimeOnly” methods #2850

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

krmahadevan
Copy link
Member

Ensure that the listeners for “firstTimeOnly”
methods are executed ONLY if the config method
was run else skip listener execution as well.

The test was failing because it relied on config
listeners

Fixes the sporadic failure in https://github.com/krmahadevan/testng/actions/runs/3611418844/jobs/6085892554

Ensure that the listeners for “firstTimeOnly” 
methods are executed ONLY if the config method
was run else skip listener execution as well.

The test was failing because it relied on config
listeners
@juherr
Copy link
Member

juherr commented Dec 4, 2022

Sorry but I don't catch the fix.
What is the relation between firstTimeOnly and the execution of all results?

Why not having the same on before listener when lastTimeOnly is involved?

Any way to find a test case where the test result is shared and where we want to run the after listener every times? Maybe something with retryAnalyzer and/or data provider.
Tbh, I'm surprised the fix is not breaking tests.

@krmahadevan
Copy link
Member Author

What is the relation between firstTimeOnly and the execution of all results?

The execution of results is related because our test is relying on listeners to gather info on what was executed and what wasn't.

Why not having the same on before listener when lastTimeOnly is involved?

I don't think we have any test that is doing that. That perhaps is why I didn't add.

Any way to find a test case where the test result is shared and where we want to run the after listener every times? Maybe something with retryAnalyzer and/or data provider.

I don't know. I will have to dig around to see if we can find something.

Tbh, I'm surprised the fix is not breaking tests.
We have tests that are checking this.

@juherr
Copy link
Member

juherr commented Dec 4, 2022

Yes, I think we should take the time to dig a bit deeper.
The problem here is more the feature than the failing test.

A good step could be to check feature tests and the more exhaustive as possible.
#1711 and #792 can be considered in the test scenarii.

The fix will come later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants