Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CV: skip sporadically failing template delivery #7344

Merged

Conversation

mbien
Copy link
Member

@mbien mbien commented Apr 29, 2024

#7333 degrades CV test stability

  • failure: "FSException: Cannot get shared access"
  • tests can be brute forced into passing.
  • It is not clear if something is wrong with the hint editor support or if this template is just degrading the sporadically failing tests further due to changed timing.

lets try to skip over it and check if it improves the stability.

other:

  • wrap masterfs.macosx in retry script due to sporadic failures in OSXNotifierTest#testNextEvent
  • NbTestCase: use Files#delete for better exception on failure, (second commit)
    • "Cannot delete file" isn't very useful, Files#delete will state the cause if available
    • might help to debug some sporadic test failures
    • global refactoring to replace similar patterns resulted in some TestUtil cleanup which was duplicated everywhere

@mbien
Copy link
Member Author

mbien commented Apr 30, 2024

so the CV tests on linux passed on first attempt, which is a good sign (esp when compared to https://github.com/apache/netbeans/actions/runs/8876164558).

platform/masterfs.macosx (testNextEvent: org.netbeans.modules.masterfs.watcher.macosx.OSXNotifierTest) seems to fail often recently, but I can't help with that one due to lack of hardware + it isn't related to this problem. I might wrap it in the retry script.

 - failure: "FSException: Cannot get shared access"
 - tests can be brute forced into passing.
 - It is not clear if something is wrong with the hint editor support
   or if this template is just degrading the sporadically failing tests
   further due to changed timing.

lets try to skip over it and check if it improves the stability.

other:

 - wrap masterfs.macosx in retry script due to sporadic failures in
   OSXNotifierTest#testNextEvent
@mbien mbien added Code cleanup CI continuous integration changes ci:all-tests [ci] enable all tests labels Apr 30, 2024
@mbien mbien force-pushed the cv-skip-sporadically-failing-template_delivery branch from 1238704 to fa58699 Compare April 30, 2024 13:47
@mbien mbien marked this pull request as ready for review April 30, 2024 13:51
@mbien mbien force-pushed the cv-skip-sporadically-failing-template_delivery branch 3 times, most recently from 9949cf7 to a8dd303 Compare April 30, 2024 16:04
Copy link
Contributor

@matthiasblaesing matthiasblaesing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course a fix for the root cause of the instability would be preferred, but not causing false-positives is more important.
In general this looks like a good cleanup. Thank you!

@mbien mbien removed the ci:all-tests [ci] enable all tests label May 1, 2024
@mbien mbien force-pushed the cv-skip-sporadically-failing-template_delivery branch from a8dd303 to 4aae710 Compare May 1, 2024 19:03
 - "Cannot delete file" isn't very useful, Files#delete will state
   the cause if available
 - might help to debug some sporadic test failures
 - global refactoring to replace similar patterns resulted in some
   TestUtil/TestFileUtils cleanup which was duplicated everywhere
@mbien mbien force-pushed the cv-skip-sporadically-failing-template_delivery branch from 4aae710 to cbd9644 Compare May 1, 2024 19:20
Copy link
Contributor

@matthiasblaesing matthiasblaesing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@ebarboni ebarboni merged commit cc29734 into apache:delivery May 3, 2024
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI continuous integration changes Code cleanup tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants