Flaky Test Framework Test

[adamgfraser]

  - test framework in a more ZIO-centric way / only execute test with specified tag
    ✗ Result was false
    output.equals(expected)
    output = Vector1("+ tag suite
    ", """  - tag suite
        Exception in thread \"zio-fiber-1027\" java.io.FileNotFoundException: target/test-reports-zio/zio.test.sbt.FrameworkSpecInstances$TagsSpec$_debug.txt (No such file or directory)
        	at java.base/java.io.FileInputStream.open0(Native Method)
        	at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
        	at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
        	at scala.io.Source$.fromFile(Source.scala:94)
        	at scala.io.Source$.fromFile(Source.scala:79)
        	at scala.io.Source$.fromFile(Source.scala:57)
        	at zio.test.TestDebug$.$anonfun$removeLine$1(TestDebug.scala:69)
        	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
        	at zio.test.TestDebug.removeLine(TestDebug.scala:68)
        	at zio.test.TestDebugFileLock.updateFile(TestDebugFileLock.scala:12)
        	at zio.test.TestOutput.TestOutputLive.printOrQueue(TestOutput.scala:116)
        	at zio.test.sbt.ZTestFrameworkZioSpec.loadAndExecuteAllZ(ZTestFrameworkZioSpec.scala:208)
        	at zio.test.sbt.ZTestFrameworkZioSpec.loadAndExecute(ZTestFrameworkZioSpec.scala:186)
        	at zio.test.sbt.ZTestFrameworkZioSpec.spec(ZTestFrameworkZioSpec.scala:138)
        	at zio.test.sbt.ZTestFrameworkZioSpec.spec(ZTestFrameworkZioSpec.scala:136)
    """)
    at /home/runner/work/zio/zio/test-sbt/jvm/src/test/scala/zio/test/sbt/ZTestFrameworkZioSpec.scala:145 

              Console IO Produced by Test         
    | + tag suite
    |   - tag suite
    Exception in thread "zio-fiber-1027" java.io.FileNotFoundException: target/test-reports-zio/zio.test.sbt.FrameworkSpecInstances$TagsSpec$_debug.txt (No such file or directory)
    	at java.base/java.io.FileInputStream.open0(Native Method)
    	at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
    	at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
    	at scala.io.Source$.fromFile(Source.scala:94)
    	at scala.io.Source$.fromFile(Source.scala:79)
    	at scala.io.Source$.fromFile(Source.scala:57)
    	at zio.test.TestDebug$.$anonfun$removeLine$1(TestDebug.scala:69)
    	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
    	at zio.test.TestDebug.removeLine(TestDebug.scala:68)
    	at zio.test.TestDebugFileLock.updateFile(TestDebugFileLock.scala:12)
    	at zio.test.TestOutput.TestOutputLive.printOrQueue(TestOutput.scala:116)
    	at zio.test.sbt.ZTestFrameworkZioSpec.loadAndExecuteAllZ(ZTestFrameworkZioSpec.scala:208)
    	at zio.test.sbt.ZTestFrameworkZioSpec.loadAndExecute(ZTestFrameworkZioSpec.scala:186)
    	at zio.test.sbt.ZTestFrameworkZioSpec.spec(ZTestFrameworkZioSpec.scala:138)
    	at zio.test.sbt.ZTestFrameworkZioSpec.spec(ZTestFrameworkZioSpec.scala:136)
    ==============================================

[swoogles]

I have some ideas for this, but I'm unable to trigger the failure locally, nomatter how much I ratchet up my nonFlaky value. Still trying to figure out a reliable reproducer.

@adamgfraser Do you recall if this failure strikes arbitrary test cases, or if it's reliably this test?
It doesn't seem like this problem would be test-specific, but just want to check.

[adamgfraser]

@swoogles Was this fixed by #8201?

[swoogles]

Yes, but as you alluded to - that's a bandaid. We shouldn't ever be trying to delete a line from a file that was already deleted, so we've got a race happening somewhere. I removed that existence check while trying to reproduce the error.

[jdegoes]

/bounty $200

[algora-pbc]

💎 $200 bounty • ZIO

Steps to solve:

1. Start working: Comment /attempt #8079 with your implementation plan
2. Submit work: Create a pull request including /claim #8079 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Additional opportunities:

• 🔴 Livestream on Algora TV while solving this bounty & earn $200 upon merge! Comment /livestream once live

Thank you for contributing to zio/zio!

Add a bounty • Share on socials

[adamgfraser]

@vitaliihonta Is this issue resolved for you?