New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

XMLLogger methods should be thread-safe #4932

Closed
soon opened this Issue Aug 11, 2017 · 6 comments

Comments

@soon
Contributor

soon commented Aug 11, 2017

Now the output of XMLLogger might be corrupted, if several threads will try to raise errors for the different files at the same time.

The fileStarted, fileFinished, addError and addException methods must be thread-safe

@soon

This comment has been minimized.

Show comment
Hide comment
@soon

soon Aug 11, 2017

Contributor

@rnveach @romani I have a question about the addException method. Is it used by anyone? I've found only tests for this method.

Contributor

soon commented Aug 11, 2017

@rnveach @romani I have a question about the addException method. Is it used by anyone? I've found only tests for this method.

@rnveach

This comment has been minimized.

Show comment
Hide comment
@rnveach

rnveach Aug 14, 2017

Member

addException method. Is it used by anyone? I've found only tests for this method.

@soon See #4377 .

Member

rnveach commented Aug 14, 2017

addException method. Is it used by anyone? I've found only tests for this method.

@soon See #4377 .

@rnveach rnveach added the approved label Aug 14, 2017

soon added a commit to soon/checkstyle that referenced this issue Aug 14, 2017

soon added a commit to soon/checkstyle that referenced this issue Aug 14, 2017

soon added a commit to soon/checkstyle that referenced this issue Aug 15, 2017

soon added a commit to soon/checkstyle that referenced this issue Aug 16, 2017

soon added a commit to soon/checkstyle that referenced this issue Aug 16, 2017

soon added a commit to soon/checkstyle that referenced this issue Aug 22, 2017

soon added a commit to soon/checkstyle that referenced this issue Aug 23, 2017

soon added a commit to soon/checkstyle that referenced this issue Aug 24, 2017

romani added a commit that referenced this issue Aug 28, 2017

@romani romani added the bug label Aug 28, 2017

@romani romani changed the title from `XMLLogger` methods should be thread-safe to XMLLogger methods should be thread-safe Aug 28, 2017

@romani romani added this to the 8.2 milestone Aug 28, 2017

@romani

This comment has been minimized.

Show comment
Hide comment
@romani

romani Aug 28, 2017

Member

fix is merged.

Member

romani commented Aug 28, 2017

fix is merged.

@romani romani closed this Aug 28, 2017

@romani

This comment has been minimized.

Show comment
Hide comment
@romani

romani Aug 29, 2017

Member

@soon , please provide fix for pitest failure ASAP:

Survived items:target/pit-reports/201708290005/com.puppycrawl.tools.checkstyle/XMLLogger.java.html:<td class='survived'> target/pit-reports/201708290005/com.puppycrawl.tools.checkstyle/XMLLogger.java.html:<td class='covered'><pre><span class='survived'> fileMessages.clear();</span></pre></td></tr>
Survived items found in reports, build will be failed

https://app.shippable.com/github/checkstyle/checkstyle/runs/4762/19/console
Please also rebase all your further PRs to avoid such issue again.

Update:
log from build , looks like unrelated to current fix

12:07:08 AM PIT >> INFO : Created  19 mutation test units
12:08:41 AM PIT >> WARNING : Minion exited abnormally due to TIMED_OUT
12:11:31 AM PIT >> WARNING : Minion exited abnormally due to TIMED_OUT
12:11:31 AM PIT >> WARNING : Minion exited abnormally due to TIMED_OUT
12:11:31 AM PIT >> WARNING : Minion exited abnormally due to TIMED_OUT
12:11:35 AM PIT >> WARNING : Minion exited abnormally due to TIMED_OUT
12:15:36 AM PIT >> WARNING : Minion exited abnormally due to TIMED_OUT
12:16:33 AM PIT >> WARNING : Minion exited abnormally due to TIMED_OUT
stderr  : Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000de900000, 165150720, 0) failed; error='Cannot allocate memory' (errno=12)
stderr  : Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000f3180000, 58720256, 0) failed; error='Cannot allocate memory' (errno=12)
12:17:27 AM PIT >> WARNING : Error while watching child process
java.util.concurrent.ExecutionException: org.pitest.util.PitError: null

Please copy and paste the information and the complete stacktrace below when reporting an issue
VM : Java HotSpot(TM) 64-Bit Server VM
Vendor : Oracle Corporation
Version : 25.72-b15
Uptime : 752776
Input -> 
 1 : -Dclassworlds.conf=/usr/local/apache-maven-3.2.5/bin/m2.conf
 2 : -Dmaven.home=/usr/local/apache-maven-3.2.5
BootClassPathSupported : true

	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.pitest.util.CommunicationThread.waitToFinish(CommunicationThread.java:60)
	at org.pitest.mutationtest.execute.MutationTestProcess.waitToDie(MutationTestProcess.java:47)
	at org.pitest.mutationtest.build.MutationTestUnit.waitForMinionToDie(MutationTestUnit.java:92)
	at org.pitest.mutationtest.build.MutationTestUnit.runTestInSeperateProcessForMutationRange(MutationTestUnit.java:85)
	at org.pitest.mutationtest.build.MutationTestUnit.runTestsInSeperateProcess(MutationTestUnit.java:68)
	at org.pitest.mutationtest.build.MutationTestUnit.call(MutationTestUnit.java:55)
	at org.pitest.mutationtest.build.MutationTestUnit.call(MutationTestUnit.java:30)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.pitest.util.PitError: null

Please copy and paste the information and the complete stacktrace below when reporting an issue
VM : Java HotSpot(TM) 64-Bit Server VM
Vendor : Oracle Corporation
Version : 25.72-b15
Uptime : 752776
Input -> 
 1 : -Dclassworlds.conf=/usr/local/apache-maven-3.2.5/bin/m2.conf
 2 : -Dmaven.home=/usr/local/apache-maven-3.2.5
BootClassPathSupported : true

	at org.pitest.util.Unchecked.translateCheckedException(Unchecked.java:20)
	at org.pitest.util.SafeDataInputStream.readByte(SafeDataInputStream.java:65)
	at org.pitest.util.SocketReadingCallable.receiveResults(SocketReadingCallable.java:65)
	at org.pitest.util.SocketReadingCallable.call(SocketReadingCallable.java:37)
	at org.pitest.util.SocketReadingCallable.call(SocketReadingCallable.java:12)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	... 1 more
Caused by: java.io.EOFException
	at java.io.DataInputStream.readByte(DataInputStream.java:267)
	at org.pitest.util.SafeDataInputStream.readByte(SafeDataInputStream.java:63)
	... 5 more
12:17:27 AM PIT >> WARNING : Error while watching child process
java.util.concurrent.ExecutionException: org.pitest.util.PitError: null

Please copy and paste the information and the complete stacktrace below when reporting an issue
VM : Java HotSpot(TM) 64-Bit Server VM
Vendor : Oracle Corporation
Version : 25.72-b15
Uptime : 752776
Input -> 
 1 : -Dclassworlds.conf=/usr/local/apache-maven-3.2.5/bin/m2.conf
 2 : -Dmaven.home=/usr/local/apache-maven-3.2.5
BootClassPathSupported : true

	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.pitest.util.CommunicationThread.waitToFinish(CommunicationThread.java:60)
	at org.pitest.mutationtest.execute.MutationTestProcess.waitToDie(MutationTestProcess.java:47)
	at org.pitest.mutationtest.build.MutationTestUnit.waitForMinionToDie(MutationTestUnit.java:92)
	at org.pitest.mutationtest.build.MutationTestUnit.runTestInSeperateProcessForMutationRange(MutationTestUnit.java:85)
	at org.pitest.mutationtest.build.MutationTestUnit.runTestsInSeperateProcess(MutationTestUnit.java:68)
	at org.pitest.mutationtest.build.MutationTestUnit.call(MutationTestUnit.java:55)
	at org.pitest.mutationtest.build.MutationTestUnit.call(MutationTestUnit.java:30)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.pitest.util.PitError: null

Please copy and paste the information and the complete stacktrace below when reporting an issue
VM : Java HotSpot(TM) 64-Bit Server VM
Vendor : Oracle Corporation
Version : 25.72-b15
Uptime : 752776
Input -> 
 1 : -Dclassworlds.conf=/usr/local/apache-maven-3.2.5/bin/m2.conf
 2 : -Dmaven.home=/usr/local/apache-maven-3.2.5
BootClassPathSupported : true

	at org.pitest.util.Unchecked.translateCheckedException(Unchecked.java:20)
	at org.pitest.util.SafeDataInputStream.readByte(SafeDataInputStream.java:65)
	at org.pitest.util.SocketReadingCallable.receiveResults(SocketReadingCallable.java:65)
	at org.pitest.util.SocketReadingCallable.call(SocketReadingCallable.java:37)
	at org.pitest.util.SocketReadingCallable.call(SocketReadingCallable.java:12)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	... 1 more
Caused by: java.io.EOFException
	at java.io.DataInputStream.readByte(DataInputStream.java:267)
	at org.pitest.util.SafeDataInputStream.readByte(SafeDataInputStream.java:63)
	... 5 more
12:17:27 AM PIT >> WARNING : Minion exited abnormally due to RUN_ERROR
12:17:27 AM PIT >> WARNING : Minion exited abnormally due to RUN_ERROR
12:17:55 AM PIT >> WARNING : Minion exited abnormally due to TIMED_OUT
12:18:13 AM PIT >> WARNING : Minion exited abnormally due to MEMORY_ERROR
12:19:35 AM PIT >> WARNING : Minion exited abnormally due to MEMORY_ERROR
12:21:01 AM PIT >> WARNING : Minion exited abnormally due to MEMORY_ERROR
12:21:02 AM PIT >> INFO : Completed in 928 seconds

I can not reproduce this crash on local, pitest validation works fine.
shippable is relaunched.

Member

romani commented Aug 29, 2017

@soon , please provide fix for pitest failure ASAP:

Survived items:target/pit-reports/201708290005/com.puppycrawl.tools.checkstyle/XMLLogger.java.html:<td class='survived'> target/pit-reports/201708290005/com.puppycrawl.tools.checkstyle/XMLLogger.java.html:<td class='covered'><pre><span class='survived'> fileMessages.clear();</span></pre></td></tr>
Survived items found in reports, build will be failed

https://app.shippable.com/github/checkstyle/checkstyle/runs/4762/19/console
Please also rebase all your further PRs to avoid such issue again.

Update:
log from build , looks like unrelated to current fix

12:07:08 AM PIT >> INFO : Created  19 mutation test units
12:08:41 AM PIT >> WARNING : Minion exited abnormally due to TIMED_OUT
12:11:31 AM PIT >> WARNING : Minion exited abnormally due to TIMED_OUT
12:11:31 AM PIT >> WARNING : Minion exited abnormally due to TIMED_OUT
12:11:31 AM PIT >> WARNING : Minion exited abnormally due to TIMED_OUT
12:11:35 AM PIT >> WARNING : Minion exited abnormally due to TIMED_OUT
12:15:36 AM PIT >> WARNING : Minion exited abnormally due to TIMED_OUT
12:16:33 AM PIT >> WARNING : Minion exited abnormally due to TIMED_OUT
stderr  : Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000de900000, 165150720, 0) failed; error='Cannot allocate memory' (errno=12)
stderr  : Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000f3180000, 58720256, 0) failed; error='Cannot allocate memory' (errno=12)
12:17:27 AM PIT >> WARNING : Error while watching child process
java.util.concurrent.ExecutionException: org.pitest.util.PitError: null

Please copy and paste the information and the complete stacktrace below when reporting an issue
VM : Java HotSpot(TM) 64-Bit Server VM
Vendor : Oracle Corporation
Version : 25.72-b15
Uptime : 752776
Input -> 
 1 : -Dclassworlds.conf=/usr/local/apache-maven-3.2.5/bin/m2.conf
 2 : -Dmaven.home=/usr/local/apache-maven-3.2.5
BootClassPathSupported : true

	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.pitest.util.CommunicationThread.waitToFinish(CommunicationThread.java:60)
	at org.pitest.mutationtest.execute.MutationTestProcess.waitToDie(MutationTestProcess.java:47)
	at org.pitest.mutationtest.build.MutationTestUnit.waitForMinionToDie(MutationTestUnit.java:92)
	at org.pitest.mutationtest.build.MutationTestUnit.runTestInSeperateProcessForMutationRange(MutationTestUnit.java:85)
	at org.pitest.mutationtest.build.MutationTestUnit.runTestsInSeperateProcess(MutationTestUnit.java:68)
	at org.pitest.mutationtest.build.MutationTestUnit.call(MutationTestUnit.java:55)
	at org.pitest.mutationtest.build.MutationTestUnit.call(MutationTestUnit.java:30)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.pitest.util.PitError: null

Please copy and paste the information and the complete stacktrace below when reporting an issue
VM : Java HotSpot(TM) 64-Bit Server VM
Vendor : Oracle Corporation
Version : 25.72-b15
Uptime : 752776
Input -> 
 1 : -Dclassworlds.conf=/usr/local/apache-maven-3.2.5/bin/m2.conf
 2 : -Dmaven.home=/usr/local/apache-maven-3.2.5
BootClassPathSupported : true

	at org.pitest.util.Unchecked.translateCheckedException(Unchecked.java:20)
	at org.pitest.util.SafeDataInputStream.readByte(SafeDataInputStream.java:65)
	at org.pitest.util.SocketReadingCallable.receiveResults(SocketReadingCallable.java:65)
	at org.pitest.util.SocketReadingCallable.call(SocketReadingCallable.java:37)
	at org.pitest.util.SocketReadingCallable.call(SocketReadingCallable.java:12)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	... 1 more
Caused by: java.io.EOFException
	at java.io.DataInputStream.readByte(DataInputStream.java:267)
	at org.pitest.util.SafeDataInputStream.readByte(SafeDataInputStream.java:63)
	... 5 more
12:17:27 AM PIT >> WARNING : Error while watching child process
java.util.concurrent.ExecutionException: org.pitest.util.PitError: null

Please copy and paste the information and the complete stacktrace below when reporting an issue
VM : Java HotSpot(TM) 64-Bit Server VM
Vendor : Oracle Corporation
Version : 25.72-b15
Uptime : 752776
Input -> 
 1 : -Dclassworlds.conf=/usr/local/apache-maven-3.2.5/bin/m2.conf
 2 : -Dmaven.home=/usr/local/apache-maven-3.2.5
BootClassPathSupported : true

	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.pitest.util.CommunicationThread.waitToFinish(CommunicationThread.java:60)
	at org.pitest.mutationtest.execute.MutationTestProcess.waitToDie(MutationTestProcess.java:47)
	at org.pitest.mutationtest.build.MutationTestUnit.waitForMinionToDie(MutationTestUnit.java:92)
	at org.pitest.mutationtest.build.MutationTestUnit.runTestInSeperateProcessForMutationRange(MutationTestUnit.java:85)
	at org.pitest.mutationtest.build.MutationTestUnit.runTestsInSeperateProcess(MutationTestUnit.java:68)
	at org.pitest.mutationtest.build.MutationTestUnit.call(MutationTestUnit.java:55)
	at org.pitest.mutationtest.build.MutationTestUnit.call(MutationTestUnit.java:30)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.pitest.util.PitError: null

Please copy and paste the information and the complete stacktrace below when reporting an issue
VM : Java HotSpot(TM) 64-Bit Server VM
Vendor : Oracle Corporation
Version : 25.72-b15
Uptime : 752776
Input -> 
 1 : -Dclassworlds.conf=/usr/local/apache-maven-3.2.5/bin/m2.conf
 2 : -Dmaven.home=/usr/local/apache-maven-3.2.5
BootClassPathSupported : true

	at org.pitest.util.Unchecked.translateCheckedException(Unchecked.java:20)
	at org.pitest.util.SafeDataInputStream.readByte(SafeDataInputStream.java:65)
	at org.pitest.util.SocketReadingCallable.receiveResults(SocketReadingCallable.java:65)
	at org.pitest.util.SocketReadingCallable.call(SocketReadingCallable.java:37)
	at org.pitest.util.SocketReadingCallable.call(SocketReadingCallable.java:12)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	... 1 more
Caused by: java.io.EOFException
	at java.io.DataInputStream.readByte(DataInputStream.java:267)
	at org.pitest.util.SafeDataInputStream.readByte(SafeDataInputStream.java:63)
	... 5 more
12:17:27 AM PIT >> WARNING : Minion exited abnormally due to RUN_ERROR
12:17:27 AM PIT >> WARNING : Minion exited abnormally due to RUN_ERROR
12:17:55 AM PIT >> WARNING : Minion exited abnormally due to TIMED_OUT
12:18:13 AM PIT >> WARNING : Minion exited abnormally due to MEMORY_ERROR
12:19:35 AM PIT >> WARNING : Minion exited abnormally due to MEMORY_ERROR
12:21:01 AM PIT >> WARNING : Minion exited abnormally due to MEMORY_ERROR
12:21:02 AM PIT >> INFO : Completed in 928 seconds

I can not reproduce this crash on local, pitest validation works fine.
shippable is relaunched.

@romani

This comment has been minimized.

Show comment
Hide comment
@romani

romani Aug 29, 2017

Member

I moved this problem to separate issue #5022, as problem is not stable and probably related to shippable ....

Member

romani commented Aug 29, 2017

I moved this problem to separate issue #5022, as problem is not stable and probably related to shippable ....

@romani romani closed this Aug 29, 2017

@soon

This comment has been minimized.

Show comment
Hide comment
@soon

soon Aug 30, 2017

Contributor

@romani Created pr which should fix the pitest coverage: #5025

Contributor

soon commented Aug 30, 2017

@romani Created pr which should fix the pitest coverage: #5025

@sabaka sabaka moved this from In Review to Done in Multi-thread mode for Java files processing Aug 31, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment