Skip to content

Fix of CME in MyJavaFileManager.list #104

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Apr 6, 2022
Merged

Fix of CME in MyJavaFileManager.list #104

merged 1 commit into from
Apr 6, 2022

Conversation

glukos
Copy link
Contributor

@glukos glukos commented Apr 5, 2022 •
edited
Loading

MyJavaFileManager#list is synchonized, but JavacFileManager#pathsAndContainersByLocationAndRelativeDirectory can be modified by another thread when executing MyJavaFileManager#handleOption.

Thread that throws CME:

"pool-1-thread-10@1712" prio=5 tid=0x1d nid=NA runnable
  java.lang.Thread.State: RUNNABLE
	  at com.sun.tools.javac.file.JavacFileManager.indexPathsAndContainersByRelativeDirectory(JavacFileManager.java:1030)
	  at com.sun.tools.javac.file.JavacFileManager$$Lambda$286/0x0000000800dad838.apply(Unknown Source:-1)
	  at java.util.HashMap.computeIfAbsent(HashMap.java:1220)
	  at com.sun.tools.javac.file.JavacFileManager.pathsAndContainers(JavacFileManager.java:1018)
	  at com.sun.tools.javac.file.JavacFileManager.list(JavacFileManager.java:774)
	  at net.openhft.compiler.MyJavaFileManager.list(MyJavaFileManager.java:91)
	  - locked <0xe0c> (a net.openhft.compiler.MyJavaFileManager)
	  at com.sun.tools.javac.api.ClientCodeWrapper$WrappedJavaFileManager.list(ClientCodeWrapper.java:223)
	  at com.sun.tools.javac.code.ClassFinder.list(ClassFinder.java:737)
	  at com.sun.tools.javac.code.ClassFinder.scanUserPaths(ClassFinder.java:674)
	  ...
	  at com.sun.tools.javac.comp.Attr.attrib(Attr.java:5141)
	  at com.sun.tools.javac.main.JavaCompiler.attribute(JavaCompiler.java:1317)
	  at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:946)
	  at com.sun.tools.javac.api.JavacTaskImpl.lambda$doCall$0(JavacTaskImpl.java:104)
	  at com.sun.tools.javac.api.JavacTaskImpl$$Lambda$204/0x0000000800d0f028.call(Unknown Source:-1)
	  at com.sun.tools.javac.api.JavacTaskImpl.invocationHelper(JavacTaskImpl.java:152)
	  at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:100)
	  at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:94)
	  at net.openhft.compiler.CachedCompiler.compileFromJava(CachedCompiler.java:112)
	  at net.openhft.compiler.CachedCompiler.loadFromJava(CachedCompiler.java:151)
	  at net.openhft.compiler.CachedCompiler.loadFromJava(CachedCompiler.java:77)
	  at net.openhft.chronicle.wire.GenerateMethodReader.createClass(GenerateMethodReader.java:104)
	  at net.openhft.chronicle.wire.VanillaMethodReaderBuilder.lambda$createGeneratedInstance$1(VanillaMethodReaderBuilder.java:132)
	  at net.openhft.chronicle.wire.VanillaMethodReaderBuilder$$Lambda$467/0x0000000800e010d8.apply(Unknown Source:-1)
	  at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1708)
	  - locked <0xe28> (a java.util.concurrent.ConcurrentHashMap$ReservationNode)
	  at net.openhft.chronicle.wire.VanillaMethodReaderBuilder.createGeneratedInstance(VanillaMethodReaderBuilder.java:132)
	  at net.openhft.chronicle.wire.VanillaMethodReaderBuilder.build(VanillaMethodReaderBuilder.java:172)
	  at net.openhft.chronicle.wire.MethodReaderInterceptorReturnsTest.doTestInterceptorSupportedInGeneratedCode(MethodReaderInterceptorReturnsTest.java:112)
	  at net.openhft.chronicle.wire.MethodReaderInterceptorReturnsTest.lambda$testInterceptingReaderConcurrentCreation$0(MethodReaderInterceptorReturnsTest.java:63)
	  at net.openhft.chronicle.wire.MethodReaderInterceptorReturnsTest$$Lambda$95/0x0000000800c8a400.run(Unknown Source:-1)
	  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	  at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264)
	  at java.util.concurrent.FutureTask.run(FutureTask.java:-1)
	  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	  at java.lang.Thread.run(Thread.java:833)

Concurrent thread:

"pool-1-thread-7@1706" prio=5 tid=0x1a nid=NA runnable
  java.lang.Thread.State: RUNNABLE
	  at net.openhft.compiler.MyJavaFileManager.handleOption(MyJavaFileManager.java:103)
	  at com.sun.tools.javac.api.ClientCodeWrapper$WrappedJavaFileManager.handleOption(ClientCodeWrapper.java:256)
	  at com.sun.tools.javac.api.JavacTool.getTask(JavacTool.java:197)
	  at com.sun.tools.javac.api.JavacTool.getTask(JavacTool.java:119)
	  at com.sun.tools.javac.api.JavacTool.getTask(JavacTool.java:68)
	  at net.openhft.compiler.CachedCompiler.compileFromJava(CachedCompiler.java:105)
	  at net.openhft.compiler.CachedCompiler.loadFromJava(CachedCompiler.java:151)
	  at net.openhft.compiler.CachedCompiler.loadFromJava(CachedCompiler.java:77)
	  at net.openhft.chronicle.wire.GenerateMethodWriter.createClass(GenerateMethodWriter.java:337)
	  at net.openhft.chronicle.wire.GenerateMethodWriter.newClass(GenerateMethodWriter.java:142)
	  at net.openhft.chronicle.wire.VanillaMethodWriterBuilder.newClass(VanillaMethodWriterBuilder.java:226)
	  at net.openhft.chronicle.wire.VanillaMethodWriterBuilder$$Lambda$171/0x0000000800cbe4b0.apply(Unknown Source:-1)
	  at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1740)
	  - locked <0xe25> (a java.util.concurrent.ConcurrentHashMap$Node)
	  at net.openhft.chronicle.wire.VanillaMethodWriterBuilder.createInstance(VanillaMethodWriterBuilder.java:208)
	  at net.openhft.chronicle.wire.VanillaMethodWriterBuilder.get(VanillaMethodWriterBuilder.java:190)
	  at net.openhft.chronicle.wire.VanillaMethodWriterBuilder.build(VanillaMethodWriterBuilder.java:135)
	  at net.openhft.chronicle.wire.MarshallableOut.methodWriter(MarshallableOut.java:223)
	  at net.openhft.chronicle.wire.MarshallableOut.methodWriter(MarshallableOut.java:201)
	  at net.openhft.chronicle.wire.MethodReaderInterceptorReturnsTestInterceptedInterfaceMethodReaderInterceptorReturnsTestInterceptedChainedBinarylightMethodWriter.lambda$new$0(MethodReaderInterceptorReturnsTestInterceptedInterfaceMethodReaderInterceptorReturnsTestInterceptedChainedBinarylightMethodWriter.java:25)
	  at net.openhft.chronicle.wire.MethodReaderInterceptorReturnsTestInterceptedInterfaceMethodReaderInterceptorReturnsTestInterceptedChainedBinarylightMethodWriter$$Lambda$407/0x0000000800df5158.get(Unknown Source:-1)
	  at java.lang.ThreadLocal$SuppliedThreadLocal.initialValue(ThreadLocal.java:305)
	  at java.lang.ThreadLocal.setInitialValue(ThreadLocal.java:195)
	  at java.lang.ThreadLocal.get(ThreadLocal.java:172)
	  at net.openhft.chronicle.wire.MethodReaderInterceptorReturnsTestInterceptedInterfaceMethodReaderInterceptorReturnsTestInterceptedChainedBinarylightMethodWriter.twoArgs(MethodReaderInterceptorReturnsTestInterceptedInterfaceMethodReaderInterceptorReturnsTestInterceptedChainedBinarylightMethodWriter.java:50)
	  at net.openhft.chronicle.wire.MethodReaderInterceptorReturnsTest.doTestInterceptorSupportedInGeneratedCode(MethodReaderInterceptorReturnsTest.java:123)
	  at net.openhft.chronicle.wire.MethodReaderInterceptorReturnsTest.lambda$testInterceptingReaderConcurrentCreation$0(MethodReaderInterceptorReturnsTest.java:63)
	  at net.openhft.chronicle.wire.MethodReaderInterceptorReturnsTest$$Lambda$95/0x0000000800c8a400.run(Unknown Source:-1)
	  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	  at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264)
	  at java.util.concurrent.FutureTask.run(FutureTask.java:-1)
	  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	  at java.lang.Thread.run(Thread.java:833)

This is obviously not a proper fundamental solution: we still have concurrent access to HashMap inside file manager. But from a practical standpoint, this fix decreases change of CME (when compiling 10+ method readers in parallel) from ~5% to "can't reproduce".

@glukos glukos requested a review from JerryShea April 5, 2022 17:08
@glukos glukos changed the base branch from ea to develop April 5, 2022 17:08
@sonarqubecloud
Copy link

sonarqubecloud bot commented Apr 5, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@glukos glukos linked an issue Apr 5, 2022 that may be closed by this pull request
Sporadic ConcurrentModificationException when compiling lots of method readers under Java 17 #103
Closed
@glukos glukos merged commit a303c11 into develop Apr 6, 2022
@glukos glukos mentioned this pull request May 11, 2022
Closed
@peter-lawrey peter-lawrey deleted the issue103 branch August 2, 2023 10:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
1 more reviewer

@JerryShea JerryShea JerryShea approved these changes

Reviewers whose approvals may not affect merge requirements
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Sporadic ConcurrentModificationException when compiling lots of method readers under Java 17
2 participants
@glukos @JerryShea