-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Race condition triggered in java-generator for very big schemas #4723
Comments
I'm completely puzzled by those errors, let me try to summon a couple of javaparser maintainers for a clue on where to start debugging here 🙏 cc. @jlerbsc @ftomassetti |
Hi @andreaTP What version of JavaParser are you using? |
The 2 stacktraces seem to correspond to processing on the annotations. So I tried to produce, with JP 3.24.10, a simple test case with one of the code sections that seems to cause the error.
Here is the produced result
Does this correspond to what you want to generate? |
Hi @jlerbsc , thanks a lot for looking into this, much appreciated!
and yes, your output is very similar to the expected one ( e.g. is comparable with the approval tests output ). |
Are you modifying the AST with multiple threads? |
No, we are processing multiple CRDs in parallel and we are parallelizing the write to disk, but the processing is executed on a single thread |
On which version of java are you getting this stacktrace?
Ideally you should be able to isolate a simplified test case that reproduces the issue because otherwise it will be difficult to help you. |
I completely understand, I will try to create a more minimal reproducer for the issue 👍 I was trying to see if the StackTrace itself rings any bell 🙂 |
Were you able to reproduce your error with a simplified test case? |
Hi @andreaTP Maybe it is possible to try to synchronize JObject.generateJava method. |
Thanks for the suggestion @jlerbsc ! I have done it(out of desperation) but it doesn't seem to have any impact. I was analyzing the issue with @mariofusco (thanks again 🙏 ) today, and we are starting to believe that there is a shared element "somewhere" in the stack that is causing errors to happen randomly because of a race condition. Please note that the error is not "stable" and it's happening at different depths of the AST and on different elements, excluding that the issue depends on the actual input. |
JP sometimes uses static methods, in particular the lexical preserving printer and the symbol solver, which is certainly the reason why the designers of this project have always proclaimed loud and clear that JP was not thead safe. There have been initiatives to try to improve things here and there, but there may still be some code that is not thread safe. |
Thanks a lot for this honest explanation @jlerbsc and for staying involved in this issue/discussion, much appreciated! Tomorrow both me and @mariofusco are probably going to spend a few more hours trying to narrow down this issue.
This action is surely going to impact the performance, but, at this stage, we should favor the correctness and get back to performance when we see the need. |
Thanks to @mariofusco that found out that the race condition is happening on this node incorrectly encoded: Thanks again for the support to everyone involved! |
Hey, I've just ran into this - is there any workaround or snapshot version I could use until 6.5.0 is out? |
Hi @lmyslinski , I failed to lookup your account when I found out that the problem was reliably reproducible with you CRD. |
@andreaTP Hmm I'm afraid that the plugin is not pushed to snapshots:
I've added the following snapshot repo:
|
I've figured it out, plugins have a separate repo scope:
All good now, thx a lot! |
@lmyslinski I was about to answer 🙂 but you already got it 👍 thanks for the feedback! |
Describe the bug
Using different versions of Java produce different errors with the
java-generator
Fabric8 Kubernetes Client version
SNAPSHOT
Steps to reproduce
curl -Ls https://raw.githubusercontent.com/istio/istio/master/manifests/charts/base/crds/crd-all.gen.yaml -o istio.yaml
sdk use java 11.0.17-tem
jbang io.fabric8:java-generator-cli:6.3.1 --add-extra-annotations --source istio.yaml --target ./src
Produces a stack-trace like:
Additional issues can be experienced by mixing and matching Java and
kubernetes-client
versions, another example stack trace is:Expected behavior
The generator correctly run and generate the sources.
Runtime
other (please specify in additional context)
Kubernetes API Server version
1.25.3@latest
Environment
Windows, Linux, macOS
Fabric8 Kubernetes Client Logs
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: