-
Notifications
You must be signed in to change notification settings - Fork 24.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
XContentBuilder.ensureNoSelfReferences has an impact on performance #26907
Comments
For the record, this sanity check is important important so that poison documents cannot fail an entire cluster. However maybe there are ways that we could speed it up a bit, eg. by
|
It's interesting that
|
I only change -Xms and -Xmx, the full JVM arguments is as follows:
|
I think the reason is the Number of executions: number of documents * number of fields. Maybe the ensureNoSelfReferences method can be overloaded with a check switch, and some internal calls can skip the check? |
Instead of explicitly checking for self references, it might be simpler/faster to track the current stack depth of the structure that's being used to populate the xcontentbuilder and just check that it does not pass a certain threshold (e.g. 100). This could be done for example by overloading some of the methods with an internal method that takes an additional "stack depth" parameter ( |
Thanks for confirming @jgq2008303393, it's unlikely to be the source of the problem here then. We will need some benchmarks to validate any of the suggested approaches. |
Currently we test all maps, arrays or iterables. However, in the case that maps contain sub maps for instance, we will test the sub maps again even though the work has already been done for the top-level map. Relates elastic#26907
Currently we test all maps, arrays or iterables. However, in the case that maps contain sub maps for instance, we will test the sub maps again even though the work has already been done for the top-level map. Relates #26907
Currently we test all maps, arrays or iterables. However, in the case that maps contain sub maps for instance, we will test the sub maps again even though the work has already been done for the top-level map. Relates #26907
Pinging @elastic/es-core-infra |
Pinging @elastic/es-perf (Team:Performance) |
calling this completed via #87337 |
Hi, all.
When using pipeline to parse log, I found that XContentBuilder.ensureNoSelfReferences has an impact on performance. This method cost about 17.36% of the total CPU resource.
The text was updated successfully, but these errors were encountered: