Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Potential NPE in AbstractBigArray.grow #53835

Closed
mattweber opened this issue Mar 20, 2020 · 2 comments
Closed

Potential NPE in AbstractBigArray.grow #53835

mattweber opened this issue Mar 20, 2020 · 2 comments

Comments

@mattweber
Copy link
Contributor

@mattweber mattweber commented Mar 20, 2020

I have a plugin that uses BigArrays obtained from getHttpTransports. Occasionally, I will get NPE errors during resize and I have no idea what is triggering it. Based on the stacktrace below the NPE is here and is the recycler cache. The only way the recycler is null when using a recycling version of big arrays is after doClose is called. I am not closing and the exception happens inside of my try-with-resources block so it has not been automatically closed.

This is an example of how I am using big arrays and what is triggering the resize and NPE.

try (final BytesStreamOutput respOut = new ReleasableBytesStreamOutput(bigArrays)) {
    XContentFactory.jsonBuilder(respOut).value(searchResponse).flush();
    // do stuff with respOut
}

Relevant part of the stacktrace.

"at org.elasticsearch.common.util.AbstractBigArray.grow(AbstractBigArray.java:100) ~[elasticsearch-7.6.0.jar:7.6.0]",
"at org.elasticsearch.common.util.AbstractBigArray.registerNewPage(AbstractBigArray.java:108) ~[elasticsearch-7.6.0.jar:7.6.0]",
"at org.elasticsearch.common.util.AbstractBigArray.newBytePage(AbstractBigArray.java:118) ~[elasticsearch-7.6.0.jar:7.6.0]",
"at org.elasticsearch.common.util.BigByteArray.resize(BigByteArray.java:143) ~[elasticsearch-7.6.0.jar:7.6.0]",
"at org.elasticsearch.common.util.BigArrays.resizeInPlace(BigArrays.java:440) ~[elasticsearch-7.6.0.jar:7.6.0]",
"at org.elasticsearch.common.util.BigArrays.resize(BigArrays.java:487) ~[elasticsearch-7.6.0.jar:7.6.0]",
"at org.elasticsearch.common.util.BigArrays.grow(BigArrays.java:505) ~[elasticsearch-7.6.0.jar:7.6.0]",
"at org.elasticsearch.common.io.stream.BytesStreamOutput.ensureCapacity(BytesStreamOutput.java:158) ~[elasticsearch-7.6.0.jar:7.6.0]",
"at org.elasticsearch.common.io.stream.ReleasableBytesStreamOutput.ensureCapacity(ReleasableBytesStreamOutput.java:71) ~[elasticsearch-7.6.0.jar:7.6.0]",
"at org.elasticsearch.common.io.stream.BytesStreamOutput.writeBytes(BytesStreamOutput.java:90) ~[elasticsearch-7.6.0.jar:7.6.0]",
"at org.elasticsearch.common.io.stream.StreamOutput.write(StreamOutput.java:517) ~[elasticsearch-7.6.0.jar:7.6.0]",
"at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2039) ~[jackson-core-2.8.11.jar:2.8.11]",
"at com.fasterxml.jackson.core.json.UTF8JsonGenerator.flush(UTF8JsonGenerator.java:1051) ~[jackson-core-2.8.11.jar:2.8.11]",
"at org.elasticsearch.common.xcontent.json.JsonXContentGenerator.flush(JsonXContentGenerator.java:459) ~[elasticsearch-x-content-7.6.0.jar:7.6.0]",
"at org.elasticsearch.common.xcontent.json.JsonXContentGenerator.writeRawField(JsonXContentGenerator.java:351) ~[elasticsearch-x-content-7.6.0.jar:7.6.0]",

"at org.elasticsearch.common.xcontent.json.JsonXContentGenerator.writeRawField(JsonXContentGenerator.java:333) ~[elasticsearch-x-content-7.6.0.jar:7.6.0]",
"at org.elasticsearch.common.xcontent.XContentBuilder.rawField(XContentBuilder.java:976) ~[elasticsearch-x-content-7.6.0.jar:7.6.0]",
"at org.elasticsearch.common.xcontent.XContentHelper.writeRawField(XContentHelper.java:322) ~[elasticsearch-7.6.0.jar:7.6.0]",
"at org.elasticsearch.search.SearchHit.toInnerXContent(SearchHit.java:629) ~[elasticsearch-7.6.0.jar:7.6.0]",
"at org.elasticsearch.search.SearchHit.toXContent(SearchHit.java:565) ~[elasticsearch-7.6.0.jar:7.6.0]",
"at org.elasticsearch.search.SearchHits.toXContent(SearchHits.java:217) ~[elasticsearch-7.6.0.jar:7.6.0]",
"at org.elasticsearch.action.search.SearchResponseSections.toXContent(SearchResponseSections.java:106) ~[elasticsearch-7.6.0.jar:7.6.0]",
"at org.elasticsearch.action.search.SearchResponse.innerToXContent(SearchResponse.java:254) ~[elasticsearch-7.6.0.jar:7.6.0]",
"at org.elasticsearch.action.search.SearchResponse.toXContent(SearchResponse.java:234) ~[elasticsearch-7.6.0.jar:7.6.0]",
"at org.elasticsearch.common.xcontent.XContentBuilder.value(XContentBuilder.java:857) ~[elasticsearch-x-content-7.6.0.jar:7.6.0]",
"at org.elasticsearch.common.xcontent.XContentBuilder.value(XContentBuilder.java:850) ~[elasticsearch-x-content-7.6.0.jar:7.6.0]",
"at org.elasticsearch.common.xcontent.XContentBuilder.unknownValue(XContentBuilder.java:828) ~[elasticsearch-x-content-7.6.0.jar:7.6.0]",
"at org.elasticsearch.common.xcontent.XContentBuilder.value(XContentBuilder.java:804) ~[elasticsearch-x-content-7.6.0.jar:7.6.0]",
@original-brownbear

This comment has been minimized.

Copy link
Member

@original-brownbear original-brownbear commented Mar 20, 2020

@mattweber

That reproducing code looks buggy to me and without seeing the full stack trace/reproducing-code it is very hard to tell what is going on here. Regardless, you will need to close the XContentBuilder after you wrote the value to it.

try (XContentBuilder builder = XContentFactory.jsonBuilder(respOut)) {
            builder.startObject();
            searchResponse.toXContent(builder, ToXContent.EMPTY_PARAMS);
            builder.endObject();
}

This will require to wrap respOut though because your builder will close the stream it wraps.

I don't see how there is a possible bug here at this point so I'm closing this. If you want to discuss the details of the above I'd ask you to open an issue on our discuss forums since we reserve Github for feature requests and confirmed bug reports.
Thanks!

@mattweber

This comment has been minimized.

Copy link
Contributor Author

@mattweber mattweber commented Mar 20, 2020

@original-brownbear thanks, the snippet was intentionally short to try and show where the exception is being thrown from. In the real one code the builder is closed, exception happens before that though. Also, using value on the builder calls toXContent just like your snippet.

Anyways, pretty sure there is a bug in the page recycling code somewhere as using the non-recycling big arrays won't reproduce. Will move to the mailing list if I can't track this down myself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.