NullPointerException when decoding an EXI stream with FidelityOptions.FEATURE_PREFIX

[rovarga]

For OpenDaylight's NETCONF implementation we have tried to switch from OpenEXI to Exificient (https://git.opendaylight.org/gerrit/35131), but our unit tests (which use FidelityOptions.FEATURE_PREFIX) are triggering:

Caused by: java.lang.NullPointerException
at com.siemens.ct.exi.util.xml.QNameUtilities.getQualifiedName(QNameUtilities.java:86)
at com.siemens.ct.exi.core.AbstractEXIBodyDecoder.getAttributeQNameAsString(AbstractEXIBodyDecoder.java:297)
at com.siemens.ct.exi.api.sax.SAXDecoder.handleAttribute(SAXDecoder.java:588)
at com.siemens.ct.exi.api.sax.SAXDecoder.parseEXIEvents(SAXDecoder.java:288)

Which seems to be indicating that the prefix passed to getQualifiedName() is null and trips on pfx.length() check. I am not sure if this is an codec problem (and pfx should never be null), or if the null should be treated as an empty string.

[danielpeintner]

Hi,

I did look at your PR which threats every prefix which is null as empty string on decoder. This might work in some cases but not in all. Moreover, I think the stream is corrupted if Preserve.Prefixes is set to TRUE and does not contain prefixes (i.e. null)

I believe that the problem is not on decoder but on encoder side. IF Preserve.Prefixes is set to TRUE the encoder needs to know the prefix. If the prefix is not reported the codec can't preserve it.

That said, the XMLReader on encoder side need to report NS prefixes properly...

// set XMLReader features
xmlReader.setFeature("http://xml.org/sax/features/namespaces", true);
// do not report namespace declarations as attributes
xmlReader.setFeature("http://xml.org/sax/features/namespace-prefixes", false);

May I ask whether this is done or can you share the test-case and the EXI options you use to track the issue.

Thanks!

[rovarga]

Hello Daniel,

Thanks for your feedback. Slight trouble in this case is the fact that the input comes from a DOM Document via a DOMSource->Transformer->SAXResult. The only reproducer I have right now is an end-to-end test at https://github.com/opendaylight/netconf/blob/master/netconf/netconf-impl/src/test/java/org/opendaylight/netconf/impl/ConcurrentClientsTest.java so getting details requires a bit of fisihing around the repository.

Running it with debugs shows the document being encoded is:
<rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" a="64" id="a" message-id="101" xmlnx="a:b:c:d"> <get-config> <source> <running/> </source> </get-config> </rpc>

so I suspect this is related to the attributes not having an actual prefix, but picking up the default from the rpc element. Output stream specification is byte-aligned, with FEATURE_DTD, FEATURE_PREFIX and FEATURE_LEXICAL_VALUE as far as I can tell.

[danielpeintner]

Hi,

I just updated the exificient-core library so that it throws an error IF prefixes are not properly reported. Please use the latest snapshot version and let me know whether you see such an EXIError("A prefix with value null cannot be used in Preserve.Prefixes mode...")

IF this is the case the issue is on encoder side.

I am currently looking into the case. I think a solution might be to use always the first reported prefix. In many cases there is just one prefix for an uri and not many. This resolves the issue and EXIficient can just report a warning which in your case is not visible.

That said, one cannot differentiate the following 2 XML snipptets.

<ns1:root xmlns:ns1='urn:foo' xmlns:ns2='urn:foo' ...

vs
<ns2:root xmlns:ns1='urn:foo' xmlns:ns2='urn:foo' ...

In most of the cases people don't have multiple prefixes for the same URI.

[danielpeintner]

Note: I think I found a way to "autofix" your issue. That said, if a prefix is empty it gets the first prefix of the given uri.
The only assumption that EXIficient still requires is that namespace declarations are reported in SAX as startPrefixMapping(...) and endPrefixMapping(...) and NOT as attributes.

Please let me know whether the latest 0.9.7-SNAPSHOT resolves your issue.

[rovarga]

I tried with latest SNAPSHOT, but it does look the same. The XML-side wiring is pretty solid, I think: we actually hit namespace issues with Transformer when using Xalan couple of weeks back. This is JDK8u102-internal Transformer and that is definitely fine. As noted, this currently works with OpenEXI (though I have not analyzed the bytestreams, NETCONF relies on proper namespace management, as some data in the XML is actually encoded based on the contents of NamespaceContext.

I don't have too many cycles to spare right now to get familiar with the code (on both sides), and I just skimmed the spec, but my hypothesis is that the decoder does not fill pfx (leaving it null) and then the if in getAttributeQNameAsString() kicks in. If that is what is happening, then checking for Preserve.Prefixes and setting pfx to an empty string should fix the issue.

Looking at

exificient-core/src/main/java/com/siemens/ct/exi/core/AbstractEXIBodyDecoder.java

Line 637 in b78e0c4

protected final void handleAttributePrefix(QNameContext qnc)

it seems very much like handleElementPrefix, but I do not think there will be any more events for the attribute.

[rovarga]

The new PR fixes the issue for me.

[danielpeintner]

Hi,

the PR might fix the issue in your case but the solution is not correct w.r.t. the EXI specification.

I would like to investigate what causes the issue. Once you find time I would to ask you to provide the XML you try to encode along with the EXI stream you get to further analyze the issue.
Note: please provide also the EXI options and schema information if necessary.

Thanks!

[danielpeintner]

713e3d2 solves the problem for me. Confirm or close right away the issue if the fix works for you also.

[andrej-mk]

It works for me too now. Thanks.

[danielpeintner]

@rovarga and @andrej-mk do you think we can close the issue?

[andrej-mk]

One of our unit tests failed before this fix. I've tried it with last 0.9.7-SNAPSHOT and test has passed, so in my opinion it is fixed and can be closed.

[danielpeintner]

Thanks for the feedback!