Encodings

Milo 1.2.0-SNAPSHOT, Java 17. API baseline 7faaf4f0516c.

Encode and decode OPC UA values with a context that supplies the matching namespace tables, limits, and codecs. An encoding module does not select a network transport or supply an HTTP, WebSocket, or PubSub runtime.

For stored data:

1. Choose a context with the matching model and namespace tables.
2. Select the encoding explicitly.
3. Encode the value and retain its format version and namespace identities.
4. Decode it with the matching context and check the result before relying on the stored representation.

Modules and contexts

Use the same Milo version for milo-stack-core (Binary), milo-encoding-xml, and milo-encoding-json. The latter two supply their respective encoding implementations. The separate codec-json project is an incubating experiment, not the milo-encoding-json module documented here. See Getting started for version alignment.

An EncodingContext supplies DataTypeManager, EncodingManager, namespace/server tables, XML namespace mappings, and EncodingLimits. DefaultEncodingContext starts with namespace 0 and uses the shared JVM-wide OpcUaDataTypeManager singleton. Use it for standard-type examples; do not install application codecs into that singleton. For application-defined types, use a client/server context with the matching model metadata. A namespace index alone is meaningful only against that context's table.

UaEncoder and UaDecoder define the shared primitive, array, Matrix, enum, and structure operations. Binary encoding serves the normal OPC TCP transport. XML and JSON represent values and structures but do not imply an alternative implemented transport. The linked transport-https and transport-websocket modules are incubating and unpublished; WebSocket subprotocol negotiation does not provide a JSON message codec.

Round-trip an ExtensionObject

An ExtensionObject records an encoded structure and its encoding/type identity. In this fragment, context is new DefaultEncodingContext() and encoding is one of OpcUaDefaultBinaryEncoding.getInstance(), OpcUaDefaultXmlEncoding.getInstance(), or OpcUaDefaultJsonEncoding.getInstance():

Range original = new Range(0.0, 100.0);
ExtensionObject encoded = ExtensionObject.encode(context, original, encoding);
Range decoded = (Range) encoded.decode(context, encoding);

Each of the three encodings reconstructs the Range with Low 0.0 and High 100.0. Binary/XML ExtensionObjects identify their encoding node; JSON uses the DataType NodeId. The selected encoding module and codec must exist. Calling the no-encoding overload of ExtensionObject.encode(...) uses Binary. If a required JSON/XML module is absent, decoding fails with Bad_EncodingError and "encoding not registered".

An ExtensionObject caches the first successful decode, even if later calls pass a different context or encoding. Use a fresh ExtensionObject, such as a new read result, after changing contexts or registrations. transcode(...) catches a serialization failure, logs it, and returns the original object; check its result instead of assuming conversion succeeded.

For direct Binary encoder/decoder use, the application owns the Netty ByteBuf and must release it. Close XML/JSON encoders and owned streams/writers as appropriate. Closing a JSON encoder also closes the supplied Writer; closing an XML encoder closes its XMLStreamWriter, while the application still owns the underlying Writer. Do not share mutable encoder/decoder instances across concurrent operations. The high-level ExtensionObject path hides intermediate buffers, but the application still owns external input/output resources.

Current JSON representation

The 1.2 mapping follows the OPC UA 1.05 representation, with differences that include those noted below. JSON syntax alone does not establish a valid OPC UA value; type identity and field rules still apply. The encoder and decoder both default to COMPACT. For VERBOSE input, call decoder.setEncoding(OpcUaJsonEncoder.Encoding.VERBOSE) before decoding.

The decoder does not detect the mode. Match it to the encoder for optional-field, union, and enum-bearing structures. Mode mismatches can raise Bad_DecodingError for unexpected members or incompatible enum tokens. COMPACT writes enum fields as numbers; VERBOSE writes named enums as Name_Value strings. The fixture checks both mismatch directions and successful same-mode decoding for an optional LogRecord and an enum-bearing ServerStatusDataType.

With a standard EncodingContext, this fragment writes and reads a scalar Int32 DataValue. It belongs inside a method that declares or catches Exception, because the encoder's close() declares it:

String json;
try (var encoder = new OpcUaJsonEncoder(context)) {
  encoder.encodeDataValue(null, DataValue.valueOnly(new Variant(42)));
  json = encoder.getOutputString();
}
DataValue decoded = new OpcUaJsonDecoder(context, json).decodeDataValue(null);

The exact output asserted by the fixture is:

{"UaType":6,"Value":42}

This is illustrative output from the executable Java block, also parsed by its decoder. The type tag and Value share the DataValue object; status/timestamp fields are siblings when present. Matrix dimensions remain alongside its flattened values. JSON decoders return primitive arrays for several built-in types; see array representations.

In Variant arrays, null String and ByteString entries use JSON null and keep their positions. String decodes as Java null; ByteString decodes as ByteString.NULL_VALUE. Other element types have no common null rule.

For example, in a Milo-to-Milo Variant array round-trip, a Java null DateTime entry is written as 0001-01-01T00:00:00Z and decodes as a DateTime object whose isNull() is false. DateTime.NULL_VALUE is written as 1601-01-01T00:00:00Z and decodes back to DateTime.NULL_VALUE. This differs from OPC 10000-6 1.05.07 §5.4.2.6, which defines 0001-01-01T00:00:00Z as the null DateTime. Map DateTime nulls explicitly when exchanging JSON with other products, and test that mapping with each peer.

Reject or replace null Boolean and numeric elements before encoding a Variant array. In both JSON modes, Boolean, SByte, Int16, and Int32 nulls encode as JSON null, but decoding fails with Bad_DecodingError. For Byte, UInt16, UInt32, Int64, UInt64, Float, and Double, the encoder throws an unchecked NullPointerException instead of UaSerializationException. Validate other element types against the representation your application needs; Java null does not imply a JSON-null round trip.

A Binary ExtensionObject, including one read over OPC TCP, becomes a JSON envelope with UaTypeId, UaEncoding 1, and a base64 UaBody. An XML ExtensionObject uses UaEncoding 2 and a base64 UTF-8 body. A UaStructuredType placed directly in a Variant first takes the Binary path and therefore also stays opaque.

For inline fields, explicitly encode the structure as JSON first. Here context is new DefaultEncodingContext() and range is new Range(0.0, 100.0); the enclosing method declares throws Exception:

ExtensionObject jsonStructure =
    ExtensionObject.encode(context, range, OpcUaDefaultJsonEncoding.getInstance());
String inline;
try (var encoder = new OpcUaJsonEncoder(context)) {
  encoder.encodeDataValue(null, DataValue.valueOnly(new Variant(jsonStructure)));
  inline = encoder.getOutputString();
}

The decoded value is the same Range; its JSON includes UaTypeId identifying the DataType and a High field of 100, with no UaBody wrapper.

To convert a received Binary/XML body, decode it with its matching context before explicitly encoding the returned structure as JSON. Custom managers must implement reverse getDataTypeId(NodeId) lookup when wrapping Binary/XML ExtensionObjects in JSON; the default null result causes Bad_EncodingError. Wrapping preserves the opaque body and does not convert its fields. The client's LazyClientDataTypeManager only performs reverse lookup on registrations already present; it does not discover an unresolved custom type during getDataTypeId(...). For a freshly read custom Binary ExtensionObject, first resolve manager.getCodec(encoded.getEncodingOrTypeId()) and check for null, or decode the structure before explicitly encoding it as JSON. The local fixture verifies failure before resolution and a successful value-preserving retry afterward.

Migrating 1.1 JSON and custom codecs

Keep an explicit format version with persisted JSON. A successful trial decode does not identify its version:

• A bare scalar Int32 Variant from 1.1.7 already uses UaType and Value; the fixture's value 42 decodes unchanged in 1.2. Compatibility depends on the payload, not just the outer Variant shape. The 1.2 decoder also accepts older Type/Body aliases for Variant members.
• A 1.1.7 DataValue nests that Variant under Value; 1.2 rejects this layout with Bad_DecodingError.
• A 1.1.7 JSON-native ExtensionObject wraps fields under UaBody and already uses the DataType ID. Its envelope can be read in 1.2, but decoding the structure then fails because the old wrapper is not unwrapped.
• A 1.1.7 Binary or XML ExtensionObject envelope uses its encoding ID as UaTypeId. The 1.2 decoder expects the DataType ID, so it rejects these legacy envelopes with Bad_DecodingError while reading them, before decoding the inner structure. This also rejects a bare Variant containing such an envelope. XML has a second change: 1.1.7 stored XML text in UaBody, while 1.2 expects base64 UTF-8. Replacing only the ID is insufficient for XML.

The two Milo versions use the same Java package names. Convert in separate processes or isolated class loaders, with the old and new codecs each matched to their model:

1. In the old process, read the explicit format version and decode the old JSON. Check quality, value types, and shape before exporting.
2. Write an intermediate both versions understand. The fixtures use OPC UA Binary plus a versioned manifest. For application models, record the old process's namespace URI table in index order alongside the intermediate, because Binary NodeIds and encoding IDs carry indexes. Decode and re-encode JSON/XML ExtensionObject bodies as Binary first; do not relabel their bytes.
3. In the 1.2 process, restore the namespace table from the recorded URIs in the same index order and use the matching codecs to read the intermediate. Check its results and explicitly encode the current JSON representation. Decode it again and compare semantic results before replacing stored data.

The paired JsonMigrationTest fixtures run against released 1.1.7 and this 1.2 baseline in separate JVMs. They migrate an Int32, unsigned array with quality and both timestamps, valid 2-by-2 Matrix, ByteString, and Range structures whose legacy envelopes contain JSON, Binary, or XML bodies. The old process exports real legacy JSON and decodes it before creating the Binary intermediate; the new process checks both envelope rejection and semantic conversion. Run the old project first and preserve its exchange directory until the new project finishes.

The old encoder already collapses a null ByteString into an empty string; the migration preserves the resulting empty bytes and cannot recover the lost null identity. The fixtures also check the old/new null Matrix field representation: the same Binary bytes decode to Java null in 1.1.7 and Matrix.ofNull() in 1.2.

The fixtures use namespace 0 only. Binary is not a blanket compatibility guarantee. Reject invalid dimensions and count mismatches before export; test application-specific structures, optional fields, unions, empty values, and actual stored records separately. Keep backups. See the versioned migration guide.

Custom codecs must replace generic UInt32 writes/reads of an EncodingMask or SwitchField with encodeEncodingMask/decodeEncodingMask and encodeSwitchField/decodeSwitchField. The decoder forms take declared field names, so VERBOSE JSON can associate named optional/union fields correctly. A codec that happens to work in Binary is not therefore valid in VERBOSE JSON. Use the corresponding struct/enum/Matrix methods instead of hand-encoding each format's representation.

XML no longer obtains JAXB dependencies through milo-encoding-xml; applications using JAXB must declare it themselves. XML field namespace mappings follow the declaring model and can differ for inherited fields. This is distinct from namespace indexes in NodeIds. Use XmlDataTypeCodec for declaring-model mappings and EncodingContext.withXmlNamespaceUris(...) for model-URI-to-schema-namespace mappings. Without a context mapping, the model URI is used as the XML namespace. Do not strip namespace information to make input appear compatible.

Limits, malformed input, and unsupported combinations

JSON decoding bounds input by MaxMessageSize UTF-16 code units, including whitespace. A value of zero or less still uses the 2 MiB default. MaxRecursionDepth bounds buffered containers and nested ExtensionObject decoding; it does not count the streamed outer DataValue/Variant wrapper. Exceeding either bound raises Bad_EncodingLimitsExceeded. These limits are separate from OPC TCP framing and service operation limits.

The decoder accepts valid JSON object field reordering. Structures and buffered Variant/ExtensionObject bodies reject duplicate members, but top-level DataValue and Variant members are not checked for duplicates: the last occurrence wins. This includes Value, Dimensions, Status, timestamps, and the UaType/Type aliases. Directly streamed built-in objects such as LocalizedText have the same limitation. Do not use this decoder as a general duplicate-key validator; reject duplicate keys before decoding if your application requires that guarantee.

The local fixture checks this top-level DataValue limitation separately from failures: duplicate members inside a buffered Value object, unexpected DataValue members, and a nested Variant without the required sibling type tag produce Bad_DecodingError. It also verifies a truncated Binary Range and an out-of-range XML Int32 fail with decoding errors. These failures are useful diagnostics, not a reason to accept malformed data as a different type.

XML arrays must use the expected direct-child structure. Invalid lexical values, integer overflow, and date-only DateTimes fail with Bad_DecodingError. Validate custom codecs for namespaces, optional/union headers, empty/null values, and unread extra fields in every format you use.

For a Matrix inside a Binary Variant, an empty Matrix with zero/negative dimensions becomes an empty one-dimensional array. A nonempty Matrix fails with Bad_EncodingError unless it has at least two dimensions and every dimension is positive. A positive dimension product that differs from the element count can encode successfully but fails decoding with Bad_DecodingError.

A Binary structure-field encodeMatrix(...) preserves nonnegative dimensions such as [0, 2]. If any dimension is zero and none is negative, the encoder writes no elements, so a nonempty Matrix silently decodes as empty. With all dimensions positive, the element count is checked only by a Java assertion, which the JVM disables by default. The encoder can write a negative dimension, but the field decoder rejects it with Bad_DecodingError. Enforce shape and element-count invariants in application code before encoding.

An empty XML structure-field Matrix (no elements and a zero or negative dimension) is written as nil and decodes as Matrix.ofNull(). Its original dimensions and empty-versus-null distinction are lost. Account for these format-specific conversions when handling null and empty values.

On JSON decode, an unknown namespace URI does not raise an error. A NodeId becomes a namespace-0 String NodeId containing the original text; a QualifiedName becomes a namespace-0 name containing it. The result will not identify the intended node or browse name. On encoding, a missing namespace index has different consequences. QualifiedName and index-based ExpandedNodeId raise Bad_EncodingError. NodeId instead becomes a namespace-0 String identity containing its original parseable text: numeric ns=5;i=1 encodes as s=ns=5;i=1. This also changes the node identity; validate the namespace table before encoding.

The encoder writes a nonzero QualifiedName namespace as nsu=<uri>;<name>. A numeric prefix such as 1:Name decodes as namespace 1, including when that text was the name of a namespace-0 QualifiedName. Such a name does not preserve its identity through this string grammar.

An unregistered codec raises Bad_DecodingError on decode; an unresolved DataType ID raises Bad_EncodingError while wrapping a Binary/XML ExtensionObject in JSON. An absent JSON/XML module raises Bad_EncodingError with "encoding not registered". Handle UaSerializationException and inspect its status and cause, including when the selected encoding does not match the ExtensionObject kind. Validate array elements before encoding; the JSON null-element cases can throw an unchecked NullPointerException. A successful local codec round-trip proves that representation for the tested context; exchange with another product still requires peer-specific testing.

Examples and reference

• Legacy JSON exporter (Milo 1.1.7)
• Current JSON importer (Milo 1.2.0-SNAPSHOT)
• Runnable fixture: WikiEncodingGuidesTest

Encoding contracts and implementation:

• Stack module list
• Shared encoder contract
• JSON encoder
• JSON decoder and limits
• XML codec namespace adapter
• ExtensionObject identity and conversions

Related pages

Reference · Data types · 1.2.0 migration guide