Diagnostics and Wireshark

Milo 1.2.0-RC1, Java 17. API baseline 69cb25087804.

When Troubleshooting points you to the boundary that failed but not to the cause, you need evidence from more than one place. This page gathers it from three sources, in the order you usually reach for them, without losing the context that connects them. You start with logs from both peers, then read the server's own diagnostics nodes, then decrypt a packet capture with a key log.

Match the evidence to the layer

Each source sees a different layer. Service results and exception causes show what the application asked for and what came back. Logs show each peer's view of the SecureChannel and Session, including causes that the wire hides, such as the certificate problem behind a Bad_SecurityChecksFailed. Server diagnostics show the server's counters and its current Sessions and subscriptions. A packet capture shows what actually crossed the network, but secured traffic stays encrypted unless one peer exports its channel keys.

Start with the narrowest source that covers the failing layer, and widen only when it runs out. Keep the identifiers that tie the sources together, namely request timing, channel and Session identifiers, and the exact request, so a log line from either peer, a diagnostics entry, and a captured packet can be matched to the same event.

Collect evidence and select logs

Start from the facts listed under Collect evidence, and add the application URIs, the timing, and the stage that failed, whether discovery, channel creation, Session activation, a service call, sampling, or recovery. A successful TCP connection does not prove that a Session exists.

Milo logs through SLF4J, so the application's logging implementation and configuration decide what is recorded. Start with DEBUG for the narrow affected component, and restore ordinary levels after the capture.

Category	Useful evidence
org.eclipse.milo.opcua.sdk.client.SessionFsm	Session creation, activation, recovery, Session service failures, and keep-alive failures.
org.eclipse.milo.opcua.sdk.client.subscriptions	Subscription and monitored-item synchronization and publishing behavior.
org.eclipse.milo.opcua.sdk.server.sampling	SDK sampling coordination. A group logs failures, stalled turns, and overruns under its concrete implementation class. For a custom group such as DeviceSamplingGroup, enable that application class's logger.
org.eclipse.milo.opcua.stack.client.ChannelFsm	Client channel connect, reconnect, and disconnect transitions.
org.eclipse.milo.opcua.stack.transport	Transport handler traffic, handshake processing, and transport errors.
org.eclipse.milo.opcua.stack.core.security	Certificate validation causes, including errors masked on the wire.
org.eclipse.milo.opcua.stack.core.util.validation.CertificateValidationUtil	Certificate path construction and suppressed-check warnings.
org.eclipse.milo.opcua.stack.core.channel.WiresharkKeyLogWriter	Key-file I/O failures.

High-volume transport logging can change timing and contain application data, so collect the smallest useful interval. Before sharing, redact user identities, credentials, payloads, and private environment details. Never paste key-log contents into an issue or an ordinary application log.

Read server diagnostics

The standard ServerDiagnostics subtree exposes summary counters plus Session and subscription diagnostics. Server.ServerDiagnostics.EnabledFlag gates the diagnostic values under ServerDiagnostics, and it starts false. While it is false, static diagnostics Variables stay browsable and readable but return Bad_OutOfService, and per-Session diagnostic Objects and array entries do not exist.

Enabling the flag creates entries for existing Sessions and Subscriptions, and disabling it removes them. A Write to the flag takes effect before the Write returns, so browse again after changing it. Summary counters keep counting while the flag is false, so enabling it later exposes totals since the server started.

The application that owns the server can switch diagnostics on directly, without a Session. This method does that for any running OpcUaServer, such as the tutorial thermostat. The owning application must restore its intended EnabledFlag after collecting evidence, for example by calling the method again with false.

static void setDiagnosticsEnabled(OpcUaServer server, boolean enabled) {
  AddressSpaceManager addressSpace = server.getAddressSpaceManager();
  UaNode node = addressSpace.getManagedNode(NodeIds.Server_ServerDiagnostics).orElseThrow();
  var diagnostics = (ServerDiagnosticsTypeNode) node;

  diagnostics.setEnabledFlag(enabled);
}

ServerDiagnosticsTypeNode comes from org.eclipse.milo.opcua.sdk.server.model.objects. The server-side setter bypasses client authorization, so keep it on a control path that only the owning application or its administrators can reach.

A client then reads the values over an ordinary Session. This method reads CurrentSessionCount and checks the Read status before using the value. Reading the summary needs no diagnostic role, so a client connected to the tutorial endpoint opc.tcp://127.0.0.1:12686/wiki, as in Your first client, can call it.

static UInteger readSessionCount(OpcUaClient client) throws UaException {
  NodeId countId = NodeIds.Server_ServerDiagnostics_ServerDiagnosticsSummary_CurrentSessionCount;
  DataValue count = client.readValue(0, TimestampsToReturn.Neither, countId);

  StatusCode status = count.statusCode();
  if (!status.isGood()) {
    throw new UaException(status);
  }

  return (UInteger) count.value().value();
}

With diagnostics enabled, the tutorial server returns 1 when this is the only client, because the count includes the client doing the Read. With diagnostics disabled, the method throws a UaException carrying Bad_OutOfService.

Changing the flag remotely and reading Session security diagnostics both need authorization. A remote Write to EnabledFlag requires a RoleMapper that grants ConfigureAdmin or SecurityAdmin, or an equivalent configured node permission. In the default RESTRICTED mode, Session security diagnostics require SecurityAdmin and a SignAndEncrypt channel. Configure identities through server role mapping. These results tell you which requirement a read or write missed.

Result	Meaning and action
Bad_OutOfService	Diagnostics are disabled. An authorized application or user must enable them before reading their values.
Bad_UserAccessDenied	The Session lacks the required role. Encryption alone does not authorize a read or write.
Bad_SecurityModeInsufficient	The channel does not meet signing and encryption restrictions. This check precedes role checks, so both None and Sign-only fail for security diagnostics.

OpcUaServerConfigBuilder.setSessionSecurityDiagnosticsAccessMode(LEGACY) lets every Session read security diagnostics and toggle EnabledFlag, subject to AccessRestrictions. That exposes other Sessions' user identities and client certificates, so prefer RESTRICTED mode with an authorized diagnostic identity.

SamplingIntervalDiagnosticsArray is removed at startup and is not available. The server diagnostics contracts describe the role and lifecycle rules above. Diagnostics values start over with each server instance, so persist selected metrics externally if you need trends across restarts.

Decrypt a capture with a key log

A packet capture usually shows transport framing, but secured service payloads stay encrypted. To read them, one peer exports the symmetric keys of its SecureChannel, and Wireshark uses them to decrypt. WiresharkKeyLogWriter writes those keys in Wireshark's OPC UA key-log format. Key export is off unless a SecurityKeysListener is configured. A None-policy channel derives no such keys and needs none, because its traffic is not encrypted. A capture of the tutorial endpoint is readable as it is.

The procedure has three parts. You attach a writer to the client or server, capture the secured operation while the writer is open, and then decode the capture with the matching key file. Afterwards, handle the files as described in Handle capture resources, because together the key file and the capture expose everything the capture covers.

Attach a key-log writer

The tutorial server has no secured endpoint, so this step needs a server with a Basic256Sha256 SignAndEncrypt endpoint that trusts the client's application certificate. This helper selects that endpoint and reads the server's CurrentTime with key logging. Provide these inputs.

• a fresh, private keyLogPath whose parent directory exists
• a reachable discovery URL
• the application key pair and certificate chain
• a configured server certificate validator, as in Client security

The writer does not set file permissions. On POSIX systems, create an owner-only directory or pre-create an owner-only file. Otherwise the process umask controls the file mode, which may allow other users to read it. Path, TimeUnit, Function, the collection types, and the Java security types come from the JDK. The linked sample source has all Milo imports.

static DataValue readWithKeyLog(
    Path keyLogPath,
    String discoveryUrl,
    KeyPair keyPair,
    X509Certificate[] certificateChain,
    CertificateValidator validator)
    throws Exception {
  Function<List<EndpointDescription>, Optional<EndpointDescription>> selectEncrypted =
      endpoints ->
          endpoints.stream()
              .filter(
                  e -> SecurityPolicy.Basic256Sha256.getUri().equals(e.getSecurityPolicyUri()))
              .filter(e -> e.getSecurityMode() == MessageSecurityMode.SignAndEncrypt)
              .findFirst();

  try (var writer = new WiresharkKeyLogWriter(keyLogPath)) {
    OpcUaClient client =
        OpcUaClient.create(
            discoveryUrl,
            selectEncrypted,
            transport -> {},
            config ->
                config
                    .setCertificateIdentity(keyPair, certificateChain)
                    .setCertificateValidator(validator)
                    .setSecurityKeysListener(writer));
    try {
      client.connectAsync().get(10, TimeUnit.SECONDS);
      return client.readValue(
          0, TimestampsToReturn.Both, NodeIds.Server_ServerStatus_CurrentTime);
    } finally {
      client.disconnectAsync().get(10, TimeUnit.SECONDS);
    }
  }
}

The order of the two scopes matters. The helper awaits the client disconnect before the outer resource scope closes the writer, so every key the channel derives reaches the file. On a server, likewise stop channel activity and await shutdown before closing its key writer.

A timeout reports failed cleanup but does not cancel arbitrary in-flight transport work. If disconnect times out, the writer can close while the channel is still open and silently drop later renewal keys, so treat that capture as possibly incomplete. In this simple helper, a cleanup exception can also replace the original exception.

OpcUaClient.create can throw UaException during discovery or endpoint selection. It retries discovery at a URL ending in /discovery, so the retry's error can hide the original failure. If selection still finds no matching endpoint, the result is Bad_ConfigurationError. Connection failures and timeouts propagate from the waits. The writer opens first, so a discovery failure such as Bad_ConnectionRejected leaves an empty key file.

What the writer records

The writer appends to an existing file and flushes each keyset. Prefer one fresh file per capture so channel and token identifiers from unrelated runs do not mix. Handshakes and renewals can add records, so pair the file with its capture and keep the writer open for the entire interval.

A client writer records both directions for its channel. A server writer records both directions for every secured client channel while configured, which can expose traffic beyond the connection under investigation. The writer never writes an application private key, but the exported symmetric keys can reveal captured payloads and credentials.

Listener callbacks run synchronously during channel processing. A slow listener delays the handshake and other channels on the same Netty event-loop thread. A listener that throws fails the channel open or renewal and closes the connection. Custom listeners must therefore be thread-safe, return promptly, contain their own errors, and avoid unbounded I/O. The built-in writer logs I/O failures instead of throwing, so a successful Session alone does not prove the file was written.

Capture and inspect

1. Before connecting, start a capture on the interface that carries the traffic. A client and server on one host need a loopback capture. Filter on the configured OPC TCP port to keep the capture focused.
2. Run the secured operation with the key writer attached. Check the operation's status independently of the capture.
3. Disconnect the client or shut down the server, close the writer, and stop the capture. Keep the packet and key files paired.
4. Inspect with the matching key file and the correct server port. In TShark 4.4.19, the relevant preferences are opcua.debug_file (labeled "OPCUA debug file") and opcua.tcp.port. Decode As alone selects the dissector but does not set the port used to tell client keys from server keys.
5. Confirm the service request, the matching response, and both service and operation statuses. Valid-looking key lines alone do not prove decryption.

This command runs step 4 with TShark 4.4.19 for a server that listens on Milo's default port 12685. The -V option prints the decoded protocol fields, not only frame summaries. Replace the two file paths under /capture and the port with your own.

tshark -r /capture/opcua.pcap -o opcua.tcp.port:12685 -o opcua.debug_file:/capture/opcua.keys -Y opcua -V

The client issues other Reads during initialization, so match the CurrentTime NodeId (i=2258) and the RequestHandle to find the helper's Read. None-policy discovery appears as a separate plaintext stream. OpenSecureChannel frames use asymmetric cryptography, so they stay encrypted even when the symmetric service traffic decrypts.

With the key file, the command decodes the CurrentTime ReadRequest and the matching ReadResponse. The response has a Good service result and a DateTime DataValue whose omitted StatusCode means Good. Without the key file, the same packets stay encrypted. Neither decode reports malformed OPC UA packets.

At this baseline, the decryption procedure on this page applies to a client-side writer with Basic256Sha256 and SignAndEncrypt, decoded with TShark 4.4.19. Server-side writers, renewals, Reverse Connect, and the Wireshark GUI are outside its scope. Milo's support for the enhanced ECC and RSA-DH policies does not mean your Wireshark release can decrypt them. Record the capture-tool version, policy, and decoded result with your evidence.

If the capture does not decrypt

If requests decode but responses appear as ServiceId 0 or malformed packets, check the server-port preference first. Then check these causes.

• whether the capture is complete, since packet loss and truncated snap lengths can prevent reassembly
• whether the key file contains the channel's current token
• whether a reconnect or renewal happened after the writer closed

Before you report a Milo service failure, make sure the problem is not a decoder or dissector limitation. The operation status you checked in step 2 tells you what Milo returned, independently of what the capture shows.

Handle capture resources

Restrict the capture, key log, and decoded output to the people performing the diagnosis. The verbose output can include Session authentication tokens, user identities, and application data. WiresharkKeyLogWriter sets no access-control or retention policy, so create the owner-only location described above before writing.

Do not commit keys, captures, or decoded output to source control, copy them into support logs, or upload them as ordinary issue attachments. Remove them when the investigation's retention requirement is met.

The client and server configuration is immutable and has no API to remove the listener. To stop key export, close the writer, which silently drops future keysets, or rebuild the client or server without the listener. Neither action revokes copies already made.

The source KeyLogExample prints key-log contents to the console as a demonstration. To capture without printing keys, use the scoped helper above.

Examples and reference

The key-writer Javadoc and KeyLogExample linked below document the TShark preferences and port setting used above. The repository guide covers callback placement, key-log format, threading, and writer tests for contributors.

• Sample source: FeatureSnippets

• Key writer contract

• Security listener contract

• Diagnostics authorization modes

• Source key-log guide and contributor details

• Demonstration KeyLogExample

Next steps

You can now collect logs, read the server's diagnostics, and decrypt a capture of a secured exchange. To compare a failing application with a known-good program, continue with Examples, which runs the source client examples against a shared example server and lists their expected results.

Related: Troubleshooting · Deployment · Client security · Server access control · Reference