Skip to content

Flaky: MessageProducerSessionJmsTest times out after the simulated-outage test poisons the shared static service #1855

Description

@jcschaff

MessageProducerSessionJmsTest.concurrentRpcsEachGetTheirOwnAnswer still times out intermittently in CI-Test-group-Fast-other after #1852 / #1853.

This is a different cause, not a regression of #1852. That fix worked: its signature (Broker localhost not started so using MessageProducerSessionJmsTestBroker instead) does not appear once in the failing log, and the three JMS classes now serialise under @ResourceLock. The remaining failure is inside one class.

Evidence

From run 31153735938, with #1853 merged in:

06:26:46.9  ERROR (MessageProducerSessionJms.java:450) - simulated broker outage
06:26:49.0  ERROR (MessageProducerSessionJms.java:450) - Cannot publish to a deleted Destination:
                                                         temp-queue://ID:runnervm...-24:1:1
06:27:49.0  ERROR (MessageProducerSessionJms.java:450) - Server is temporarily not responding
                                                         (server testing, method echo)

A deleted temp reply queue, then exactly 60s of waiting, then the RPC timeout that fails the test.

Mechanism

Strongly indicated by the above, though not yet reproduced locally:

  • The class shares one private static CountingMessagingService service across every test method, created in @BeforeAll.
  • One test sets service.failConnections = true so createConnection() throws simulated broker outage. The flag itself is not leaked — it is reset in a finally (lines 173/179), so that is not the bug.
  • What survives the outage is the damage to the shared service: the connection is torn down, taking its temporary reply destinations with it. A later RPC then publishes to a temp queue that no longer exists, nobody consumes it, and the caller waits out its full timeout.

Intermittent because it depends on whether that teardown completes before the next test builds its reply consumer. Method-level parallelism is not involved — CI sets parallel.mode.default=same_thread, so methods within a class run sequentially; only classes run concurrently.

Suggested fixes

  1. Give the outage test its own service rather than mutating the shared static one — the flag is a property of the fixture, not of the test run.
  2. Failing that, rebuild the shared service after the outage test (@AfterEach that recreates it when poisoned), so no later test inherits dead destinations.
  3. Consider whether VCMessagingServiceJms should discard a connection whose creation failed instead of leaving its destinations referenced — if so, the test is surfacing a real production weakness, which is the same shape as the shared-session races fixed in Build the RPC request message on a session, not a second connection #1844/Give each RPC its own JMS session so concurrent callers cannot corrupt each other #1845.

Impact

Not caused by any product change — it fails on commits that touch no JMS code (it blocked #1829, which adds no server code at all), which is what makes it worth fixing rather than re-running.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions