[monitorlib] Make UTMClientSession a singleton

[the-glu]

Proposition 2 to fix connections leaking. See #1408 for alternative version (or we could have both, but that probably not needed).

The test suite is currently creating a set of UTMClientSession during initialization, for each test and endpoint used.

This create issues, as connections use keep-alive mechanism and are never closed. #1407 is an example of that with a lot USS, reaching the limit after a while.

This version of the fix make UTMClientSession a singleton, based on it parameters, meaning it will be reused across tests.
This may have side effects, but I'm not sure how it's could be used across the whole codebase (eg. if the session / AuthAdapter are dynamically modified).

Tested manually by checking the number of open sockets that stay to a reasonable value. Connections are still reused.

[BenjaminPelletier]

I do think we want something functionally like this (there is no reason to have two separate instances with identical construction parameters), but

The test suite is currently creating a set of UTMClientSession during initialization, for each test and endpoint used.

I don't think that's correct if "each test" is referring to test scenarios. Nearly all test scenarios obtain their UTMClientSession from a resource's single client, and a resource is created only once for a test run and then reused across all applicable scenarios. So, I think this change should have minimal practical impact if everything was designed appropriately.

The only slightly-questionable design I see is uss_qualifier.scenarios.astm.utm.make_uss_report where a separate UTMClientSession is created at test scenario run time to avoid the need to create an "F3548 USS" resource type and declaration just for this one endpoint when that information can be extracted from previous test operations. But, this test scenario is only run once near the very end of the only test suite in which it's used, so I don't see how it could contribute appreciably to the problem.

Therefore, I'm surprised that this PR has any appreciable effect. Could you elaborate on your observations re: "Tested manually by checking the number of open sockets that stay to a reasonable value."? If we added a log statement to the constructor of UTMClientSession (post singleton-pattern check in this PR), I would expect to see roughly the same number of log statements after this PR as before. Or, via alternate observation, I would expect this hypothetical log statement to be triggered very rarely (only when the base URL for two different APIs happened to be the same):

        if hasattr(
            self, "_prefix_url"
        ):  # If set, we have been reused from the singleton pattern, no need to do anything
            logger.info("Reusing existing UTMClientSession with identical parameters")
            return

Do you expect to see something different than those two observations? If not, how would this PR be expected to improve #1407?

Separately, I would rather implement this functionality via a factory pattern to satisfy the principle of least surprise. The implementation in this PR certainly has the fewest lines changed, but it seems like it sets a behavior-differs-from-expectations landmine for the future. Instead, it seems like we can achieve the same result without the landmine by creating a UTMClientSessionFactory + singleton and then replacing all existing instantiations of UTMClientSession with a call to the UTMClientSessionFactory singleton's create_instance method.

[the-glu]

Therefore, I'm surprised that this PR has any appreciable effect. Could you elaborate on your observations re: "Tested manually by checking the number of open sockets that stay to a reasonable value."? If we added a log statement to the constructor of UTMClientSession (post singleton-pattern check in this PR), I would expect to see roughly the same number of log statements after this PR as before. Or, via alternate observation, I would expect this hypothetical log statement to be triggered very rarely (only when the base URL for two different APIs happened to be the same):

With some debug statements, running configurations.dev.uspace:

2026-04-07 06:52:51.485 | INFO     | __main__:execute_test_run:127 - Instantiating resources
New UTMClientSession http://scdsc.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://scdsc.uss1.localutm/flight_planning/v1 <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://scdsc.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://scdsc.uss2.localutm/flight_planning/v1 <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://scdsc.log.uss6.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://scdsc.log.uss6.localutm/flight_planning/v1 <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://v22a.riddp.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://v22a.riddp.uss1.localutm/flight_planning/v1 <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://v22a.ridsp.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://v22a.ridsp.uss1.localutm/flight_planning/v1 <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://scdsc.uss1.localutm/versioning <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://scdsc.uss2.localutm/versioning <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://scdsc.uss1.localutm/versioning <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://scdsc.uss2.localutm/versioning <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://scdsc.uss1.localutm/flight_planning/v1 <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://scdsc.uss2.localutm/scdsc <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://v22a.ridsp.uss1.localutm/ridsp/injection <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://v22a.riddp.uss1.localutm/riddp/observation <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://dss.uss1.localutm/rid/v2 <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://dss.uss2.localutm/rid/v2 <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
2026-04-07 06:52:51.555 | INFO     | __main__:execute_test_run:140 - Instantiating top-level test suite action
2026-04-07 06:52:52.875 | DEBUG    | monitoring.uss_qualifier.suites.suite:__init__:269 - Overriding local resource system_identity in test suite suites.astm.utm.f3548_21 with externally-provided resource
New UTMClientSession http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://dss.uss1.localutm None None
New UTMClientSession http://dss.uss1.localutm <monitoring.monitorlib.auth.InvalidTokenSignatureAuth object at 0x7f8d40f4aba0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d41d22fd0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d41d22fd0> None
2026-04-07 06:52:53.257 | WARNING  | monitoring.uss_qualifier.suites.suite:__init__:295 - Skipping action 17 (ActionType.TestScenario scenarios.astm.utm.dss.DatastoreAccess) because Test suite action to run ActionType.TestScenario scenarios.astm.utm.dss.DatastoreAccess could not find required resource ID "dss_datastore_cluster" used to populate child resource ID "datastore_cluster"
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://dss.uss2.localutm None None
New UTMClientSession http://dss.uss2.localutm <monitoring.monitorlib.auth.InvalidTokenSignatureAuth object at 0x7f8d40e107d0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d41d22fd0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d41d22fd0> None
2026-04-07 06:52:53.312 | WARNING  | monitoring.uss_qualifier.suites.suite:__init__:295 - Skipping action 17 (ActionType.TestScenario scenarios.astm.utm.dss.DatastoreAccess) because Test suite action to run ActionType.TestScenario scenarios.astm.utm.dss.DatastoreAccess could not find required resource ID "dss_datastore_cluster" used to populate child resource ID "datastore_cluster"
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f8d419467b0> None
New UTMClientSession http://dss.uss1.localutm/rid/v2 <monitoring.monitorlib.auth.InvalidTokenSignatureAuth object at 0x7f8d40cfcb90> None
New UTMClientSession http://dss.uss1.localutm/rid/v2 None None
2026-04-07 06:52:54.524 | WARNING  | monitoring.uss_qualifier.suites.suite:__init__:295 - Skipping action 9 (ActionType.TestScenario scenarios.astm.netrid.v22a.dss.DatastoreAccess) because Test suite action to run ActionType.TestScenario scenarios.astm.netrid.v22a.dss.DatastoreAccess could not find required resource ID "dss_datastore_cluster" used to populate child resource ID "datastore_cluster"
New UTMClientSession http://dss.uss2.localutm/rid/v2 <monitoring.monitorlib.auth.InvalidTokenSignatureAuth object at 0x7f8d40d22780> None
New UTMClientSession http://dss.uss2.localutm/rid/v2 None None
2026-04-07 06:52:54.536 | WARNING  | monitoring.uss_qualifier.suites.suite:__init__:295 - Skipping action 9 (ActionType.TestScenario scenarios.astm.netrid.v22a.dss.DatastoreAccess) because Test suite action to run ActionType.TestScenario scenarios.astm.netrid.v22a.dss.DatastoreAccess could not find required resource ID "dss_datastore_cluster" used to populate child resource ID "datastore_cluster"
2026-04-07 06:52:54.592 | INFO     | __main__:execute_test_run:149 - Running top-level test suite action

As you can see there are indeed a lot of duplicate UTMClientSession beeing created, so something is not working as expected somewhere.

Adding logging on scenario creation, we can see that the ones generated by FlightPlannerCombinations are not reusing resources, so with a high number of combinations like reported in #1407 this is going to generate a lot of session.

make_test_scenario <class 'monitoring.uss_qualifier.scenarios.astm.utm.nominal_planning.conflict_higher_priority.conflict_higher_priority.ConflictHigherPriority'>
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f60629627b0> None
make_test_scenario <class 'monitoring.uss_qualifier.scenarios.astm.utm.nominal_planning.conflict_higher_priority.conflict_higher_priority.ConflictHigherPriority'>
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f60629627b0> None
make_test_scenario <class 'monitoring.uss_qualifier.scenarios.astm.utm.nominal_planning.conflict_higher_priority.conflict_higher_priority.ConflictHigherPriority'>
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f60629627b0> None
make_test_scenario <class 'monitoring.uss_qualifier.scenarios.astm.utm.nominal_planning.conflict_higher_priority.conflict_higher_priority.ConflictHigherPriority'>
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f60629627b0> None

Even on a single scenario, resources are not reused (uss1 instanced twice):

make_test_scenario <class 'monitoring.uss_qualifier.scenarios.astm.utm.dss.synchronization.uss_availability_synchronization.USSAvailabilitySynchronization'>
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f60629627b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss1.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f60629627b0> None
Reusing existing UTMClientSession with identical parameters http://dss.uss2.localutm <monitoring.monitorlib.auth.DummyOAuth object at 0x7f60629627b0> None

Is there an issue somewhere else? Because if you expect resources to be reused in general, they may be something wrong elsewhere and we shouldn't need to make it a singleton.

[the-glu]

We did some debugging with @mickmis and the root cause seems to be that:

Test scenarios use get_instance on __init__ on the (probably shared) resource:

monitoring/monitoring/uss_qualifier/scenarios/astm/utm/off_nominal_planning/down_uss.py

Line 77 in bfdba7c

self.dss = dss.get_instance(self._dss_req_scopes)

get_instance create a new DSSInstance every time: https://github.com/interuss/monitoring/blob/main/monitoring/uss_qualifier/resources/astm/f3548/v21/dss.py#L771
And DSSInstance create a new UTMClientSession every time: https://github.com/interuss/monitoring/blob/main/monitoring/uss_qualifier/resources/astm/f3548/v21/dss.py#L94

Should DSSInstance should be a singleton in DSSInstanceResource.get_instance instead?

[BenjaminPelletier]

And DSSInstance create a new UTMClientSession every time: https://github.com/interuss/monitoring/blob/main/monitoring/uss_qualifier/resources/astm/f3548/v21/dss.py#L94

Ah, there we go -- DSSInstance and DSSInstanceResource are treating UTMClientSessions as cheap. That's a great discovery and something I think we can fix closer to the root.

Should DSSInstance should be a singleton in DSSInstanceResource.get_instance instead?

Each DSSInstance has different restrictions on the kinds of operations that can be performed, so making them singletons would be somewhat challenging I think. I believe we may be able to fix the proliferation of UTMClientSessions by DSSInstances by changing the DSSInstance constructor to accept a UTMClientSession rather than AuthAdapter, and populating that parameter with a single UTMClientSession created once by the resource spawning DSSInstances.

[BenjaminPelletier]

I think the investigation associated with this PR was primarily responsible for finding the likely root causes of #1407, but the mitigation concept in this PR was applied in a different form in #1413; should we close this PR?

[the-glu]

Yes :)