transport improvements

[leshy]

• implements exception passing to RPC clients (if there is an exception on the server, it will be raised on the caller side)

• stress tests for all transport protocols (we expect 10k messages < sec, 100% reliability expected across all transports)

• stress tests for RPC (1k rpc calls < sec, 100% reliability expected) - SHM actually fails this - has 25% reliability on RPC tests due to how it handles message reception, this is an issue that means we can't use SHM across the board in dimos

• moves rpc call threadpool from LCMRPC to the rpc base so it's used cross-protocol

• shares a single cpp LCM instance/loop per process for all instantiated transports

• investigated kernel networking stats and LCM cpp implementation, optimized details around this to get LCM to perform much better

[leshy]

@paul-nechifor passing tests now

[paul-nechifor]

@paul-nechifor passing tests now

@leshy pytest dimos/robot/cli/test_dimos_robot_e2e.py fails for me. The e2e test only runs locally. It's not run by github because it runs into the same issue as the dimos/utils/cli/lcmspy/test_lcmspy.py tests which are marked with @pytest.mark.lcm.

This is the error I'm getting:

dimos/robot/cli/test_dimos_robot_e2e.py::test_dimos_robot_demo_e2e FAILED                                             [100%]

========================================================= FAILURES ==========================================================
_________________________________________________ test_dimos_robot_demo_e2e _________________________________________________

lcm_spy = <test_dimos_robot_e2e.LCMSpy object at 0x7e1d211b05f0>
dimos_robot_call = <test_dimos_robot_e2e.DimosRobotCall object at 0x7e1d211b0980>
human_input = <function human_input.<locals>.send_human_input at 0x7e1d211ef600>

    @pytest.mark.skipif(bool(os.getenv("CI")), reason="LCM spy doesn't work in CI.")
    def test_dimos_robot_demo_e2e(lcm_spy, dimos_robot_call, human_input):
        lcm_spy.wait_for_topic("/rpc/DemoCalculatorSkill/set_LlmAgent_register_skills/res")
>       lcm_spy.wait_for_topic("/rpc/HumanInput/start/res")

dimos/robot/cli/test_dimos_robot_e2e.py:148: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <test_dimos_robot_e2e.LCMSpy object at 0x7e1d211b05f0>, topic = '/rpc/HumanInput/start/res', timeout = 30.0

    def wait_for_topic(self, topic: str, timeout: float = 30.0) -> list[bytes]:
        start_time = time.time()
        while time.time() - start_time < timeout:
            if topic in self.messages:
                return self.messages[topic]
            time.sleep(0.1)
>       raise TimeoutError(f"Timeout waiting for topic {topic}")
E       TimeoutError: Timeout waiting for topic /rpc/HumanInput/start/res

dimos/robot/cli/test_dimos_robot_e2e.py:45: TimeoutError
================================================== short test summary info ==================================================
FAILED dimos/robot/cli/test_dimos_robot_e2e.py::test_dimos_robot_demo_e2e - TimeoutError: Timeout waiting for topic /rpc/HumanInput/start/res
==================================================== 1 failed in 50.11s =====================================================

The test passes on dev.

---

Also can I merge this into your PR: #744 (some autofixes)

[leshy]

oh got it, thought after merging dev the test got picked up by CI, tnx, I'll look into it