3.3.1: Bugfixes and MessageLogger/Replay Enhancements

ezmsg.core 3.3.1

This pull request primarily addresses a bug introduced in v3.3.0 where multiple components can be run using ez.run syntax. This new feature came with an unforeseen consequence where these components could possibly (and with startling frequency tended to) have duplicate names, resulting in multiple subscribers/publishers sharing the same topic names. This has been addressed with a change to the ez.run call signature, requiring unique names for every component to be run by ezmsg. Additionally, when run this way in v3.3.0, every component was run in its own process which could be circumvented by using force_single_process but ultimately did not give developers the flexibility they needed when running multiple components. To address this, we have added a process_components keyword argument to the ez.run call signature which specifies which components are to reside in dedicated processes. By default, all components passed to ez.run will now run in the same process (unless sub-components specify their own process_components).

Example v3.3.1 call signature for ez.run

ez.run(
    # All components are specified as name: component pairs.
    # This construction requires all components to have unique names at this level
    # All caps are used for these names in this example just for convention.
    REPLAY = replay, 
    PREPROC = preproc, 
    SEQ_COLL = sequence_collector, 
    FILT_COLL = filt_collector,
    TERM = term,

    # Use the root_name keyword argument to have all of these units reside under a designated root collection
    root_name = "PROJECT",

    # Connections between these components and streams can be specified just as before
    connections = [
        (replay.OUTPUT_MESSAGE, preproc.INPUT_SIGNAL),
        (preproc.OUTPUT_SEQUENCE, sequence_collector.INPUT_MESSAGE),
        (preproc.OUTPUT_FILTERED, filt_collector.INPUT_MESSAGE),
        (sequence_collector.OUTPUT_MESSAGE, term.INPUT)
    ],

    # Here, you can specify which of the components passed into ez.run should have a dedicated process.
    process_components = [preproc]
)

If you used to call ez.run with a single component, you should consider adapting your code from ez.run(system) to ez.run(SYSTEM = system). Although your code will still function when passing a single component, you should receive a little warning from your typechecker, and this usage is now considered deprecated, and you will receive a DeprecationWarning

Additional features introduced:

• ezmsg.util.messagelogger.MessageLogger now logs timestamps for every message in the text file
• ezmsg.util.messagereplay.MessageReplay can be run dynamically by publishing Path objects; message rate can be controlled to replay files as fast as possible, at specified message rates, or "as recorded" using these new timestamps
  • NOTE: MessageLogger has a new message at the start of every log file that just indicates the start time of the log file. This is currently used by MessageReplay when replaying messages "as recorded". If you've been using ezmsg.util.messagecodec.MessageDecoder to directly decode json messages from these files, you may now notice a LogStart message in your file that you didn't put there. To address this, ezmsg.util.messagecodec.message_log has been introduced as a generator that opens the log file, decodes the messages using MessageDecoder and skips over this LogStart message.

Bugfixes

• Typing for process_components only requires a typing.Collection[Component] instead of a typing.Tuple[Component, ...]
• ezmsg.util.messages.axisarray.AxisArray:
  • isel allows integers, slices, and index arrays
  • Users can specify indexers keyword argument, or just use kwargs to specify indexers, just like xarray
• ezmsg.util.messagereplay.MessageLoder is gone. It was not a meaningful addition to the core.
• Fixed tests/test_perf.py message channels buffers testing.
• Addressed #28