Dev companion v2#42
Merged
rightup merged 55 commits intopyMC-dev:feat/companionfrom Mar 6, 2026
Merged
Conversation
- Added conditional import for CompanionRadio to manage availability. - Updated Dispatcher class to support raw packet subscribers for enhanced logging and data handling. - Improved AdvertHandler to utilize a new payload parsing method and publish events for node discovery. - Enhanced ControlHandler with optional debug logging for better traceability. - Refactored GroupTextHandler to improve message handling and event publishing. - Introduced binary response callback in ProtocolResponseHandler for better response management. - Added X25519 scalar clamping utility in CryptoUtils for compatibility with firmware. - Updated LocalIdentity to ensure ECDH matches firmware specifications. - Improved packet building methods to align with routing specifications.
- Introduced REQ_TYPE_GET_STATUS for requesting repeater stats. - Updated send_status_request method to handle status requests with improved error handling and response parsing. - Enhanced message type determination in send methods to differentiate between flood and direct routes. - Refactored contact_store to handle out_path_len more robustly. - Added CayenneLPP decoding functionality in protocol response handling for better sensor data interpretation.
…tures - Introduced methods for sending trace packets with explicit paths and handling path discovery responses. - Added support for sending binary requests with improved error handling and response management. - Updated existing path discovery methods to utilize new tagging and telemetry request mechanisms. - Enhanced logging for error handling in packet sending operations.
- Improved path discovery methods with enhanced tagging and telemetry requests. - Updated binary request handling to include better error management and response processing. - Enhanced logging for packet sending operations to improve traceability and debugging.
…thHandler - Updated logging in CompanionBase to skip debug messages for small payloads, improving log clarity. - Added a new method in MessageQueue to pop the most recent message, enhancing message retrieval capabilities. - Simplified PATH packet analysis logging in PathHandler, consolidating log statements for better readability and efficiency. - Removed redundant logging in ProtocolResponseHandler to streamline response processing.
- Implemented a mechanism to deduplicate incoming direct and channel messages by packet hash, preventing the same packet from being queued multiple times during reconnects. - Increased the default offline queue size to accommodate more messages. - Updated the TextMessageHandler to include packet hash in message data for deduplication purposes. - Added unit tests to verify the deduplication functionality in the CompanionBridge.
- Added a new Companion Guide to the documentation for better user guidance. - Refactored several companion modules to improve type hinting and code clarity, including the addition of return types for methods. - Enhanced the CompanionBase class with new methods for managing contacts and advertisements, improving overall functionality. - Updated constants and models to support new features and ensure consistency across the codebase.
- Added methods to set flood regions and apply transport keys for scoped flooding in the CompanionBase class. - Enhanced packet handling to include flood scope transport codes, ensuring flood packets are tagged and routed correctly. - Updated CompanionRadio and Dispatcher classes to propagate flood scope settings and manage flood transport keys. - Documented flood scope usage in the companion guide for better user understanding.
… contribution guide - Simplified function calls in discover_nodes.py for better clarity. - Reorganized import statements in test_modem_crypto.py and companion modules to follow PEP 8 guidelines. - Enhanced code formatting in various files for improved readability, including consistent spacing and line breaks. - Updated comments and docstrings for clarity and conciseness across multiple modules.
- Introduced a shared factory for core protocol handlers to streamline handler registration in the Dispatcher. - Refactored CompanionBase, CompanionBridge, and CompanionRadio to utilize new handler access methods, enhancing code clarity and maintainability. - Added new methods for sending text, channel messages, and raw data in CompanionBase, improving messaging capabilities. - Updated documentation and comments for better understanding of the messaging flow and handler interactions.
Move the full companion frame protocol implementation from pyMC_repeater into pyMC_core as a reusable base class. The base CompanionFrameServer handles TCP framing, command dispatch, push callbacks, and contact/message/channel management. Persistence is handled through overridable hook methods (_persist_companion_message, _sync_next_from_persistence, _save_contacts, _save_channels) so the base class works standalone with in-memory stores while subclasses can add SQLite or other backends. Also adds ~90 frame protocol constants (CMD_*, RESP_CODE_*, PUSH_CODE_*, ERR_CODE_*) and frame delimiters to companion/constants.py, and exports CompanionFrameServer from companion/__init__.py.
…ooks Add 11 new command handler methods and 13 dispatch branches to CompanionFrameServer for binary frame types needed by CompanionBridge and CompanionRadio: CMD_GET_DEVICE_TIME, CMD_SET_DEVICE_TIME, CMD_SET_RADIO_PARAMS, CMD_SET_RADIO_TX_POWER, CMD_SHARE_CONTACT, CMD_EXPORT_CONTACT, CMD_SET_TUNING_PARAMS, CMD_LOGOUT, CMD_GET_CUSTOM_VARS, CMD_SET_CUSTOM_VAR, CMD_SET_AUTOADD_CONFIG, CMD_GET_AUTOADD_CONFIG. Add get_time(), set_time(), and send_logout() to CompanionBase. Add no-op _save_prefs()/_load_prefs() hooks to CompanionBase following the existing persistence hook pattern (_save_contacts, _save_channels, etc.). _save_prefs() is called automatically after preference-mutating methods (set_radio_params, set_tx_power, set_tuning_params, set_autoadd_config, set_other_params, set_advert_name, set_advert_latlon). _load_prefs() is called once at the end of _init_companion_stores(). Subclasses override to persist to SQLite, JSON, etc.
Update the companion module documentation to clarify the roles of three main classes: `CompanionRadio`, `CompanionBridge`, and `CompanionFrameServer`. Introduce new methods for sending text and channel messages, and improve the persistence hooks for preferences. Document the TCP frame protocol used by `CompanionFrameServer`, including frame format and quick start examples. Enhance the description of preference management and device configuration methods, ensuring clarity on their functionality and usage.
- Updated `_get_channel_by_hash` to return all matching channels due to potential hash collisions, improving robustness. - Introduced `_get_channels_by_hash` to handle multiple channel matches and updated documentation for clarity. - Modified `_decrypt_channel_message` to return `None` on HMAC mismatch, aligning with expected behavior during candidate iteration. - Adjusted message parsing logic to iterate through candidate channels until a valid decryption is found, logging potential hash collisions for better diagnostics.
- Added documentation comments in `_cmd_send_telemetry_req` to specify the expected byte structure for the CMD_SEND_TELEMETRY_REQ protocol, detailing reserved bytes and public key format. - Updated comments in `GroupTextHandler` to improve clarity on hash collision handling and logging for unknown channels.
… handling of path, req, and non_req packets. - Introduced support for the OWNER_INFO request type in the binary parsing logic, allowing the parsing of owner information from binary responses. - Implemented the `_parse_owner_info` function to extract and structure owner data from the response payload. - Enhanced the `send_anon_req` method in `CompanionBase` to facilitate sending anonymous requests for owner information. - Updated the `CompanionFrameServer` to handle the new CMD_SEND_ANON_REQ command, ensuring proper processing of anonymous requests. - Added constants for the OWNER_INFO request type and updated relevant documentation for clarity on the new functionality. - Fix hang due to race condition in kiss_modem_wrapper.py
…core into dev-companion-v2
- Introduced `firmware_ver_level` to the login response parsing in `LoginResponseHandler`, allowing the extraction of firmware version information from the response payload. - Updated `CompanionBase` to include `firmware_ver_level` in the login success response. - Enhanced `CompanionFrameServer` to utilize the new firmware version level during login, providing a fallback mechanism for compatibility. - Adjusted comments and documentation to reflect changes in the response format and handling logic.
- Added a check in `_handle_new_channel_message` to ignore messages marked as outgoing, ensuring that only incoming messages are processed and sent to the client. This change improves message handling and prevents unnecessary duplication in the client interface.
… reciprocal PATH - Send reciprocal PATH after flooded PATH login (mirrors C++ Mesh.cpp:168-169) so remote repeaters learn the return route and respond via DIRECT - Await reciprocal PATH TX before signaling login success to prevent race - Add propagation delay for multi-hop stats/telemetry requests - Fix binary request tag mismatch: use timestamp from create_protocol_request as tag (matching C++ companion pattern) instead of random value the firmware never echoes back — fixes neighbors, ACL, and all CMD_SEND_BINARY_REQ ops - Add sanity checks to stats parser to prevent misidentifying binary responses
_save_contacts() was only called after explicit add/remove commands, so advert-driven name changes and PATH route updates were lost on restart.
… store management - Added functionality to automatically add contacts based on their type, mirroring C++ logic for selective contact addition. - Introduced methods for handling contact deletion and full contact store scenarios, including callbacks for notifying the application. - Enhanced the contact store to support adding or overwriting contacts when the store is full, ensuring non-favourite contacts are replaced as needed. - Updated the CompanionBridge to utilize the new auto-add logic during advert processing, improving contact management efficiency.
…ement - Updated the CompanionBridge to preserve additional fields when updating existing contacts, ensuring important data is retained during advert processing. - Refactored CompanionFrameServer to implement asynchronous contact persistence methods, improving efficiency and reliability in contact management. - Introduced a heartbeat mechanism to maintain TCP connections, enhancing stability during client-server interactions. - Improved error handling for connection issues and added TCP keepalive settings for better network resilience.
…ge synchronization - Updated `_cmd_sync_next_message` to use `asyncio.to_thread` for better performance when retrieving the next message from persistence. - Enhanced stats retrieval logic to support both coroutine and non-coroutine functions, ensuring compatibility and efficiency in fetching statistics. - Improved error handling by ensuring that stats are fetched correctly from the appropriate source, maintaining robustness in the CompanionFrameServer's operations.
…dling - Introduced eviction logic for managing client connections, allowing a new connection to replace an existing one. - Added `client_idle_timeout_sec` parameter to disconnect clients after a period of inactivity, improving resource management. - Updated documentation to reflect changes in connection handling and timeout behavior.
…h command frame - Increased the `client_idle_timeout_sec` from 90 to 120 seconds to allow for longer periods of inactivity before disconnecting clients. - Updated documentation to reflect the new default timeout value, enhancing clarity for operators managing client connections.
- Extracted SNR and RSSI values from network information in the CompanionBase class, allowing for better monitoring of network conditions. - Updated QueuedMessage model to include SNR and RSSI attributes, enhancing message detail. - Modified CompanionFrameServer to handle SNR and RSSI in message processing, improving data accuracy in communication.
…dling - TODO pyMC-dev#7 - Refactored `_build_advert_push_frames` to accept a `Contact` object instead of a dictionary, improving type safety and clarity. - Introduced `_contact_from_dict` to construct `Contact` instances from dictionaries, enhancing code readability and maintainability. - Updated the handling of optional fields and data extraction within the frame building process, ensuring consistency and reducing redundancy.
…t natively -- TODO pyMC-dev#4 - Added `_apply_advert_to_stores` method to streamline the application of adverts to the contact store and path cache, improving contact handling efficiency. - Updated `CompanionBridge` to utilize the new method for processing adverts during node discovery, ensuring contacts are added or updated correctly. - Refactored `Contact` model to include a `from_dict` class method for constructing contact instances from various data sources, enhancing flexibility and maintainability. - Improved error handling in advert processing to ensure robustness during contact updates and path cache management.
…ODO pyMC-dev#3 - Added `wait_for_ack` parameter to `send_text_message`, allowing for non-blocking message sending. - Updated documentation to clarify the behavior of `wait_for_ack` and its impact on message acknowledgment. - Refactored advert handling in `CompanionBridge` to utilize `inbound_path`, improving the processing of node discovery events. - Enhanced test coverage for node discovery and advert received callbacks, ensuring correct handling of single-path events.
…med - TODO #1 and #2 - Introduced `_pending_ack_crcs` to track pending ACK CRCs in `CompanionBase`, enhancing the handling of confirmed sends. - Refactored `_track_pending_ack` and `_apply_ack` methods to utilize the new `_try_confirm_send` method for improved clarity and functionality. - Updated `CompanionRadio` to clear pending ACKs upon key import and added an ACK received listener to the dispatcher for better integration. - Enhanced the dispatcher to support asynchronous ACK handling, ensuring efficient processing of received ACKs.
- Updated the CompanionFrameServer to accept companion hash values in both decimal and hexadecimal formats, enhancing flexibility in hash handling. - Modified logging to reflect the parsed integer value of the companion hash, ensuring accurate representation in server status messages.
- Enhanced the CompanionBase class to extract SNR and RSSI values from network information, improving network condition monitoring. - Updated the QueuedMessage model to include SNR and RSSI attributes for more detailed message data. - Modified the CompanionFrameServer to process and persist SNR and RSSI in received messages, ensuring accurate communication data.
…ng, add SNR response for channel v3 frame - Added detailed disconnect reasons for client disconnections in the CompanionFrameServer, enhancing debugging and monitoring capabilities. - Updated exception handling to capture specific error types and log them appropriately, improving error traceability. - Ensured that the client disconnection log message includes the reason for disconnection, providing clearer insights into client behavior.
- Adjusted MAX_FRAME_SIZE to 172 bytes to align with firmware constraints and BLE MTU. - Introduced MAX_PAYLOAD_SIZE to ensure payloads are correctly truncated to prevent overflow. - Enhanced packet injection logic to avoid duplicate messages when multiple bridges are present, ensuring consistent deduplication across local and over-the-air deliveries. - Updated documentation to reflect changes in frame size and deduplication strategy.
…eHandler ProtocolResponseHandler now supports an optional callback when it updates a contact's out_path from a decrypted PATH packet. CompanionRadio and CompanionBridge set this callback to fire contact_path_updated so the companion layer can persist or react. Bridge no longer updates path or fires the callback in process_path_ack_variants; LoginResponseHandler skips forwarding PATH packets to the protocol handler to avoid double invocation.
- Add channel_updated and channel_removed callbacks. - Add raw_data_received callback with SNR and RSSI for parity with RX_RAW - Add RX_LOG_DATA for parity with firmware LOG_RX_DATA with data, SNR, and RSSI.
…ework - Added CMD_SEND_RAW_DATA command to send raw payloads directly. - Introduced PUSH_CODE_RAW_DATA for receiving raw custom payloads with SNR and RSSI. - Implemented callbacks for raw data reception in CompanionBridge and CompanionRadio. - Enhanced PacketBuilder to create raw custom packets. - Updated tests to cover new raw data functionalities and ensure proper handling.
…framework - Introduced CMD_EXPORT_PRIVATE_KEY and CMD_IMPORT_PRIVATE_KEY commands for handling private key operations. - Implemented _cmd_export_private_key to export the private/signing key in 64-byte MeshCore format. - Added a stub for _cmd_import_private_key, which currently does not perform any operations but may support dynamic imports in the future. - Updated CryptoUtils with a method to expand a 32-byte Ed25519 seed to the 64-byte MeshCore format. - Enhanced tests to verify the new private key functionalities and ensure compatibility with MeshCore key generation.
…rivate key - Introduced ed25519_public_from_meshcore_64 method in CryptoUtils to derive an Ed25519 public key from a 64-byte MeshCore private key. - Added validation to ensure the input private key is 64 bytes in length. - Integrated new functionality to enhance key management capabilities within the crypto module.
…ssure - Updated `push_trace_data` and `push_rx_raw` methods to be asynchronous, allowing for proper backpressure handling. - Introduced `push_rx_raw_async` for async usage, ensuring that raw RX packets can be pushed with backpressure. - Modified internal `_write_push` method to await the drain process, preventing concurrent drain tasks and improving reliability. - Enhanced documentation and tests to reflect the new async behavior and ensure correct functionality in various contexts.
- Skip _on_contact_path_updated for contacts not in the store, matching companion firmware behavior. Also persist the updated path to the in-memory ContactStore before firing callbacks. - Add defense-in-depth check in frame_server's on_contact_path_updated push callback to avoid pushing or persisting phantom contacts. - Make _drain_writer return bool; break the client loop immediately on drain failure instead of waiting for the next read to discover a dead connection. - Drain every 10 frames in _cmd_get_contacts to avoid unbounded write-buffer growth for large contact lists. - Extract _build_message_frame helper shared by base and repeater, eliminating ~60 lines of duplicated message encoding and fixing the repeater's hard-coded SNR=0 for V3 frames. - Remove unnecessary hasattr checks on Contact dataclass fields, redundant fallback loop in _cmd_send_txt_msg, and fix get_full_list evaluation order in _cmd_get_channel. - Simplify on_advert_received dict/object handling; warn on unexpected non-Contact input instead of silently converting.
…proved backpressure handling - Replaced asynchronous push methods with synchronous versions that enqueue frames into a write queue, enhancing backpressure management. - Introduced a dedicated writer task to process the queue, ensuring reliable frame transmission. - Updated tests to validate the new frame queuing behavior and confirm correct frame formatting for trace and RX raw data pushes.
…tion - Added a method to synchronize the current node name to group text handlers in CompanionBridge and CompanionRadio. - Implemented the `_sync_our_node_name_to_handlers` method in both classes to update the node name for echo detection. - Enhanced the GroupTextHandler to include a method for updating the stored node name. - Updated tests to verify the correct functionality of the new synchronization feature.
Matches SX1262Wrapper convention so engine.py getattr() reads actual configured values instead of falling back to defaults. Adds set_tx_power() and optional kwargs to configure_radio().
This reverts commit df073cb.
Matches SX1262Wrapper convention so engine.py getattr() reads actual configured values instead of falling back to defaults. Adds set_tx_power() and optional kwargs to configure_radio().
Override set_radio_params/set_tx_power as no-ops in CompanionBridge and re-apply host radio_config after _load_prefs() to prevent persisted values from overwriting the host's actual settings. Made-with: Cursor
…core into dev-companion-v2
- Changed frequency validation to accept values in kHz instead of Hz, aligning with firmware specifications. - Updated the frequency parameter in set_radio_params to convert kHz back to Hz for internal processing. - Adjusted the power validation to allow a maximum of 30 instead of 20.
- Introduced support for multi-byte path lengths in CompanionBase, allowing for 1-byte, 2-byte, and 3-byte hash sizes. - Added methods to set and apply path hash mode, ensuring proper encoding in originated packets. - Updated constants and packet handling to accommodate new path length encoding, enhancing compatibility with firmware. - Enhanced tests to validate the new path hash functionality and ensure correct behavior across various scenarios.
feat(companion): implement multi-byte path hash encoding and management
- Added logging for optional LPP parse failures in binary parsing functions to improve debugging. - Introduced a method to retrieve the group text handler for node name synchronization in CompanionBase, CompanionBridge, and CompanionRadio classes. - Updated path length validation in Packet class to provide clearer error messages for invalid encodings. - Enhanced tests to ensure proper handling of path length constraints and error scenarios.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Companion module
packet_injectorandprocess_received_packet()so a companion identity can share the repeater’s dispatcher._send_packetabstraction; subclasses implement transport and_get_group_text_handler()for name sync.Multi-byte paths
PathUtilsprovides encode/decode and validation;Packet.read_from()rejects invalid or oversized path_len with clear errors. Tests intest_packet_path_and_hashandtest_packetcover round-trips and bad deserialization for 1/2/3-byte path encodings.Other