Parse Fail Reduction By Message Splitting

[AarjavJain101]

Problem:

• When running the battery's CAN messages and the MCB's CAN messages we noticed that we got many parse fails that looked like the image below. This was essentially 2 CAN messages concatenated together. The reason for this is still not explicitly determined, however, tests show that a lack of processing capability in sending HTTP requests and receiving their callbacks is causing this problem. Please see the Monday Update for more details.

Solution:

• To combat this we decided to resolve the case of length 45 CAN messages which are two CAN messages concatenated together with a "\r" character separating the two. Essentially, in the parser, if we received a message whose length is >= 45 we will cut that message into 2 valid CAN messages and send both those back to link_telemetry.py in the callback (see changes for details).

Changes:

• .gitignore now contains the log folders
• API.md docs changed to reflect the necessary modifications to support main.py having to split the incoming chunk into valid messages. The parser then sends back a dictionary containing one field called all_responses which maps to a list of dictionaries of the parsed messages that could be obtained from the big chunk passed. Each dictionary could contain a PARSE_FAIL or OK or INFLUX_WRITE_FAIL status message.
• link_telemetry.py: Chunk size is now 24 * 21 which is 21 CAN messages. This equals 504 bytes total and is as close to 512 bytes as possible. I chose to make it close to 512 because we were previously using a Chunk Size of 512 when we fixed IMU parse fails so I did not see reason to diverge from what the system can already accomplish.
• link_telemetry.py: Modified process_response function to handle the new all_responses list.
• main.py: Removed unnecessary function try_extract_measurements.
• main.py: Added logic to split length >= 45 messages for no write and bucket endpoints.
• tools/MemoratorUploader.py: Fixed LOG file path to have the extension at the end (was missing before).

Next Steps:

• Now that the live and log filters are back we need to do another full test on Saturday to make sure the load of these filters is handled correctly still. If not may need to look into optimizing load.

[ishanjoshi23]

Tested.

[ishanjoshi23]

Only feature that needs to be tested is filters in the payload from link_telemetry to the parser. Will release after this is tested.