Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed IMU parse fails using chunking #78

Merged
merged 4 commits into from
May 3, 2024

Conversation

AarjavJain101
Copy link
Contributor

Previously we had IMU parse fails due to sunlink unable to keep up with messages coming in. We confirmed this by checking the raw UART stream coming to my comuter's port and noticed that all messages were indeed sent and sunlink could not get all the messages because it was too slow.

To fix this, I implemented a simple chunking method where we read 512 bytes from the stream at a time (variable and can be adjusted. See next steps). Once the chunk is received, it is extracted for the messages inside it which are then sent to the parser.

To verify this algorithm I ran it with a load of 1926 bytes per second (78 imu messages and 3 GPS messages ran in their respect tasks instead of the normal 13 and 1) and it got all the data except the last 13 or so messages which could have still been in the 512 byte chunk (cannot access this until the full amount is read). Quantitatively, 614/623 lines were received where all lines before line 616 matched the SD log.

Downside:

  • With this method you will have to parse messages in bursts, however, it guarantees all messages are parsed. The lag is about 0.5 seconds but will be reduced if the chunk is filled faster.

Specific Changes:

  • removed unused imports
  • added CHUNK_SIZE constant
  • added process_message function which takes in the 512 byte chunk and returns a list of all the valid messages
  • added the sendToParser function to abstract the functionally of creating a payload, sending to the parser, and expecting a callback.
  • added appropriate changes in main of link_telemetry to use these functions.

Next Steps:

  • Perhaps using a multiprocess approach (different from multithreading so that we can actually achieve parallelization to boost throughput in case of parse fails later on).
  • We can optimize the algorithm with a better choice of CHUNK_SIZE in the case that parse fails are received.

Copy link
Contributor

@ishanjoshi23 ishanjoshi23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Tested with IMU data from TEL board.

@ishanjoshi23 ishanjoshi23 merged commit 39f819b into main May 3, 2024
1 check passed
@AarjavJain101 AarjavJain101 deleted the user/AarjavJain101/fix_imu_parse_fails branch May 19, 2024 06:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants