Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memorator Log Uploading Script #86

Merged
merged 14 commits into from
May 23, 2024

Conversation

AarjavJain101
Copy link
Contributor

@AarjavJain101 AarjavJain101 commented May 21, 2024

Purpose:

  • With the new Kvaser Memorator Pro 2xHs we want to use it to log our CAN messages.
  • After obtaining these logs, to put them to use we must upload the logfiles to the parser which will than parse the data, send it to InfluxDB and also Grafana.
  • The script in the tools/ folder aims to accomplish this by using the ./link_telemetry.py -u command.

Changes:

  • Imported main script function in link_telemetry.py.
  • Changed the previous upload_logs function to simply call the script function. Note that the script function takes the sendToParser function as an argument because otherwise I would have to copy the sendToParser and process_response functions over to tools/MemoratorUploader.py.
  • The MemoratorUploader.py script was created with features like displaying sizes of log files as well as going through log files individually.
  • Updated README.md by removing the Setting up PCAN Drivers section and editing the .env file construction to only set an INFLUX_INIT_BUCKET and an INFLUX_DEBUG_BUCKET for successful startup.
  • delete_all is now implemented and automatically done (there is no option if you choose to upload logs).
  • Does not have choice for what logfile will be uploaded (uploads all of them)
  • Option for showing pretty tables (do --table-on by default it is off).
  • Logs to the _log bucket suffixes on Influx.

Notes:

  • The 15 log files are 1GB each. In our testing so far, the logs we created have not gone over 1GB. As such we suspect that when we read the logfiles inside one of the .KMFs they will be the exact same logfiles as any of other 15 .KMFs.
  • This means when doing log upload you will effectively upload the same data 15 times in a row.

What Needs to be Done Before Merge:

  • You cannot run ./link_telemetry.py on this branch without having canlib correctly installed. You will get this error:
    image
  • I need to fix/understand why id=0 messages are appearing in the logs.

Copy link
Contributor

@ishanjoshi23 ishanjoshi23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@ishanjoshi23 ishanjoshi23 merged commit 9af803b into main May 23, 2024
1 check passed
@ishanjoshi23 ishanjoshi23 deleted the user/Aarjavjain101/upload_memorator_logs branch June 5, 2024 06:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants