Added osi file format converter and read converter #358
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reference to a related issue in the repository
There is an issue on how to define files which can be read by the visualizer.
Add a description
This PR defines the OSI format for storing multiple messages in one file based on the length of the message. It also provides a script to convert from currently used separated trace files by
$$__$$to to the OSI format. This is made so we avoid confusion on how to correctly define OSI file formats and to use Google official suggestion on how storage of multiple messages should be handled. It is optional to convert into lzma compressed files since this compression algorithm does a good job on compressing ~1.3gb *.txt into 2,4mb *.txt.lzma (540x smaller, compression time 2min, decompression time 0.5s).I also provide a osi2read.py script which converts serialized trace files to human readable files for plausibliity checks of the trace files. It should be avoided for now to use it on to huge trace files since the readable files could not be opened anyway :).
For tests the osi-validator can be used that can now validate both type of formats but warns the user that the old format will be deprecated in the near future.
Mention a member
@jdsika @pmai let me know your thoughts, suggestions and improvements ;)
Check the checklist