Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Support multi-file logfiles #118
Yesterday we already reached 1 hour recording (1.3GB) for SubT ROS simulation in Virtual Track:
So it is time to support indexed files, right? The default split criteria should be time=>3600s, but it can be also shorter time or size. I would use ADTF scheme for DAT files (and probably many others) with
On the other hand, if you open the first file with default parameters it should go through all files and with some extra parameter it would read a single file only.
Do we want to mark in the first file, that the recording is not complete?
It is surely necessary to copy all named streams from the zero stream, but should we also add command line and config? (replay functions would probably fail anyway on the other parts)
Note, that 3600s limit is due to our timestamp representation, but it is never-the-less reasonable to split several GB files into smaller pieces.
I would add Zbynek comment from yesterday, that maybe we should have all nodes serialized at the beginning of a log file. Then they can be "deserialized in the middle" (say at the beginning of the next log file) and you do not need to re-run everything in order to replay some part.
Another note is related to #115 (logging of shutdown) - after that we do not need any "mark" that this is the end of split log file.