Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restructure recorded_data folder #260

Open
Nanoseb opened this issue Sep 12, 2018 · 4 comments
Open

Restructure recorded_data folder #260

Nanoseb opened this issue Sep 12, 2018 · 4 comments

Comments

@Nanoseb
Copy link
Member

Nanoseb commented Sep 12, 2018

For now all the logs are in the same folder, which lead to 1300 files in one place. The filenames not starting with the timestamp but their content (like param-dump) is not helping the sorting.
To improve that I thought about having a folder per launch with the name: 2018-08-28T12.41.55-calshot-triangular-race
And then in it:

  • rosbag.bag
  • param-dump.json
  • gps-raw-nmea
  • gps-trace.csv
  • map-gps-trace.html
  • rosbag.bag.metadata

When/If we changes this file structure it will lead to adapting the launch files that stores bags etc. as well as the index_recorded_data.py script that generates the html page and .metadata.

@takluyver
Copy link
Member

The index was designed to alleviate this a bit, but I agree that it would make sense to arrange the files into folders as well.

@Nanoseb
Copy link
Member Author

Nanoseb commented Sep 12, 2018

Also having this kind of folder structure means that we can have a script that easily in one command: run roscore, load the parameter dump, play the rosbag and launch the 2D visualisation for example. Which is tedious at the moment because each file has a different name structure.

@Nanoseb
Copy link
Member Author

Nanoseb commented Sep 12, 2018

What needs to be edited:

  • path/filename in script/debugging_dump_params
  • path/filename in script/debugging_gps_log
  • path/filename in launchfile.launch (rosbag record line)
  • path/filename in script/sensor_driver_gps for raw-nmea file

One of the difficult aspect here is to have a consistent timestamp that is propagated and used by all the files (=how to set a parameter with the timestamp of the beginning of the launch)

And also:

  • index_recorded_data.py file handling

  • write a one time script to convert the old file structure to the new one (part of index_recorded_data.py can be reused to associate file belonging to the same mission together)

@smaria
Copy link
Member

smaria commented Nov 9, 2018

IIRC there is no easy way of making rosbag record append a postfix rather than using a prefix. For Delphin2 I ended up using a bash script for launching, which recorded the start time and then handled all the naming at the end.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants