Using CSV format by default for simulations

[al8n]

I suppose using CSV format by default should be straightforward, but unfortunately, I forgot that complex structs and vec (nested with struct) are not supported by csv crate. I solve this problem by manually implementing CarnotState::serialize deeper and adding a new Record type.

[danielSanchezQ]

Did the performance improved?

[al8n]

Did the performance improved?

The changes in this PR seem not to improve the performance. However, I found several small places about serialization (mainly about avoiding deep clones and the serialization output format) that can be optimized. I will make more small PRs to optimize them.

[bacv]

Minor fix needed for csv writer, otherwise looks good.

[bacv]

The changes in this PR seem not to improve the performance. However, I found several small places about serialization (mainly about avoiding deep clones and the serialization output format) that can be optimized. I will make more small PRs to optimize them.

When application uses json as output format for step state, every line of generated json file contains the list of node states during that step. When using csv file, node state during the step is saved as an individual line. The end user might need take that into account when processing data.
One idea for a new PR is to add a STEP_ID to the record keys.

[al8n]

The changes in this PR seem not to improve the performance. However, I found several small places about serialization (mainly about avoiding deep clones and the serialization output format) that can be optimized. I will make more small PRs to optimize them.

When application uses json as output format for step state, every line of generated json file contains the list of node states during that step. When using csv file, node state during the step is saved as an individual line. The end user might need take that into account when processing data. One idea for a new PR is to add a STEP_ID to the record keys.

I like the STEP_ID idea. I make some new changes to match CSV format, in JSON, the node states also be flattened to match the CSV format now. I will open a new PR to optimize the serialization so that a step_id is assigned to each node state.

[bacv]

Tested, all fields serializes as it should in json and csv, amazing work. 🚀