Time the i/o section

[MatthieuSchaller]

Would it be possible to time the i/o part of the code? Even better if we could time individually the writing of the properties catalogs, the parttype and the rest.

[rtobar]

Yes, this should be possible I think. It would also alter probably the same portion of code touched by #27, I'll do this first though.

[MatthieuSchaller]

I should have given you some more context, sorry. It looks like compressing the .properties file saves a factor of 3 in disk space. We would like to see whether that does not come at a huge penalty in writing time. For instance, it could be that writing the compressed id catalogue file is very slow for little benefit but the .properties is fast and compresses well.

[rtobar]

Thanks @MatthieuSchaller for the extra context, it definitely helps in deciding how far to go with this. I'm doing something quite simple to begin with, and it sounds like it should be enough to do the comparisons you are describing.

[rtobar]

How does this look as a first iteration? (output trimmed to final lines)

[...]
saving SO particle lists to lala.catalog_SOlist.0
Wrote lala.catalog_SOlist.0 in 3.104 [s]
Done SO masses for field objects in 20.098
Memory report, func = SortAccordingtoBindingEnergy--line--4719 task = 0 : Average = 1.746150 GB, Data = 1.640755 GB, Dirty = 0.000000 GB, Library = 0.000000 GB, Peak = 2.915272 GB, Resident = 1.612049 GB, Shared = 0.017147 GB, Size = 1.838261 GB, Text = 0.001534 GB, 
Done
saving property data to lala.properties.0
Wrote lala.properties.0 in 51 [ms]
saving group catalog to lala.catalog_groups.0
saving particle catalog to lala.catalog_particles.0
Wrote catalogues in 2.816 [s]
saving hierarchy data to lala.catalog_groups.0
saving hierarchy data to lala.hierarchy.0
Wrote lala.hierarchy.0 in 3 [ms]
saving particle type info to lala.catalog_parttypes.0
Wrote particle type info in 202 [ms]
saving profiles lala.profiles.0
Wrote lala.profiles.0 in 10 [ms]
TIME::0 took 304.657 in all
0 : finished running VR 

[MatthieuSchaller]

Yes, perfect. Exactly what I was hoping for.

[rtobar]

I've pushed these changes to the io-timing branch, could you give them a try? If you're happy with how they look I'll merge to master.

[MatthieuSchaller]

saving property data to halos_0036.compressed.properties.0
Wrote halos_0036.compressed.properties.0 in 3.354 [s]
saving group catalog to halos_0036.compressed.catalog_groups.0
saving particle catalog to halos_0036.compressed.catalog_particles.0
Wrote catalogues in 1.198 [s]
saving hierarchy data to halos_0036.compressed.catalog_groups.0
saving hierarchy data to halos_0036.compressed.hierarchy.0
Wrote halos_0036.compressed.hierarchy.0 in 10 [ms]
saving particle type info to halos_0036.compressed.catalog_parttypes.0
Wrote particle type info in 762 [ms]
saving profiles halos_0036.compressed.profiles.0
Wrote halos_0036.compressed.profiles.0 in 307 [ms]
TIME::0 took 1391.607 in all

All looks good to me, thanks.

[rtobar]

Merged to master now.