Extend ForceField to read compressed ffxml files?

[jchodera]

To save storage space, it may be useful to extend ForceField to be able to read .gz compressed ffxml files.

[jchodera]

@peastman : Given the size of the CHARMM ffxml file (many MB), what would you think of supporting automatic handling of gzipped ffxml files?

I wonder if we may even want to support compressed serialized System XML files at the C++ layer too. These files can be enormous.

[peastman]

what would you think of supporting automatic handling of gzipped ffxml files?

What would be the benefit of that? It would save a little disk space, but on the scale of modern disks, that's irrelevant. And it would have no effect on download time when installing it, because git and conda both already transfer data in compressed form.

[jchodera]

What would be the benefit of that? It would save a little disk space, but on the scale of modern disks, that's irrelevant. And it would have no effect on download time when installing it, because git and conda both already transfer data in compressed form.

For state.xml and system.xml files, for Folding@home, we're talking about many, many TB of data on our FAH servers and a huge amount of network traffic and download/upload times. For example, for project 10496, the state.xml file is 44MB uncompressed, 15MB compressed. That project has 1,930,000 of those files, so this makes a difference of (1.93e6)(44-15)M = 55 TB of data.

For the ffxml file, it's less of a concern, but the compatible patch lists does inflate things quite a bit for CHARMM, weighing in around 10MB in that initial case.

There's essentially no drawback to compressing these files---it's very fast and takes little CPU time---but once you have more than a few of these files hanging around, the space savings adds up quite a bit.