Distributing bxdecay0 data and gA data on CVMFS?

[drbenmorgan]

For SuperNEMO (and DUNE do as well) we have a CVMFS repository available for distributing software and large datasets. The gA data's an obvious candidate here, so I wanted to start a discussion on the best way to organize this with future versions/patches in mind (and also for packaging with separate packages for bxdecay0 and "bxdecay0-data").

At least at first sight, the CVMFS case would appear to be simple, in that given a CVMFS root, things could be organised like:

<CVMFSRoot>/
+- bxdecay0/
   +- vX.Y.Z/
      +- resources/
         ... files ...

Versions/resources levels could be swapped on preference. A client wanting to use this would then AFAIK, just need to set the BXDECAY0_RESOURCE_DIR to, e.g. <CVMFSRoot/bxdecay0/vX.Y.Z/resources (and that could be done in suitable scripts).

CVMFS has a nice de-duplication feature (it's content based), so it wouldn't be expensive to have direct copies of all the gA data for each bxdecay0 version. The only thing I'm not sure about here is matching bxdecay0 and gA data versions, but I think bxdecay0 does for users?

Anyway, I can start experimenting with the above for SuperNEMO and happy for our CVMFS to be a test bed here!

[fmauger]

Basically the resources/data dir contains various types of informations:

a) descriptive files that are read from the code to generate list of identifiers programmatically:
- background_isotopes.lis
- dbd_isotopes.lis
- dbd_modes.lis
b) physics data files like in the dbd_gA directory

They are not of the same nature. The a) files should be distributed with the code for they are read from the bb_utils.cc source at runtime.
I see no problem to make b) optionnaly distributable provided it does not enforce additional constraint or technological dependencies. The only case is for dbd_gA of course and it makes sense to be able to handle these large files on its own way like we can do with G4 datasets.

I think the a) files should be moved in some specific "resources/description/" directory and let the "resources/data" for specific datasets possibly managed by other means.

[fmauger]

Additionally, there is no reason why BXDECAY0_RESOURCE_DIR would be the shared root of BxDecay0 data.
I prefer BXDECAY0_RESOURCE_DIR to be used for non-data resources files (*.lis, legacy code...) and introduce specific BXDECAY0_XXX_DATA_DIR envs for specific datasets possibly installed/published through external tools.

[fmauger]

From commit #25718c8, the BXDECAY0_DBD_GA_DATA_DIR environment variable is used to locate a base directory which must contains the DBD gA datafiles. Such a directory must store the data/dbd_gA/vX.Y/ directories to fulfill the path naming scheme used so far in BxDecay0.