CMB-S4 data portal prototype

Implementation of data cataloging and sharing for the CMB-S4 experiment based on the LSST DESC Data Portal.

Permissions

The portal supports 3 levels of permissions:

• Public: accessible to all users that can login to Globus
• CMB-S4: members of the CMB-S4 collaboration (Membership is synchronized using the Globus SDK)
• CMB-S4 DM: members of the Data Management group

Please Open an issue if permissions are not up to date.

Add a dataset to the portal

The data is stored on CMB-S4 project space in the NERSC Community File System (CFS) at:

/global/project/projectdirs/cmbs4/gsharing/globus

The portal doesn't support nested folders, so you first need to restructure the data so that they just a single folder or a list of a few folders with only files in them.

Then we need to add the dataset to the JSON dataset description in:

web/portal/data/datasets.json

copy-paste one of the other definitions, generate a new UUID with uuidgen, adjust the tags.

We can optionally create a "Dataset filter" (for example in the current portal we have "Data Challenge 1" and "Design tool simulations 1" to group folders together.

There are convenience functions to download an entire folder or the users can select individual files.