added metadata JSONs for the hdf and container repos at DESY as test run

[thdesy]

Hi,

for a test run, I added metadata to our minor/toy repos (hdf.desy.de, containers.desy.de), i.e., the generic info
http://hdf-cvmfs.desy.de/cvmfs/info/v1/meta.json {repositories.json}
and updated the per-repo jsons.
So far, we generate the JSONs automatically as a puppet template from Hiera infos - would it be reasonable to auto-commit/push any changes to a fork and create a pull request - or would that maybe overkill (I guess, the info would not change much)

btw:
where can I actual find the repo meta JSONs on the server? I tried mutations of
http://hdf-cvmfs.desy.de/cvmfs/info/v1/hdf.desy.de.json
and such but without success. The documentation under https://cvmfs.readthedocs.io/en/latest/cpt-servermeta.html just mentions the general meta and repository jsons urls

Cheers,
Thomas

[jblomer]

Thanks Thomas! The repository specific JSON snippet is stored in content-addressed form. Once set, you'll find the M key in the manifest (.cvmfspublished) that points to the repository meta information.

Regarding the json files on the cvmfs stratum servers, I'm not sure I understand. Is it about opening a PR with the puppet-cvmfs git repository?

[jblomer]

It's live but it would still need a little adjustment to the repository specific meta-data. The stratum 0 and stratum 1 URLs need to be the complete URLs, e.g. http://grid-cvmfs-null.desy.de:8000/cvmfs/containers.desy.de instead of grid-cvmfs-null.desy.de.

[thdesy]

Hi Jakob,

thanks for the merge.

Can I actually query the repository information somewhere from the CVMFS server's API path, i.e.,
http://hdf-cvmfs.desy.de/cvmfs/info/v1/...
I thought, that the M keys are the signature hashes of the JSONs, - but I have not found the repos' metainfos in a http URL.

The second question was maybe more for my local setup. My current setup looks like
Hiera --> Puppet Template --> Stratum:/etc/cvmfs/metadata/hdf.json
SSo, if someone changes a value in Hiera, I have to add the changed JSON to a branch/pull request. I could in principle add a short script/hook to the Puppet manifest, so that the stratum itself pushes the changed JSONs in a new branch and creates a request. But that might all be premature optimization...

Cheers,
Thomas

[1]
Me70e50087684cf002fc4adcbfc5169c86b278c02
M01e60338a2db9a71f5068a776ed2b96b512d4d60

[thdesy]

It's live but it would still need a little adjustment to the repository specific meta-data. The stratum 0 and stratum 1 URLs need to be the complete URLs, e.g. http://grid-cvmfs-null.desy.de:8000/cvmfs/containers.desy.de instead of grid-cvmfs-null.desy.de.

oops - OK. I just updated the repo metadata

[jblomer]

The server (or "global") meta-data are indeed accessible simply by the /info/v1/... URL. For the repository meta-data, you can

1. Use attr -g repo_metainfo /cvmfs/$repo on a mounted repository
2. Use curl and zlib decompression like this:

# Note the trailing `M` in the URL
curl http://grid-cvmfs-null.desy.de:8000/cvmfs/containers.desy.de/data/01/e60338a2db9a71f5068a776ed2b96b512d4d60M \
  | cvmfs_swissknife zpipe -d

I'd expect that neither the repo metainfo nor the server metainfo change very often. That might be taken into consideration when it comes to automating the deployment.