New stratum1

[lecorguille]

Hi here,

We finally have our new storage infrastructure [and time], to become a stratum1 for the galaxyproject cvmfs repos.
(Note that we will format it completely mid-June because the installation wasn't well done by the saler 😭 ).

Anyway ! I wanted to focus on the singularity one and extend to the other one soon (or if it's mandatory).
I ran the ansible playbook and

$ nohup /usr/bin/cvmfs_server snapshot  singularity.galaxyproject.org &


$ tail nohup.out
Replicating from catalog at /.cvmfs
  Processing chunks [1 registered chunks]:  fetched 0 new chunks out of 1 unique chunks
Replicating from update-subset-a-2020-03-26T03:24:51Z repository tag
  Processing chunks [33426 registered chunks]: .................... fetched 0 new chunks out of 19951 unique chunks
Replicating from generic-2020-03-25T15:10:35Z repository tag
  Processing chunks [1 registered chunks]:  fetched 0 new chunks out of 1 unique chunks
Stopping 16 workers
Uploading manifest ensemble
Serving revision 811
Fetched 1219941 new chunks out of 388842575 processed chunks


$ du -sh singularity.galaxyproject.org/
6.3T	singularity.galaxyproject.org/

• Can I check the integrity or the completness of the repo?

$ /usr/bin/cvmfs_server check singularity.galaxyproject.org
Verifying integrity of singularity.galaxyproject.org...
failed to fetch repository manifest (1 - failed to download)
failed to load repository manifest

Is it normal?

• What next?
  • Do we need to make this server available? with the 80 port?

[lecorguille]

Up, harder :/

[mvdbeek]

@natefoo can you take a look at this ?

[natefoo]

This fetches from your local stratum 1. If the manifest fetch is failing, can you verify that you can e.g. curl -o cvmfspublished http://localhost/cvmfs/singularity.galaxyproject.org/.cvmfspublished from the command line? If not, it's probably an issue with the squid or Apache configuration on the system - logs in /var/log/squid and/or /var/log/httpd should provide some insight on that.

[abretaud]

That's fixed, it was a permission problem due to our storage (local apache user could not access cvmfs data dir)

(I can't close the issue though)