Open
Description
The record https://opendata.cern.ch/record/14011 says:
"number_events": 75482193,
"number_files": 28954,
"size": 166030973333340CMS DAS for /HIAllPhysics/HIRun2010-ZS-v2/RECO says:
"nevents": 79329398,
"nfiles": 32715,
"size": 190908704707416The record file index says:
- 32715 files
- 190908704707416 size
This number of files, the total size, and the listing of individual 32715 file paths from the record index file is fully consistent with what is being reported by CMS DAS.
However, the actual directory content on EOSPUBLIC under /eos/opendata/cms/hidata/HIRun2010/HIAllPhysics/RECO/ZS-v2/ shows:
- 31600 files
- 184065703469069 size
-
We should check and update total number of events and files as appropriate.
-
There appears to be 1115 missing files; the full list is at https://simko.web.cern.ch/tmp/recid-14011-missing-files.txt
Metadata
Metadata
Assignees
Type
Projects
Milestone
Relationships
Participants
Activity
tiborsimko commentedon Oct 15, 2024
@katilp @jmhogan @tpmccauley Can you please have a look at the list of missing files above? We might need to schedule data transfer to recover these files from the grid?
jmhogan commentedon Oct 16, 2024
@tiborsimko thanks for the poke, I had missed this in my spam folder (I've got filtering set up now, hopefully that happens less).
It does look like these missing files are only reported as existing on T1_US_FNAL_TAPE, so perhaps their transfer didn't complete successfully (the DAS record seems to suggest that).
@katilp Do I have permission to set up Rucio rules as DPOA? I haven't done it before, so I'm not sure.
katilp commentedon Oct 23, 2024
@jmhogan Sorry for the delay, it should all be in https://cms-opendata-releaseguide.docs.cern.ch/transfer/site_setup/
Let me check.
katilp commentedon Oct 23, 2024
@jmhogan I've added you to t3_ch_cern_opendata_local
jmhogan commentedon Oct 23, 2024
Missing blocks are:
None of them have a rule placing them at our site, so I think I just need to make 7 new rules for these particular blocks.
jmhogan commentedon Oct 23, 2024
Rules:
bash-4.4$ rucio add-rule cms:/HIAllPhysics/HIRun2010-ZS-v2/RECO#2c90d627-4424-4126-869a-840defda80f1 1 T3_CH_CERN_OpenData
5d3b53e10ca34e868923fad580f99dc9
bash-4.4$ rucio add-rule cms:/HIAllPhysics/HIRun2010-ZS-v2/RECO#bebc4ce9-7fad-411b-a789-d9b981e46f6f 1 T3_CH_CERN_OpenData
92373b0f64674b1c8ed6bd4b0cb4093a
bash-4.4$ rucio add-rule cms:/HIAllPhysics/HIRun2010-ZS-v2/RECO#52656ab9-000d-4b32-ae37-6e60f0dc48e9 1 T3_CH_CERN_OpenData
63d6f7ce6daa4248adbbd8f4d6bd7a73
bash-4.4$ rucio add-rule cms:/HIAllPhysics/HIRun2010-ZS-v2/RECO#51e3f795-4299-4bf9-b1d1-75064bddb15d 1 T3_CH_CERN_OpenData
48cbd8bcd7974cb0a0714e3d4a31cd49
bash-4.4$ rucio add-rule cms:/HIAllPhysics/HIRun2010-ZS-v2/RECO#50e679df-4d8c-4a6e-82dd-80cf5bf69d62 1 T3_CH_CERN_OpenData
70eaa77fd2ff41a79a5d588591fbd79e
bash-4.4$ rucio add-rule cms:/HIAllPhysics/HIRun2010-ZS-v2/RECO#3953b27d-5e00-444b-9621-f007c8f6c7f9 1 T3_CH_CERN_OpenData
70d16e65cc7c4d209ebd62236545a3b8
bash-4.4$ rucio add-rule cms:/HIAllPhysics/HIRun2010-ZS-v2/RECO#3852bf2b-3173-4df9-b267-f3a276a7ccdb 1 T3_CH_CERN_OpenData
905f6160f5c140d6bc7294aa116833e9