New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
A tool to change the GUID of a file manually for 8034-FileNameInconsistentWithGUID
issues
#37742
Comments
A new Issue was created by @haozturk Hasan ztrk. @Dr15Jones, @perrotta, @dpiparo, @makortel, @smuzaffar, @qliphy can you please review it and eventually sign/assign? Thanks. cms-bot commands are listed here |
assign core |
@Dr15Jones @dan131riley Any thoughts? |
New categories assigned: core @Dr15Jones,@smuzaffar,@makortel you have been requested to review this Pull request/Issue and eventually sign? Thanks |
@pcanal thoughts? |
The CMS file GUID is the only entry on the FileIdentifier branch, so in principle it should be possible to update it, or at worst make a copy with the old GUID. Doing so will change the file hash, which might have other implications for DBS entries, maybe the transfer system? For the future, I would much rather have a parameter to the PoolOutputModule to set the GUID so the replacement file gets created with the GUID of the original. I think this would be a lot cleaner for future file replacement, but I don't know how much of a manual process it would entail. |
I agree that approach would be cleaner. Copying @nsmith-'s messages from mattermost https://mattermost.web.cern.ch/cms-o-and-c/pl/5ssbgjxc4bnkdxipof3mhcasth
@drkovalskyi Can you comment if setting an additional parameter for |
An alternative solution to this problem is for T0 to invalidate the lost file in DBS and inject a new replacement file into the dataset, with the GUID being whatever the replacement file creates. |
@nsmith- our current procedure is to do exactly that (e.g CMSTRANSF-344). I'm not sure what was done for this run in 2018. I was not able to find any report of issues concerning run 322179. |
We can certainly add an option to set GUID in PoolOutputModule, but I don't see why we need it all. What's the use case for GUID check? Keep in mind that it's a "normal" practice for Tier0 to re-create files like that and we have a number of RAW files like that. This number is certainly small and with enough manual work it's possible to replace all such files updating everything everywhere, but do we need it? |
@drkovalskyi The option to enforce the GUID in the file name and inside the file to be consistent was requested by CompOps in dmwm/WMCore#9432 to detect the cases where xrootd was silently serving wrong files on some sites (at the time). |
Ok, that's an important use case. @germanfgv could you please add an option to set GUID in the PR that you are working for the compression? |
We would need to first add such an option to PoolOutputModule (so far it wasn't clear if such an option would be really needed, now the needs seems clear). |
Yes, we need this. We had enough cases where we had to do a manual recovery like that. |
#37806 adds an |
@drkovalskyi Would you need a backport for 12_3_X? (I presume earlier releases can be left out by now) |
We do need a back-port to 12_3_X since it's our current data taking release and such issues may raise anytime. Thanks. |
@drkovalskyi #37806 was merged, so the new parameter will be in 12_4_0_pre4. @drkovalskyi, can you test the parameter in the pre4 pre-release in any meaningful way, or should we just proceed with the 12_3_X backport? |
We should be able to run a replay with 12_4_0_pre4 and try manually to recreate a file. @germanfgv could you please help Jhonatan to make a test like that? |
+1 I suppose a 12_3_X backport is not needed anymore |
This issue is fully signed and ready to be closed. |
We came across w/ the following issue in production a few times:
Processing of such files were suggested to be done by bypassing this check, since the file content looked proper. However, it's not feasible w/ the existing workload management system as described at [1] Matti suggested to open this issue to discuss/develop a tool to update the GUID of files manually when necessary to allow their processing.
[1] https://its.cern.ch/jira/browse/CMSCOMPPR-24122
The text was updated successfully, but these errors were encountered: