Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shorten the names of two ALCARECOStreams in order to fit into DBS database schema limitations #38186

Merged
merged 3 commits into from Jun 2, 2022

Conversation

mmusich
Copy link
Contributor

@mmusich mmusich commented Jun 1, 2022

PR description:

It was pointed out in https://cms-talk.web.cern.ch/t/alcaprompt-datasets-not-loaded-in-dbs/11146/1 that, quoting:

Since the last update of the alcareco matrix, two new producers were added to ExpressPhysics:

  • PromptCalibProdSiPixelLorentzAngle
  • PromptCalibProdSiStripHitEfficiency
    while the resulting (harvested) ROOT files are regularly displayed in the DQM, the corresponding ALCAPROMPT datasets in DAS are missing.

The names of the datasets expected are:

/StreamExpress/Run2022A-PromptCalibProdSiStripHitEfficiency-Express-v1/ALCAPROMPT
/StreamExpress/Run2022A-PromptCalibProdSiPixelLorentzAngle-Express-v1/ALCAPROMPT

According to Tier0 experts the problem is related to the name of the dataset, which is too long to be accepted, and the WMAgent software at present is not signalling this but keeps trying the upload. While probably a better handling of this issue should be thought for future, we need to fix the cause of the problem by shortening the names.

This PR performs such shortening:

  • ALCARECOPromptCalibProdSiPixelLorentzAngle is renamed ALCARECOPromptCalibProdSiPixelLA
  • ALCARECOStreamPromptCalibProdSiStripHitEfficiency is renamed ALCARECOStreamPromptCalibProdSiStripHitEff

PR validation:

Run the following commands:

cmsDriver.py testReAlCa -s ALCA:PromptCalibProdSiPixelLA --conditions 123X_dataRun3_Express_v6 --scenario pp --data --era Run2_2018 --datatier ALCARECO --eventcontent ALCARECO --processName=ReAlCa -n 10000 --dasquery='file dataset=/StreamExpress/Run2022A-SiPixelCalSingleMuon-Express-v1/ALCARECO run=352572' --customise_commands='process.ALCARECOCalSignleMuonFilterForSiPixelLorentzAngle.TriggerResultsTag = cms.InputTag ( "TriggerResults","","HLT" ) ; process.ALCARECOCalSignleMuonFilterForSiPixelLorentzAngle.HLTPaths = ["*"]' --nThreads=4

cmsDriver.py stepHarvest -s ALCAHARVEST:SiPixelLA --conditions 123X_dataRun3_Express_v6  --scenario pp --data --era Run2_2018 --filein file:PromptCalibProdSiPixelLA.root -n -1

and

cmsDriver.py testReAlCa -s ALCA:PromptCalibProdSiStripHitEff --conditions 123X_dataRun2_v2 --scenario pp --data --era Run2_2018 --datatier ALCARECO --eventcontent ALCARECO --processName=ReAlCa -n 100000 --dasquery='file dataset=/StreamExpress/Run2018D-SiStripCalMinBias-Express-v1/ALCARECO run=325172' --nThreads=4

msDriver.py stepHarvest -s ALCAHARVEST:SiStripHitEff --conditions 123X_dataRun2_v2 --scenario pp --data --era Run2_2018 --filein file:PromptCalibProdSiStripHitEff.root -n -1 

Also exercised successfully the two runTheMatrix workflows running the PCL workflows:

  • runTheMatrix.py -l 1001,1001.2

if this PR is a backport please specify the original PR and why you need to backport that PR:

Not a backport, but needs to be backported to 12.4.X and 12.3.X

@mmusich
Copy link
Contributor Author

mmusich commented Jun 1, 2022

@fabiocos FYI

@tvami
Copy link
Contributor

tvami commented Jun 1, 2022

urgent

  • needed for datataking

@tvami
Copy link
Contributor

tvami commented Jun 1, 2022

type trk

@mmusich
Copy link
Contributor Author

mmusich commented Jun 1, 2022

@tvami, this is not a bug-fix, but just an adaptation to the inadequacies of the CMS offline tools.

…treamPromptCalibProdSiStripHitEff and ALCARECOStreamPromptCalibProdSiPixelLorentzAngle -> ALCARECOStreamPromptCalibProdSiPixelLA
@tvami
Copy link
Contributor

tvami commented Jun 1, 2022

this is not a bug-fix, but just an adaptation to the inadequacies of the CMS offline tools.

Ok, I edited my comment, the bugfix label should be gone soon

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 1, 2022

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38186/30340

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 1, 2022

A new Pull Request was created by @mmusich (Marco Musich) for master.

It involves the following packages:

  • Calibration/TkAlCaRecoProducers (alca)
  • Configuration/PyReleaseValidation (pdmv, upgrade)
  • Configuration/StandardSequences (operations)

@perrotta, @malbouis, @yuanchao, @jordan-martins, @bbilin, @tvami, @cmsbuild, @AdrianoDee, @srimanob, @kskovpen, @qliphy, @francescobrivio, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@mtosi, @beaucero, @fabiocos, @makortel, @felicepantaleo, @slomeo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @Martin-Grunewald, @missirol, @ebrondol, @tocheng, @lecriste, @trtomei, @mmusich, @threus, @dgulhan, @kpedro88 this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

@mmusich
Copy link
Contributor Author

mmusich commented Jun 1, 2022

For the record, as per https://cms-talk.web.cern.ch/t/alcaprompt-datasets-not-loaded-in-dbs/11146/2 the output_module_label name should not be longer than 45 chars:

$ echo ALCARECOStreamPromptCalibProdSiStripHitEfficiency | wc -c
50
$ echo ALCARECOStreamPromptCalibProdSiStripHitEff | wc -c
43
$ echo ALCARECOStreamPromptCalibProdSiPixelLorentzAngle | wc -c
49
$ echo ALCARECOStreamPromptCalibProdSiPixelLA | wc -c
39

@mmusich
Copy link
Contributor Author

mmusich commented Jun 1, 2022

test parameters:

  • workflow = 1001.2

@mmusich
Copy link
Contributor Author

mmusich commented Jun 1, 2022

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 2, 2022

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38186/30351

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 2, 2022

Pull request #38186 was updated. @perrotta, @malbouis, @yuanchao, @jordan-martins, @bbilin, @tvami, @cmsbuild, @AdrianoDee, @srimanob, @kskovpen, @qliphy, @francescobrivio, @fabiocos, @davidlange6 can you please check and sign again.

@mmusich
Copy link
Contributor Author

mmusich commented Jun 2, 2022

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 2, 2022

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-fa3a63/25203/summary.html
COMMIT: c011b6e
CMSSW: CMSSW_12_5_X_2022-06-02-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38186/25203/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-fa3a63/1001.2_RunZeroBias2017F+RunZeroBias2017F+TIER0EXPRUN2+ALCAEXPRUN2+ALCAHARVDSIPIXELCAL+ALCAHARVDSIPIXELCALLA

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 4 differences found in the comparisons
  • DQMHistoTests: Total files compared: 51
  • DQMHistoTests: Total histograms compared: 3650105
  • DQMHistoTests: Total failures: 8
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3650075
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
  • Checked 213 log files, 45 edm output root files, 51 DQM output files
  • TriggerResults: no differences found

@tvami
Copy link
Contributor

tvami commented Jun 2, 2022

+alca

  • resign

@srimanob
Copy link
Contributor

srimanob commented Jun 2, 2022

+Upgrade

Resign

@kskovpen
Copy link
Contributor

kskovpen commented Jun 2, 2022

+pdmv

@tvami
Copy link
Contributor

tvami commented Jun 2, 2022

@cms-sw/orp-l2 we are fully signed (again :D )

@perrotta
Copy link
Contributor

perrotta commented Jun 2, 2022

+1

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 2, 2022

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

@cmsbuild cmsbuild merged commit 1d3b5c4 into cms-sw:master Jun 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants