Ecal laser alphas

[depasse]

Laser alpha now Xtal by Xtal from a txt file. Upgrade : before, only a mean for Russian/Chinese Xtals.

[cmsbuild]

A new Pull Request was created by @depasse for CMSSW_9_0_X.

It involves the following packages:

CalibCalorimetry/EcalTrivialCondModules
CondTools/Ecal

@ghellwig, @arunhep, @cerminar, @cmsbuild, @franzoni, @ggovi, @mmusich, @davidlange6 can you please review it and eventually sign? Thanks.
@ghellwig, @apfeiffer1, @tocheng, @argiro this is something you requested to watch as well.
@davidlange6, @smuzaffar you are the release manager for this.

cms-bot commands are listed here #13028

[arunhep]

@cmsbuild please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/17351/console Started: 2017/01/20 20:37

[cmsbuild]

+1
Tested at: 3e25660
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17234/17351/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17234/17351/summary.html

[franzoni]

hello @depasse
what are you trying to achieve by loading alpha values in the release as text files ?
Can db tas and candidate GT's help you
( instructions on how to compose candidate GT's https://indico.cern.ch/event/507993/ - we're willing to help of course)

[depasse]

The aim of the PR is to create a sqlite_file from simple txt files given by anyone.
EBLaserAlpha_fromFile.txt and EELaserAlpha_fromFile.txt are only examples to show the format needed (may be removed if necessary).
But EBLaserAlpha.txt and EELaserAlpha.txt are essential : there are the only source of the construction barcode of each Xtal, allowing to distinguish between Russian/Chinese.

[franzoni]

hello @depasse , I see that the data files range between 0.2 to 3 MB

@smuzaffar @davidlange6
can you remind us the policy, what is the size above which we need to move data files out of the standard CMSSW REPO ?
Would in that case https://github.com/cms-sw/cmsdist be the place, can you please remind us the policy and recomandations ?

Thanks.

[franzoni]

@smuzaffar @davidlange6
can you remind us the policy, what is the size above which we need to move data files out of the standard CMSSW REPO ?

Thanks!

[smuzaffar]

@franzoni , although there is no min-size limit policy but in practice any thing above 10K (or binary files) should go to https://github.com/cms-data . All of data files in this PR are good candidate to go to https://github.com/cms-data/CalibCalorimetry-EcalTrivialCondModules (which does not exist yet).

[davidlange6]

Right - with one additional caveat - that the data is needed for cmssw applications. (eg, its not a good home for a backup cache of something in a db)

…

On Feb 10, 2017, at 12:05 PM, Malik Shahzad Muzaffar ***@***.***> wrote: @franzoni , although there is no min-size limit policy but in practice any thing above 10K (or binary files) should go to https://github.com/cms-data . All of data files in this PR are good candidate to go to https://github.com/cms-data/CalibCalorimetry-EcalTrivialCondModules (which does not exist yet). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[depasse]

The aim of the PR is to create a sqlite_file from simple txt files given by anyone : it is the usual way to proceed). It is not a DB backup.
Could you then give me the instructions to create https://github.com/cms-data/CalibCalorimetry-EcalTrivialCondModules/data (if I'm authorized to proceed), how to copy the txt files and how to modify my python files in CondTools/Ecal/test/alpha (presently : process.EcalTrivialConditionRetriever.EELaserAlphaFile = cms.untracked.string('CalibCalorimetry/EcalTrivialCondModules/d
ata/EELaserAlpha_fromFile.txt'). Thanks a lot.

[franzoni]

hello @smuzaffar @davidlange6
would you have a moment to point @depasse to instructions on how to do that ?
Thanks!

[franzoni]

hello @depasse

creating the folder suggested by @smuzaffar in https://github.com/cms-data follows the same logic as any other git-based project: clone the repository locally, add your files in the proposed directory, make a pull request to the github repository.

This post gives you the elements to than access the files from your test .py configurations: cms-data/GeneratorInterface-EvtGenInterface#9 (comment)

HTH!

[depasse]

Hi Giovanni, so I must import in my github area https://github.com/cms-data ? As the directory https://github.com/cms-data/CalibCalorimetry-EcalTrivialCondModules/data does not exist yet. Then New "File" ? Sorry but I don't see where to find the info in cms-data/GeneratorInterface-EvtGenInterface#9 (comment)<cms-data/GeneratorInterface-EvtGenInterface#9 (comment)> How must I change process.EcalTrivialConditionRetriever.EELaserAlphaFile = cms.untracked.string('CalibCalorimetry/EcalTrivialCondModules/d ata/EELaserAlpha_fromFile.txt' : process.EcalTrivialConditionRetriever.EELaserAlphaFile = cms.untracked.string('https://github.com/cms-data/CalibCalorimetry-EcalTrivialCondModules/data/EELaserAlpha_fromFile.txt') ? Pierre

…

________________________________ From: Giovanni Franzoni [notifications@github.com] Sent: 17 February 2017 04:09 To: cms-sw/cmssw Cc: Pierre Depasse; Mention Subject: Re: [cms-sw/cmssw] Ecal laser alphas (#17234) hello @depasse<https://github.com/depasse> creating the folder suggested by @smuzaffar<https://github.com/smuzaffar> in https://github.com/cms-data follows the same logic as any other git-based project: clone the repository locally, add your files in the proposed directory, make a pull request to the github repository. This post gives you the elements to than access the files from your test .py configurations: cms-data/GeneratorInterface-EvtGenInterface#9 (comment)<cms-data/GeneratorInterface-EvtGenInterface#9 (comment)> HTH! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#17234 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AEaCW3awAd0u-BwRn6WlUUIqeZYf0Yrsks5rdQ9_gaJpZM4LplNR>.

[smuzaffar]

@depasse , repository https://github.com/cms-data/CalibCalorimetry-EcalTrivialCondModules now exists. Please fork it and then make a PR with your updates

[depasse]

How may I reach the files in the new repository in a python file ? I used : process.EcalTrivialConditionRetriever.EELaserAlphaFile = cms.untracked.string('CalibCalorimetry/EcalTrivialCondModules/d ata/EELaserAlpha_fromFile.txt' on CMSSW. Thanks

…

________________________________ From: Malik Shahzad Muzaffar [notifications@github.com] Sent: 17 February 2017 09:39 To: cms-sw/cmssw Cc: Pierre Depasse; Mention Subject: Re: [cms-sw/cmssw] Ecal laser alphas (#17234) @depasse<https://github.com/depasse> , repository https://github.com/cms-data/CalibCalorimetry-EcalTrivialCondModules now exists. Please fork it and then make a PR with your updates — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#17234 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AEaCWyOvFXiKe8LTqJnQVnJh2jwX6Altks5rdVzCgaJpZM4LplNR>.

[smuzaffar]

@depasse , you can access using FileInPath see some examples here https://github.com/cms-sw/cmssw/search?l=Python&q=FileInPath&utf8=%E2%9C%93

[smuzaffar]

there is now cmsdist PR available for the data files cms-sw/cmsdist#2831.
@depasse , you can remove the data files from this PR and then once can start tests by using
please test with cms-sw/cmsdist#2831 cms-bot command

[smuzaffar]

closing in favour of #17546

[depasse]

Hi, when I want to test the "fromFile" version : $ cmsRun create_alpha_trivial_fromFILE.py ----- Begin Fatal Exception 18-Feb-2017 09:09:20 CET----------------------- An exception of category 'ConfigFileReadError' occurred while [0] Processing the python configuration file named create_alpha_trivial_fromFILE.py Exception Message: python encountered the error: <type 'exceptions.RuntimeError'> edm::FileInPath unable to find file CalibCalorimetry/EcalTrivialCondModules/data/EBLaserAlpha_fromFile.txt anywhere in the search path. The search path is defined by: CMSSW_SEARCH_PATH ${CMSSW_SEARCH_PATH} is: /afs/cern.ch/work/d/depasse/cmssw/CMSSW_9_0_0_pre4/src:/afs/cern.ch/work/d/depasse/cmssw/CMSSW_9_0_0_pre4/external/slc6_amd64_gcc530/data:/cvmfs/cms.cern.ch/slc6_amd64_gcc530/cms/cmssw/CMSSW_9_0_0_pre4/src:/cvmfs/cms.cern.ch/slc6_amd64_gcc530/cms/cmssw/CMSSW_9_0_0_pre4/external/slc6_amd64_gcc530/data Current directory is: /afs/cern.ch/work/d/depasse/cmssw/CMSSW_9_0_0_pre4/src/CondTools/Ecal/test/alpha

…

----- End Fatal Exception ------------------------------------------------- What must I add/do ? Thanks. Pierre

________________________________ From: Malik Shahzad Muzaffar [notifications@github.com] Sent: 17 February 2017 18:21 To: cms-sw/cmssw Cc: Pierre Depasse; Mention Subject: Re: [cms-sw/cmssw] Ecal laser alphas (#17234) closing in favour of #17546<#17546> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#17234 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AEaCW3vSWuxreencSPyv-SUfxdQTs7G9ks5rddcegaJpZM4LplNR>.

[davidlange6]

You need to use a release with your data- I guess this is CMSSW_9_0_X_2017-02-17-2300 for example. pre4 is too old.

…

On Feb 18, 2017, at 9:12 AM, depasse ***@***.***> wrote: Hi, when I want to test the "fromFile" version : $ cmsRun create_alpha_trivial_fromFILE.py ----- Begin Fatal Exception 18-Feb-2017 09:09:20 CET----------------------- An exception of category 'ConfigFileReadError' occurred while [0] Processing the python configuration file named create_alpha_trivial_fromFILE.py Exception Message: python encountered the error: <type 'exceptions.RuntimeError'> edm::FileInPath unable to find file CalibCalorimetry/EcalTrivialCondModules/data/EBLaserAlpha_fromFile.txt anywhere in the search path. The search path is defined by: CMSSW_SEARCH_PATH ${CMSSW_SEARCH_PATH} is: /afs/cern.ch/work/d/depasse/cmssw/CMSSW_9_0_0_pre4/src:/afs/cern.ch/work/d/depasse/cmssw/CMSSW_9_0_0_pre4/external/slc6_amd64_gcc530/data:/cvmfs/cms.cern.ch/slc6_amd64_gcc530/cms/cmssw/CMSSW_9_0_0_pre4/src:/cvmfs/cms.cern.ch/slc6_amd64_gcc530/cms/cmssw/CMSSW_9_0_0_pre4/external/slc6_amd64_gcc530/data Current directory is: /afs/cern.ch/work/d/depasse/cmssw/CMSSW_9_0_0_pre4/src/CondTools/Ecal/test/alpha ----- End Fatal Exception ------------------------------------------------- What must I add/do ? Thanks. Pierre ________________________________ From: Malik Shahzad Muzaffar ***@***.*** Sent: 17 February 2017 18:21 To: cms-sw/cmssw Cc: Pierre Depasse; Mention Subject: Re: [cms-sw/cmssw] Ecal laser alphas (#17234) closing in favour of #17546<#17546> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#17234 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AEaCW3vSWuxreencSPyv-SUfxdQTs7G9ks5rddcegaJpZM4LplNR>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[depasse]

I will wait for a next week version. It appears that the new github directory is still not active? or my syntax is not correct ? : process.EcalTrivialConditionRetriever.EBLaserAlphaFile = cms.untracked.FileInPath('CalibCalorimetry/EcalTrivialCondModul es/data/EBLaserAlpha_fromFile.txt') ~/WORK/cmssw/CMSSW_9_0_X_2017-02-17-2300/src/CondTools/Ecal/test/alpha $ cmsRun create_alpha_trivial_fromFILE.py ----- Begin Fatal Exception 18-Feb-2017 10:19:28 CET----------------------- An exception of category 'ConfigFileReadError' occurred while [0] Processing the python configuration file named create_alpha_trivial_fromFILE.py Exception Message: python encountered the error: <type 'exceptions.RuntimeError'> edm::FileInPath unable to find file CalibCalorimetry/EcalTrivialCondModules/data/EBLaserAlpha_fromFile.txt anywhere in the search path. The search path is defined by: CMSSW_SEARCH_PATH ${CMSSW_SEARCH_PATH} is: /afs/cern.ch/work/d/depasse/cmssw/CMSSW_9_0_X_2017-02-17-2300/src:/afs/cern.ch/work/d/depasse/cmssw/CMSSW_9_0_X_2017-02-17-2300/external/slc6_amd64_gcc530/data:/cvmfs/cms-ib.cern.ch/week1/slc6_amd64_gcc530/cms/cmssw/CMSSW_9_0_X_2017-02-17-2300/src:/cvmfs/cms-ib.cern.ch/week1/slc6_amd64_gcc530/cms/cmssw/CMSSW_9_0_X_2017-02-17-2300/external/slc6_amd64_gcc530/data Current directory is: /afs/cern.ch/work/d/depasse/cmssw/CMSSW_9_0_X_2017-02-17-2300/src/CondTools/Ecal/test/alpha

----- End Fatal Exception ------------------------------------------------- Thanks a lot.

________________________________ From: David Lange [notifications@github.com] Sent: 18 February 2017 09:39 To: cms-sw/cmssw Cc: Pierre Depasse; Mention Subject: Re: [cms-sw/cmssw] Ecal laser alphas (#17234) You need to use a release with your data- I guess this is CMSSW_9_0_X_2017-02-17-2300 for example. pre4 is too old.

On Feb 18, 2017, at 9:12 AM, depasse ***@***.***> wrote: Hi, when I want to test the "fromFile" version : $ cmsRun create_alpha_trivial_fromFILE.py ----- Begin Fatal Exception 18-Feb-2017 09:09:20 CET----------------------- An exception of category 'ConfigFileReadError' occurred while [0] Processing the python configuration file named create_alpha_trivial_fromFILE.py Exception Message: python encountered the error: <type 'exceptions.RuntimeError'> edm::FileInPath unable to find file CalibCalorimetry/EcalTrivialCondModules/data/EBLaserAlpha_fromFile.txt anywhere in the search path. The search path is defined by: CMSSW_SEARCH_PATH ${CMSSW_SEARCH_PATH} is: /afs/cern.ch/work/d/depasse/cmssw/CMSSW_9_0_0_pre4/src:/afs/cern.ch/work/d/depasse/cmssw/CMSSW_9_0_0_pre4/external/slc6_amd64_gcc530/data:/cvmfs/cms.cern.ch/slc6_amd64_gcc530/cms/cmssw/CMSSW_9_0_0_pre4/src:/cvmfs/cms.cern.ch/slc6_amd64_gcc530/cms/cmssw/CMSSW_9_0_0_pre4/external/slc6_amd64_gcc530/data Current directory is: /afs/cern.ch/work/d/depasse/cmssw/CMSSW_9_0_0_pre4/src/CondTools/Ecal/test/alpha ----- End Fatal Exception ------------------------------------------------- What must I add/do ? Thanks. Pierre ________________________________ From: Malik Shahzad Muzaffar ***@***.*** Sent: 17 February 2017 18:21 To: cms-sw/cmssw Cc: Pierre Depasse; Mention Subject: Re: [cms-sw/cmssw] Ecal laser alphas (#17234) closing in favour of #17546<#17546> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#17234 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AEaCW3vSWuxreencSPyv-SUfxdQTs7G9ks5rddcegaJpZM4LplNR>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#17234 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AEaCW5SD7fTv0-XUdGt_T_Ksx58mJYpmks5rdq5HgaJpZM4LplNR>.