Change DBS url to cmsweb-prod in MatrixInjector #33209

justinasr · 2021-03-18T10:11:05Z

PR description:

Use cmsweb-prod.cern.ch for "DbsUrl" attribute in the dictionary that is submitted to ReqMgr2 by MatrixInjector.

Relevant ticket on gitlab: https://gitlab.cern.ch/cms-http-group/doc/-/issues/256

PR validation:

None.

If this PR is a backport please specify the original PR and why you need to backport that PR:

Not a backport.

cmsbuild · 2021-03-18T10:16:22Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33209/21646

This PR adds an extra 20KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/MatrixInjector.py modified in PR(s): Pilot PR for the GPU attributes in workflow injected by runTheMatrix #33057

cmsbuild · 2021-03-18T10:16:47Z

A new Pull Request was created by @justinasr (Justinas Rumševičius) for master.

It involves the following packages:

Configuration/PyReleaseValidation

@jordan-martins, @chayanit, @wajidalikhan, @kpedro88, @cmsbuild, @srimanob can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @fabiocos, @slomeo this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

chayanit · 2021-03-18T11:03:29Z

please test

smuzaffar · 2021-03-18T13:40:49Z

Configuration/PyReleaseValidation/python/MatrixInjector.py

@@ -71,7 +71,7 @@ def __init__(self,opt,mode='init',options=''):
        if not self.wmagent:
            if not opt.testbed :
                self.wmagent = 'cmsweb.cern.ch'
-                self.DbsUrl = "https://"+self.wmagent+"/dbs/prod/global/DBSReader"
+                self.DbsUrl = "https://cmsweb-prod.cern.ch/dbs/prod/global/DBSReader"


although it was already hardcoded, why not configure it via env variable. This will help in future when we have to change it again.

That would be the most flexible way, however it would then depend on each user having to export it before every use. Unless you are talking about env variables that are set by cmsenv, then yes, it could be set there. I also don't expect this to change that often that it would cause too much of inconvenience to change the hardcoded value.

@justinasr , I mean use the env variable to override the default. If env variable is set then we use it otherwise fallback to default ( hardcoded in the release).

@smuzaffar sure, should the whole url be configurable or just the host? Would these be appropriate names for the env variables: DBSREADER_URL (if the whole url is configurable) or DBSREADER_HOST (if only host is configurable)?

@justinasr , I would suggest to make whole url configurable. About the name, I would suggest to use CMS_DBSREADER_URL . If you want, you can also add command-line option e.g. --dbs-url ful-dbs-url to runTheMatrix.py :-)

cmsbuild · 2021-03-18T13:43:27Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-f3e720/13589/summary.html
COMMIT: fc3c7b9
CMSSW: CMSSW_11_3_X_2021-03-17-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/33209/13589/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 37
DQMHistoTests: Total histograms compared: 2639935
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 2639906
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 36 files compared)
DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
Checked 155 log files, 37 edm output root files, 37 DQM output files
TriggerResults: no differences found

srimanob · 2021-03-26T01:56:51Z

Hi @smuzaffar @justinasr
Do you converge on this PR? Thanks.

smuzaffar · 2021-03-26T08:31:34Z

@srimanob , as we are changing it from one hardcoded value to another, I suggested to provide a way to override it in future via an environment variable ( see #33209 (comment) ) .

cmsbuild · 2021-03-29T13:32:33Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33209/21806

This PR adds an extra 24KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/MatrixInjector.py modified in PR(s): Pilot PR for the GPU attributes in workflow injected by runTheMatrix #33057
- File Configuration/PyReleaseValidation/scripts/runTheMatrix.py modified in PR(s): Add interactive option to runTheMatrix #33082, Pilot PR for the GPU attributes in workflow injected by runTheMatrix #33057

cmsbuild · 2021-03-29T13:32:59Z

Pull request #33209 was updated. @jordan-martins, @chayanit, @wajidalikhan, @kpedro88, @cmsbuild, @srimanob can you please check and sign again.

justinasr · 2021-03-29T13:36:21Z

@smuzaffar Updated so it would be configurable with env variable and --dbs-url. I believe this PR still needs some tests triggered.

srimanob · 2021-03-29T14:32:51Z

Please test

cmsbuild · 2021-03-29T17:14:06Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-f3e720/13827/summary.html
COMMIT: 704a42e
CMSSW: CMSSW_11_3_X_2021-03-28-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/33209/13827/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

The workflows 140.53 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary. You can check it in the "files" directory in the results of the comparisons

Summary:

No significant changes to the logs found
Reco comparison results: 1264 differences found in the comparisons
DQMHistoTests: Total files compared: 37
DQMHistoTests: Total histograms compared: 2639935
DQMHistoTests: Total failures: 3674
DQMHistoTests: Total nulls: 19
DQMHistoTests: Total successes: 2636220
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 45.703 KiB( 36 files compared)
DQMHistoSizes: changed ( 140.53 ): 44.531 KiB Hcal/DigiRunHarvesting
DQMHistoSizes: changed ( 140.53 ): 1.172 KiB RPC/DCSInfo
Checked 155 log files, 37 edm output root files, 37 DQM output files
TriggerResults: no differences found

srimanob · 2021-03-30T06:26:00Z

+Upgrade

This is a technical PR to assign dbsUrl attribute in the dictionary, which is submitted to ReqMgr2 by MatrixInjector.

@smuzaffar This update looks OK for me. Is it as you expect? Thanks.

smuzaffar · 2021-03-30T06:37:27Z

yes @srimanob this looks good now.

chayanit · 2021-03-30T09:07:36Z

+1

cmsbuild · 2021-03-30T09:07:59Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2021-03-30T09:21:08Z

+1

Change DBS url to cmsweb-prod in MatrixInjector

fc3c7b9

cmsbuild added this to the CMSSW_11_3_X milestone Mar 18, 2021

cmsbuild added code-checks-pending orp-pending pdmv-pending pending-signatures tests-pending upgrade-pending labels Mar 18, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 18, 2021

This was referenced Mar 18, 2021

[Backport to 11_2_X] Change DBS url to cmsweb-prod in MatrixInjector #33210

Merged

[Backport to 10_6_X] Change DBS url to cmsweb-prod in MatrixInjector #33211

Merged

cmsbuild added tests-started and removed tests-pending labels Mar 18, 2021

smuzaffar reviewed Mar 18, 2021

View reviewed changes

cmsbuild added tests-approved and removed tests-started labels Mar 18, 2021

Make DbsUrl configurable via runTheMatrix argument and env variable

704a42e

cmsbuild added code-checks-pending tests-pending and removed code-checks-approved tests-approved labels Mar 29, 2021

cmsbuild removed the code-checks-pending label Mar 29, 2021

cmsbuild added the code-checks-approved label Mar 29, 2021

cmsbuild added tests-started and removed tests-pending labels Mar 29, 2021

cmsbuild added tests-approved and removed tests-started labels Mar 29, 2021

cmsbuild added upgrade-approved and removed upgrade-pending labels Mar 30, 2021

cmsbuild added fully-signed pdmv-approved and removed pdmv-pending pending-signatures labels Mar 30, 2021

cmsbuild added orp-approved and removed orp-pending labels Mar 30, 2021

cmsbuild merged commit b4193ae into cms-sw:master Mar 30, 2021

justinasr deleted the cmsweb-prod branch May 4, 2021 17:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Change DBS url to cmsweb-prod in MatrixInjector #33209

Change DBS url to cmsweb-prod in MatrixInjector #33209

justinasr commented Mar 18, 2021

cmsbuild commented Mar 18, 2021

cmsbuild commented Mar 18, 2021

chayanit commented Mar 18, 2021

smuzaffar Mar 18, 2021

justinasr Mar 18, 2021

smuzaffar Mar 26, 2021

justinasr Mar 26, 2021

smuzaffar Mar 26, 2021

cmsbuild commented Mar 18, 2021

srimanob commented Mar 26, 2021

smuzaffar commented Mar 26, 2021 •

edited

cmsbuild commented Mar 29, 2021

cmsbuild commented Mar 29, 2021

justinasr commented Mar 29, 2021

srimanob commented Mar 29, 2021

cmsbuild commented Mar 29, 2021

srimanob commented Mar 30, 2021

smuzaffar commented Mar 30, 2021

chayanit commented Mar 30, 2021

cmsbuild commented Mar 30, 2021

qliphy commented Mar 30, 2021

Change DBS url to cmsweb-prod in MatrixInjector #33209

Change DBS url to cmsweb-prod in MatrixInjector #33209

Conversation

justinasr commented Mar 18, 2021

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Mar 18, 2021

cmsbuild commented Mar 18, 2021

chayanit commented Mar 18, 2021

smuzaffar Mar 18, 2021

Choose a reason for hiding this comment

justinasr Mar 18, 2021

Choose a reason for hiding this comment

smuzaffar Mar 26, 2021

Choose a reason for hiding this comment

justinasr Mar 26, 2021

Choose a reason for hiding this comment

smuzaffar Mar 26, 2021

Choose a reason for hiding this comment

cmsbuild commented Mar 18, 2021

Comparison Summary

srimanob commented Mar 26, 2021

smuzaffar commented Mar 26, 2021 • edited

cmsbuild commented Mar 29, 2021

cmsbuild commented Mar 29, 2021

justinasr commented Mar 29, 2021

srimanob commented Mar 29, 2021

cmsbuild commented Mar 29, 2021

Comparison Summary

srimanob commented Mar 30, 2021

smuzaffar commented Mar 30, 2021

chayanit commented Mar 30, 2021

cmsbuild commented Mar 30, 2021

qliphy commented Mar 30, 2021

smuzaffar commented Mar 26, 2021 •

edited