Fixes for integration tests with partial pileup features

[vkuznet]

Fixes #11736

Status

ready

During integration tests we found that written JSON contains empty structure like {'mc': {}}. During debugging process we found two issues:

• updateBlockInfo may provide empty json dict which is passed to json config file
• getBlockInfo4PU from WMCore/Services/DBS/DBSConcurrency.py module contains hard-coded cmsweb production dbs URL which may not provide correct content when we perform tests on testbed due to different databases used in production and testbed DBS.

This PR fixes both issues. In addition, it adds proper configuration to

1. WMAgent.production, and WMAgent.testbed files
2. update the default config WMAgentConfig.py file

Is it backward compatible (if not, which system it affects?)

YES

Related PRs

Here are additional changes applied to other repositories which should be applied together with this PR:

1. to eployment repo: Add DBS3_READER_URL deployment#1313
2. to CMSKubernetes repo: Add dbs3_reader_url option CMSKubernetes#1465

I made this new PR in favor of #11969 which was damaged during rebase procedure and changes were not propagated back to PR.

External dependencies / deployment changes

[cmsdmwmbot]

Jenkins results:

• Python3 Unit tests: succeeded
  • 1 changes in unstable tests
• Python3 Pylint check: failed
  • 5 warnings and errors that must be fixed
  • 4 warnings
  • 53 comments to review
• Pylint py3k check: succeeded
• Pycodestyle check: succeeded
  • 7 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/15026/artifact/artifacts/PullRequestReport.html

[amaltaro]

It looks good to me! Thank you for keeping test and src changes separated!