Fixes related to beamspot DIP server for pilot run [12.0.X]

[sikler]

PR description:

• fixing warning related to missing log4cplus.properties file
• solved by performing a local copy to the working directory

[cmsbuild]

A new Pull Request was created by @sikler (Ferenc Siklr) for CMSSW_12_0_X.

It involves the following packages:

• DQM/BeamMonitor (dqm)
• DQM/Integration (dqm)

@emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @pmandrik, @pbo0, @rvenditti can you please review it and eventually sign? Thanks.
@battibass, @threus, @batinkov, @francescobrivio this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

• Backported from [12.1.X] Fixes related to beamspot DIP server for pilot run #35780

[tvami]

@cmsbuild , please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-bec523/19709/summary.html
COMMIT: b103a86
CMSSW: CMSSW_12_0_X_2021-10-18-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/35714/19709/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 39
• DQMHistoTests: Total histograms compared: 2998564
• DQMHistoTests: Total failures: 0
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2998542
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
• Checked 165 log files, 37 edm output root files, 39 DQM output files
• TriggerResults: no differences found

[tvami]

Hi @sikler I think it will be also requested to have a version of this in the master too.

[pmandrik]

Hello, @sikler ,
please add following line somewhere at the end of your client config in order to enable creation of DQM monitoring files :
process.DQMMonitoringService = cms.Service("DQMMonitoringService")

[pmandrik]

Also, with this PR we have a following connection error:

10/18/21 19:23:28 [139988269217088] ERROR system.dimerror %% - DIM message: Server Connecting to DIM_DNS@DIPNSDEV.cern.ch: Connection timed out

[cmsbuild]

Pull request #35714 was updated. @emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @pmandrik, @pbo0, @rvenditti can you please check and sign again.

[sikler]

@pmandrik requested line added in the client config. On the "Connection timed out" problem I have contacted dip-support@cern.ch. The issue is that servers dipns1.cern.ch and dipns2.cern.ch are not visible from the technical network (more precisely, from fu-c2f11-11-03). https://its.cern.ch/jira/browse/CMSONS-13415

[pmandrik]

@pmandrik requested line added in the client config. On the "Connection timed out" problem I have contacted dip-support@cern.ch. The issue is that servers dipns1.cern.ch and dipns2.cern.ch are not visible from the technical network (more precisely, from fu-c2f11-11-03).

do we need to solve this?

[sikler]

@pmandrik requested line added in the client config. On the "Connection timed out" problem I have contacted dip-support@cern.ch. The issue is that servers dipns1.cern.ch and dipns2.cern.ch are not visible from the technical network (more precisely, from fu-c2f11-11-03).

do we need to solve this?

Yes, asap.

[pmandrik]

(just would like to put here the link to the JIRA ticket not to forget https://its.cern.ch/jira/browse/CMSONS-13415)

[tvami]

@cmsbuild , please test

[francescobrivio]

From the JIRA it seems that the DIP servers are now accessible.
Should we schedule another try with the DIP client?
@pmandrik @ahmad3213 maybe a playback?

[sikler]

I am sorting out some subscription issues with NXCALS expert (Piotr Sowinski). Some cryptic message appears "Access point 'DIP.CMS/LuminousRegion' doesn't support selectors, but provided selector 'LHC.USER.ALL' is not empty." in ccde
(attached). Likely solution is to simply remove the selector (why is it there in the first place?), but need access for that.

Our data (dip/CMS/LHC/LuminousRegion), advertised in the DIP system, are supposed to be displayed in timber.cern.ch under the "LHC > Data from Experiments > CMS > Luminous Region" hierarchy.

[jfernan2]

+1
Tested in Online DQM at P5

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[jfernan2]

@sikler could we have a forwardport to master to keep releases in sync? Thanks!

[sikler]

@sikler could we have a forwardport to master to keep releases in sync? Thanks!
Yes, very soon. I am still waiting for the solution of the NXCALS/TIMBER issue.

[francescobrivio]

+1 Tested in Online DQM at P5

That's great @jfernan2!
Could you kindly provide the output/log of the test? Just for my education!

@sikler if you prefer I can prepare the forward port to master, just let me know!

[sikler]

@francescobrivio If you could help out with the forward port to master, that would be great!

[francescobrivio]

@francescobrivio If you could help out with the forward port to master, that would be great!

ok sure I will open the forward port in few minutes!

[pmandrik]

+1 Tested in Online DQM at P5

That's great @jfernan2! Could you kindly provide the output/log of the test? Just for my education!

@sikler if you prefer I can prepare the forward port to master, just let me know!

Hello, @francescobrivio , please find following logs:
DIP_content_parser_playbackPARSER_run502088PARSER_job2.log

[sikler]

Hello, @francescobrivio , please find following logs: DIP_content_parser_playbackPARSER_run502088PARSER_job2.log

Hi, how can I force the client to be (much) more verbose? There are couple of LogInfo messages in the code. Need to include MessageLogger?

[francescobrivio]

@sikler is the DIP client now in "production" mode? i.e. redy for publishing the values in DIP or still in "testing" mode?
In my opinion we should be in production mode now, what do you think @gennai @ggovi ?

[sikler]

@sikler is the DIP client now in "production" mode? i.e. redy for publishing the values in DIP or still in "testing" mode? In my opinion we should be in production mode now, what do you think @gennai @ggovi ?

From our side, it should be in production. The remaining issue is at the NXCALS level.

[francescobrivio]

From our side, it should be in production. The remaining issue is at the NXCALS level.

Sorry @sikler I don't know what NXCALS level is, could you clarify? Does it have effect on the pubblication?

Also, are you planning other changes to this PR? If yes I will wait before creating the forward port to master, if not I can open the forward in few minutes.

[sikler]

Sorry @sikler I don't know what NXCALS level is, could you clarify? Does it have effect on the pubblication?

NXCALS is the system that collects DIP info on the other side (not our business) and pushes that to TIMBER.

Also, are you planning other changes to this PR? If yes I will wait before creating the forward port to master, if not I can open the forward in few minutes.

If you have a good tip on increasing verbosity...

[francescobrivio]

backport of #35780

[francescobrivio]

@sikler I don't have a good tip for increasing the verbosity at the moment. I think we can put it on our to do list for later.

Could you also change the title of this PR to include [12.0.X]?

[perrotta]

+1

• Successfully tested in online DQM at P5
• Corresponding master PR merged since CMSSW_12_1_X_2021-10-22-2300, with no issues