Providing CMSSW version info to ElasticSearch

[sciaba]

Doing analysis in ElasticSearch it is very useful to have the CMSSW version that was used by the job. I'd like to ask that this information is made available for exporting to ElasticSearch.

[ticoann]

@amaltaro, Alan which code is putting this information?

[amaltaro]

I think this is part of the htcondor monitoring. Is it enough to add a new job classad or we have to do something else @sciaba ? If so, then it has to be a comma separated string (or a list...) since there are jobs running multiple CMSSW releases.

[sciaba]

Good question. A list of releases would be good but one would need to know something about what CMSSW did each time it ran. Let me discuss it with Brian at the earliest opportunity.

[hufnagel]

Seems like an oversight that this is not already available from CMSSW via a ChirpCMSSW_cmsRunN_Param fields. I would open a CMSSW ticket for it.

Otherwise please don' add more classadds, just chirp it directly from the CMSSW runtime executor, that knows what CMSSW version it's executing.

That way you get per step information only, but it's easy enough to search for that as long as you know the naming conventions and per step CMSSW version will be needed anyways at some point.

[sciaba]

That would be perfect.

[amaltaro]

@bbockelm as we discussed yesterday. What WMAgent has to provide is a list of CMSSW releases in the job classad, correct?

[hufnagel]

Isn't it more important that CMSSW provides this information so you can match per step parameters to the CMSSW version ?

[bbockelm]

Hi Dirk,

To first order, it is a bit of a toss-up whether done in the agent or CMSSW. I think the agent is the right place though because:

• If done in CMSSW, we'd only get the information for future releases. Implementing in agent would work today (already done in CRAB).
• If done by the agent, we'd get this information prior to job starting up (someday could be used for matchmaking in condor) or for steps that didn't run (because of job failure in an early step).

I suppose there's a reasonably good argument that we should do it in both places (think: CMSSW runs in other places, not just WMAgent). But I think it really should be in the agent for Andrea's use case.

Brian

[bbockelm]

@amaltaro - can we pull this one back up from the dead? Is it relatively simple to fix?

[amaltaro]

It should be fairly simple to implement it. We just have to decide how to publish this information for multi-steps jobs (I assume a comma separated ordered string), e.g.:

• step1 on 911, step2 on 911 and step3 on 900 --> "CMSSW_9_1_1,CMSSW_9_1_1,CMSSW_9_0_0"
• step1 on 800 and step2 on 800 --> "CMSSW_8_0_0,CMSSW_8_0_0"

Does it sound good? How about the scramArchs? Do we actually care about them?

[sciaba]

How can one know the nature of the steps? Is it always obvious what stepN is? If There is no way to know what CMSSW was doing on a particular step, the information is almost useless.

[amaltaro]

HTCondor classads don't carry information step-wise, instead they contain information about a job/task as a whole. You'd probably need to correlate a job information with the ReqMgr request information, unless CMSSW provides some of that meta data already via condor_chirp...

[sciaba]

I hope it does, and if not it should (and the same for the CMSSW version as well, as Dirk had suggested). Correlating information from different sources is often unwieldy.
In general we should find a sound solution for expressing the information about the steps, considering that multi-step jobs are more and more common. This goes beyond the scope of this ticket, though.

[bbockelm]

@amaltaro - we don't really care about the SCRAM arch because we get the runtime OS information from other sources.

Nowhere in CMS do we define what a DIGI job is: that's been a long-standing problem in all our monitoring, but we can't really fix this in this ticket.