Crash in wf 1325.5 caused by PR 22131

[fabiocos]

After the integration of #22131 a reproducible crash is observed in the IB in wf 1325.5, see

https://cmssdt.cern.ch/SDT/cgi-bin/buildlogs/slc6_amd64_gcc630/CMSSW_10_1_X_2018-02-14-0000/pyRelValMatrixLogs/run/1325.5_TTbar_13_reminiaodINPUT+TTbar_13_reminiaodINPUT+REMINIAOD_mc2016+HARVESTDR2_REMINIAOD_mc2016/step2_TTbar_13_reminiaodINPUT+TTbar_13_reminiaodINPUT+REMINIAOD_mc2016+HARVESTDR2_REMINIAOD_mc2016.log

@gpetruc could you please have a look?

[cmsbuild]

A new Issue was created by @fabiocos Fabio Cossutti.

@davidlange6, @Dr15Jones, @smuzaffar, @fabiocos can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[fabiocos]

@perrotta @slava77 @rekovic FYI

[gpetruc]

Uhm, apparently the MC event believes to be using prescale column 1 (read from the event), while the prescale table (retrieved from the event setup has only one column), so the thing crashes.
I can put a protection like in https://github.com/cms-sw/cmssw/blob/master/L1Trigger/L1TGlobal/src/L1TGlobalUtil.cc#L157-L164 but it would be better to understand upstream why this is happening.

[rekovic]

attn @apana

[slava77]

assign l1

[cmsbuild]

New categories assigned: l1

@thomreis,@nsmith-,@rekovic you have been requested to review this Pull request/Issue and eventually sign? Thanks

[slava77]

Notably, 1325.5 is "--era Run2_2016,run2_miniAOD_80XLegacy"
the input is a file made in 80X release.

If the column choice was different there and is a property of the 80X data, then the run2_miniAOD_80XLegacy modifier can be used to pick a more appropriate value.

[slava77]

assign reconstruction

[cmsbuild]

New categories assigned: reconstruction

@slava77,@perrotta you have been requested to review this Pull request/Issue and eventually sign? Thanks

[slava77]

@gpetruc
given that we were "happy" running on 1325.5 with the older setting of ReadPrescalesFromFile, perhaps you can modify
that value for run2_miniAOD_80XLegacy and we will have the wf running

[gpetruc]

On Wed, Feb 14, 2018 at 3:44 PM, Slava Krutelyov ***@***.***> wrote: @gpetruc <https://github.com/gpetruc> given that we were "happy" running on 1325.5 with the older setting of ReadPrescalesFromFile, perhaps you can modify that value for run2_miniAOD_80XLegacy and we will have the wf running

we were happy in the sense of "no crashes", but it's not clear to me if we were reading any sensible value for the L1 prescales. Giovanni

…

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#22207 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEbbR9Or88uS0Y0guh8EdOOwrB_X7lUCks5tUvFlgaJpZM4SE_PK> .

[slava77]

we were happy in the sense of "no crashes", but it's not clear to me if we were reading any sensible value for the L1 prescales.

right. If this lingers unanswered for more than a day, I'd rather we restore running in 1325.5 around Feb 15

[apana]

Hi Slava, Giovanni,

Sorry, I was offline most of the day and only now saw this issue. I'm looking into it and will report back later today.

[fabiocos]

@apana I agree with Slava, either this we have a solution by today, or better to revert the change until it is provided.

[apana]

Hi Fabio, I want to take a little time and make sure I fully investigate the situation. I also don't want to hold up the release cycle so please feel free to go ahead and revert the change. I'll try to come up with a proper fix in the next day or so.

…

-Len

On Thu, Feb 15, 2018 at 2:20 AM, Fabio Cossutti ***@***.***> wrote: @apana <https://github.com/apana> I agree with Slava, either this we have a solution by today, or better to revert the change until it is provided. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#22207 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABX7c_6Xu3ITywW_7W42Cmp_seP4Tm76ks5tU-i5gaJpZM4SE_PK> .

[gpetruc]

If this is the only failing workflow, I'll then temporarily reset the parameter to false in the run2_miniAOD_80XLegacy era.

Giovanni

[gpetruc]

done in #22239

[fabiocos]

@gpetruc @apana thanks, this is the only wf suffering a crash from the original PR by Giovanni, I think the best thing is finding a proper solution with calm. I will integrate the temporary solution by Giovanni, and when we have the fix from L1 we can just remove it.

[fabiocos]

@apana I understand #22314 is meant to fix this issue, right?

[apana]

Hi Fabio, Yes. BTW, we should also remove the temporary fix for this crash which was made in PR #22239 ( it sets ReadPrescalesFromFile to True in run2_miniAOD_80XLegacy workflows). I probably should have added that to my PR, but wasn't sure. Please let me know if you would like me to do that now.

…

-Len

On Fri, Feb 23, 2018 at 8:49 AM, Fabio Cossutti ***@***.***> wrote: @apana I understand #22314 is meant to fix this issue, right? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[fabiocos]

@apana @gpetruc I have just made #22338 to go back to the original proposal by Giovanni

[fabiocos]

The issue looks closed with the last PR merge.