Training files for DNN-based Tau-Ids

[mbluj]

As title says: this PR adds initial versions of training files for DNN-based Tau-Ids. Related to cms-sw/cmssw#25016

[cmsbuild]

A new Pull Request was created by @mbluj for branch master.

@cmsbuild, @smuzaffar, @gudrutis, @mrodozov can you please review it and eventually sign? Thanks.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.

external issue cms-sw/cmsdist#4455

[slava77]

@mbluj
the size is pretty large.
Is it the model optimized/minimized for inference or is it the training model (which is usually much larger and contains extra data not needed for inference).

[mbluj]

@slava77, do you refer to a particular training file (the biggest one?) or all of them? Regardless, this question is beyond my knowledge, so I redirect it to authors of the discriminants: @kandrosov, @ocolegrove could you comment please?

[kandrosov]

@mbluj
the size is pretty large.
Is it the model optimized/minimized for inference or is it the training model (which is usually much larger and contains extra data not needed for inference).

@slava77 I can answer about DeepTauId/deepTau_2017v1_20L1024N.pb (the biggest file): yes, this is a model, which is optimized for inference.

[slava77]

@slava77, do you refer to a particular training file (the biggest one?) or all of them? Regardless, this question is beyond my knowledge, so I redirect it to authors of the discriminants: @kandrosov, @ocolegrove could you comment please?

I had in mind all of them (there are just 3 with somewhat similar size).

[ocolegrove]

Right now we can probably do with just one of the two graphs (for DeepPF), but the code as written does require both.

…

On Fri, Oct 26, 2018 at 4:14 PM Slava Krutelyov ***@***.***> wrote: @slava77 <https://github.com/slava77>, do you refer to a particular training file (the biggest one?) or all of them? Regardless, this question is beyond my knowledge, so I redirect it to authors of the discriminants: @kandrosov <https://github.com/kandrosov>, @ocolegrove <https://github.com/ocolegrove> could you comment please? I had in mind all of them (there are just 3 with somewhat similar size). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ANs5Z4F3JQHv3dvZ9m_DKKSQ5g0THcb5ks5uoxjAgaJpZM4X8N8S> .

[mbluj]

Hi Owen,

On Fri, Oct 26, 2018 at 4:16 PM ocolegrove ***@***.***> wrote: Right now we can probably do with just one of the two graphs (for DeepPF), but the code as written does require both.

yes, right. However the main question is about size of the files and in particular if the files contain a model optimized/minimized for inference or a training model (usually bigger).

[ocolegrove]

I will investigate if the model size could be reduced

…

On Fri, Oct 26, 2018 at 4:29 PM mbluj ***@***.***> wrote: Hi Owen, On Fri, Oct 26, 2018 at 4:16 PM ocolegrove ***@***.***> wrote: > Right now we can probably do with just one of the two graphs (for DeepPF), > but the code as written does require both. yes, right. However the main question is about size of the files and in particular if the files contain a model optimized/minimized for inference or a training model (usually bigger). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ANs5Z6e5WfE3cEYI0SLgjYzIcpiM1K5Kks5uoxxQgaJpZM4X8N8S> .

[slava77]

I will investigate if the model size could be reduced

please get in touch with BTV developers @riga @mverzett @pablodecm may be able to check quickly.

[riga]

Hi @ocolegrove ,
in case you want to convert your model to a constant graph which reduces the size quite a lot, I summarized the procedure here: https://gitlab.cern.ch/mrieger/CMSSW-DNN#constant-graphs

[ocolegrove]

Thanks Marcel! I will investigate this on Monday

…

On Fri, Oct 26, 2018 at 7:49 PM Marcel R. ***@***.***> wrote: Hi @ocolegrove <https://github.com/ocolegrove> , in case you want to convert your model to a constant graph which reduces the size quite a lot, I summarized the procedure here: https://gitlab.cern.ch/mrieger/CMSSW-DNN#constant-graphs — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ANs5Z2E6TOpQe87ad6cGmsLtZ-f8M58Cks5uo0sigaJpZM4X8N8S> .

[cmsbuild]

Pull request #1 was updated.

external issue cms-sw/cmsdist#4455

[mbluj]

The most recent update replaces original training files by their quantized versions with size reduced by a factor of ~4.

[fabiocos]

@mbluj @slava77 @perrotta I would say we should merge this PR and the corresponding cmsdist one

[perrotta]

@fabiocos These are the training files needed by #25016, and therefore they must be made available to it.

[fabiocos]

+externals

[fabiocos]

merge

[smuzaffar]

cmsdist PR cms-sw/cmsdist#4554 includes this data package.

[mbluj]

Hello,
maybe it is obvious, but it will be good to have these files also in new 102X (and 94X?) releases where the DNN tau-Ids are backported. Is it considered?