Patch MXNet to use a thread-local engine #4207

hqucms · 2018-07-22T20:14:30Z

The original NaiveEngine in MXNet is not thread safe. Here we change the engine to be a thread local object, so it will allow each thread to use a different engine:
hqucms/incubator-mxnet@f31352f

cmsbuild · 2018-07-22T20:14:59Z

A new Pull Request was created by @hqucms (Huilin Qu) for branch IB/CMSSW_10_3_X/gcc700.

@cmsbuild, @smuzaffar, @gudrutis, @mrodozov can you please review it and eventually sign? Thanks.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.

slava77 · 2018-07-23T00:00:56Z

@cmsbuild please test

for a test by itself

cmsbuild · 2018-07-23T00:01:09Z

The tests are being triggered in jenkins.

slava77 · 2018-07-23T00:01:36Z

instead of pulling from https://hqu.web.cern.ch/hqu/tools/mxnet, shouldn't we use a CMS/SW repo?

cmsbuild · 2018-07-23T00:03:12Z

The tests are being triggered in jenkins.
Using cmssw from cms-sw/cmssw#23768
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/29301/console

cmsbuild · 2018-07-23T05:31:09Z

+1
Tested at: c375c23
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4207/29301/summary.html

cmsbuild · 2018-07-23T05:31:11Z

Comparison job queued.

cmsbuild · 2018-07-23T07:05:47Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4207/29301/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 31
DQMHistoTests: Total histograms compared: 2889291
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2889100
DQMHistoTests: Total skipped: 190
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 30 files compared)
Checked 128 log files, 14 edm output root files, 31 DQM output files

smuzaffar · 2018-07-23T07:43:12Z

+externals

cmsbuild · 2018-07-23T07:43:59Z

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_10_3_X/gcc700 IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

cmsbuild · 2018-07-24T20:12:57Z

The tests are being triggered in jenkins.
Using cmssw from cms-sw/cmssw#23768
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/29429/console

cmsbuild · 2018-07-25T02:53:36Z

+1
Tested at: c375c23
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23768/29429/summary.html

cmsbuild · 2018-07-25T02:53:40Z

Comparison job queued.

Use cms-externals repo. The branch 1.2.1.mod3 is orphan with the content of https://hqu.web.cern.ch/hqu/tools/mxnet/apache-mxnet-src-1.2.1.mod3-incubating.tar.gz as the repo contains git submodules which cmsBuild is not supporting.

cmsbuild · 2018-08-14T14:34:21Z

Pull request #4207 was updated.

cmsbuild · 2018-08-14T14:40:16Z

The tests are being triggered in jenkins.
Using cmssw from cms-sw/cmssw#23768
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/29813/console

cmsbuild · 2018-08-14T14:57:26Z

-1

Tested at: f9818e2

Build:

I found an error when building:

+ tar -xf -
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ cd %releasename
/build/cmsbld/jenkins/workspace/ib-any-integration/testBuildDir/tmp/rpm-tmp.hMVp5E: line 39: cd: %releasename: No such file or directory
error: Bad exit status from /build/cmsbld/jenkins/workspace/ib-any-integration/testBuildDir/tmp/rpm-tmp.hMVp5E (%prep)


RPM build errors:
Bad exit status from /build/cmsbld/jenkins/workspace/ib-any-integration/testBuildDir/tmp/rpm-tmp.hMVp5E (%prep)

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4207/29813/summary.html

cmsbuild · 2018-08-14T15:02:00Z

Pull request #4207 was updated.

cmsbuild · 2018-08-14T15:02:12Z

The tests are being triggered in jenkins.
Using cmssw from cms-sw/cmssw#23768
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/29816/console

cmsbuild · 2018-08-14T19:34:31Z

+1
Tested at: 9a17eb4
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23768/29816/summary.html

cmsbuild · 2018-08-14T19:34:37Z

Comparison job queued.

davidlange6 · 2018-08-14T21:51:21Z

unhold

davidlange6 · 2018-08-14T22:15:03Z

hi @slava77 - this PR does not require cms-sw/cmssw/#23768, right? (eg, only the reverse is true)

…

On Aug 14, 2018, at 4:51 PM, David Lange ***@***.***> wrote: unhold — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

slava77 · 2018-08-14T22:55:25Z

On 8/14/18 3:15 PM, David Lange wrote: hi @slava77 - this PR does not require cms-sw/cmssw/#23768, right? (eg, only the reverse is true)

right

davidlange6 · 2018-08-14T22:57:00Z

merge

…

On Aug 14, 2018, at 5:55 PM, Slava Krutelyov ***@***.***> wrote: On 8/14/18 3:15 PM, David Lange wrote: > hi @slava77 - this PR does not require cms-sw/cmssw/#23768, right? (eg, > only the reverse is true) right — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

Patch MXNet to use a thread local engine.

c375c23

cmsbuild added externals-pending orp-pending pending-signatures tests-pending labels Jul 22, 2018

hqucms mentioned this pull request Jul 22, 2018

DeepAK8 tagger integration cms-sw/cmssw#23768

Merged

cmsbuild added tests-started and removed tests-pending labels Jul 23, 2018

cmsbuild added tests-approved and removed tests-started labels Jul 23, 2018

cmsbuild added externals-approved fully-signed and removed externals-pending pending-signatures labels Jul 23, 2018

cmsbuild added tests-started and removed tests-approved labels Jul 24, 2018

cmsbuild removed the tests-started label Jul 25, 2018

Update mxnet-predict.spec

f9818e2

Use cms-externals repo. The branch 1.2.1.mod3 is orphan with the content of https://hqu.web.cern.ch/hqu/tools/mxnet/apache-mxnet-src-1.2.1.mod3-incubating.tar.gz as the repo contains git submodules which cmsBuild is not supporting.

cmsbuild added tests-started and removed tests-pending labels Aug 14, 2018

cmsbuild added tests-rejected and removed tests-started labels Aug 14, 2018

Update mxnet-predict.spec

9a17eb4

cmsbuild added tests-pending and removed tests-rejected labels Aug 14, 2018

cmsbuild added tests-started and removed tests-pending labels Aug 14, 2018

cmsbuild added tests-approved and removed tests-started labels Aug 14, 2018

cmsbuild removed the hold label Aug 14, 2018

cmsbuild added orp-approved and removed orp-pending labels Aug 14, 2018

cmsbuild merged commit 24fc83e into cms-sw:IB/CMSSW_10_3_X/gcc700 Aug 14, 2018

This was referenced Sep 11, 2018

Backport MXNet and DeepBoostedJet models. #4326

Merged

Backport MXNet and DeepBoostedJet models to 94X. #4327

Merged

hqucms mentioned this pull request Aug 13, 2019

Update MXNet to 1.5.0 #5148

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Patch MXNet to use a thread-local engine #4207

Patch MXNet to use a thread-local engine #4207

hqucms commented Jul 22, 2018

cmsbuild commented Jul 22, 2018

slava77 commented Jul 23, 2018

cmsbuild commented Jul 23, 2018

slava77 commented Jul 23, 2018

cmsbuild commented Jul 23, 2018 •

edited

cmsbuild commented Jul 23, 2018

cmsbuild commented Jul 23, 2018

cmsbuild commented Jul 23, 2018

smuzaffar commented Jul 23, 2018

cmsbuild commented Jul 23, 2018

cmsbuild commented Jul 24, 2018 •

edited

cmsbuild commented Jul 25, 2018

cmsbuild commented Jul 25, 2018

cmsbuild commented Aug 14, 2018

cmsbuild commented Aug 14, 2018 •

edited

cmsbuild commented Aug 14, 2018

cmsbuild commented Aug 14, 2018

cmsbuild commented Aug 14, 2018 •

edited

cmsbuild commented Aug 14, 2018

cmsbuild commented Aug 14, 2018

davidlange6 commented Aug 14, 2018

davidlange6 commented Aug 14, 2018 via email

slava77 commented Aug 14, 2018 via email

davidlange6 commented Aug 14, 2018 via email

Patch MXNet to use a thread-local engine #4207

Patch MXNet to use a thread-local engine #4207

Conversation

hqucms commented Jul 22, 2018

cmsbuild commented Jul 22, 2018

slava77 commented Jul 23, 2018

cmsbuild commented Jul 23, 2018

slava77 commented Jul 23, 2018

cmsbuild commented Jul 23, 2018 • edited

cmsbuild commented Jul 23, 2018

cmsbuild commented Jul 23, 2018

cmsbuild commented Jul 23, 2018

smuzaffar commented Jul 23, 2018

cmsbuild commented Jul 23, 2018

cmsbuild commented Jul 24, 2018 • edited

cmsbuild commented Jul 25, 2018

cmsbuild commented Jul 25, 2018

cmsbuild commented Aug 14, 2018

cmsbuild commented Aug 14, 2018 • edited

cmsbuild commented Aug 14, 2018

cmsbuild commented Aug 14, 2018

cmsbuild commented Aug 14, 2018 • edited

cmsbuild commented Aug 14, 2018

cmsbuild commented Aug 14, 2018

davidlange6 commented Aug 14, 2018

davidlange6 commented Aug 14, 2018 via email

slava77 commented Aug 14, 2018 via email

davidlange6 commented Aug 14, 2018 via email

cmsbuild commented Jul 23, 2018 •

edited

cmsbuild commented Jul 24, 2018 •

edited

cmsbuild commented Aug 14, 2018 •

edited

cmsbuild commented Aug 14, 2018 •

edited