Add onnxruntime #5080

hqucms · 2019-07-08T13:29:28Z

@davidlange6 This builds only the C API -- but the python package might also be helpful if analyzers want to run some ML inference in their python-based analyses.

cmsbuild · 2019-07-08T13:29:56Z

A new Pull Request was created by @hqucms (Huilin Qu) for branch IB/CMSSW_11_0_X/gcc700.

@cmsbuild, @smuzaffar, @gudrutis, @mrodozov can you please review it and eventually sign? Thanks.
cms-bot commands are listed here

davidlange6 · 2019-07-08T13:37:40Z

please test

(the cmake update is indeed required - that will rebuild lots of things I suspect)

cmsbuild · 2019-07-08T13:38:06Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/1358/console Started: 2019/07/08 15:40

slava77 · 2019-07-08T13:54:23Z

onnxruntime.spec

+   -Donnxruntime_USE_FULL_PROTOBUF=OFF \
+   -Donnxruntime_DISABLE_CONTRIB_OPS=OFF \
+   -Donnxruntime_BUILD_UNIT_TESTS=OFF \
+   ../cmake


I do not see any specifics for the build architecture.
What does this default to? We still need to restrict to SSE4 (or is it SSE3?)

It seems that it supports dynamic architecture -- I built it once on a machine with AVX and it runs fine on another machine without it.

cmsbuild · 2019-07-08T16:11:00Z

-1

Tested at: 005f931

Build:

I found compilation error when building:

linked by target "igprof" in directory /build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc700/external/igprof/5.9.16-cms/igprof-4235186fd1a9b9adb86a8a45bb7a8ece74953039

-- Configuring incomplete, errors occurred!
See also "/build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc700/external/igprof/5.9.16-cms/igprof-4235186fd1a9b9adb86a8a45bb7a8ece74953039/CMakeFiles/CMakeOutput.log".
See also "/build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc700/external/igprof/5.9.16-cms/igprof-4235186fd1a9b9adb86a8a45bb7a8ece74953039/CMakeFiles/CMakeError.log".
error: Bad exit status from /build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.IFpsKn (%build)


RPM build errors:
Bad exit status from /build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.IFpsKn (%build)

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c31479/1358/summary.html

hqucms · 2019-07-09T13:51:30Z

I cannot reproduce the error -- igprof builds fine with the cmake update on my side.

smuzaffar · 2019-07-09T13:58:18Z

please test

cmsbuild · 2019-07-09T13:58:41Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/1371/console Started: 2019/07/09 16:00

smuzaffar · 2019-07-09T14:00:30Z

@hqucms , the build error is this. May be you have unwind installed on the system that is why cmake did not fail.

CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
UNWIND_INCLUDE_DIR

smuzaffar · 2019-07-09T15:32:36Z

During PR tests we create symlink for packages which are not changed. In this case we create symlink for unwwind. I think problem is here
https://github.com/cms-sw/cmsdist/blob/IB/CMSSW_11_0_X/gcc700/igprof.spec#L16
which just rsync symlink instead of contents of unwind and atoim_ops. I am looking in to this now.

cmsbuild · 2019-07-09T18:32:06Z

-1

Tested at: 005f931

Build:

I found compilation error when building:

linked by target "igprof" in directory /build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc700/external/igprof/5.9.16-cms/igprof-4235186fd1a9b9adb86a8a45bb7a8ece74953039

-- Configuring incomplete, errors occurred!
See also "/build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc700/external/igprof/5.9.16-cms/igprof-4235186fd1a9b9adb86a8a45bb7a8ece74953039/CMakeFiles/CMakeOutput.log".
See also "/build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc700/external/igprof/5.9.16-cms/igprof-4235186fd1a9b9adb86a8a45bb7a8ece74953039/CMakeFiles/CMakeError.log".
error: Bad exit status from /build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.G0vmFi (%build)


RPM build errors:
Bad exit status from /build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.G0vmFi (%build)

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c31479/1371/summary.html

hqucms · 2019-07-09T19:58:22Z

During PR tests we create symlink for packages which are not changed. In this case we create symlink for unwwind. I think problem is here
https://github.com/cms-sw/cmsdist/blob/IB/CMSSW_11_0_X/gcc700/igprof.spec#L16
which just rsync symlink instead of contents of unwind and atoim_ops. I am looking in to this now.

@smuzaffar Thank you for looking into this! Indeed when I built igprof, unwind was downloaded rather than symlinked, which is probably why it built successfully on my machine.

cmsbuild · 2019-07-10T05:54:06Z

Pull request #5080 was updated.

cmsbuild · 2019-08-08T00:40:22Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c31479/1882/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 33
DQMHistoTests: Total histograms compared: 2715989
DQMHistoTests: Total failures: 2
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2715658
DQMHistoTests: Total skipped: 329
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 32 files compared)
Checked 137 log files, 14 edm output root files, 33 DQM output files

smuzaffar · 2019-08-08T06:30:20Z

please test

cmsbuild · 2019-08-08T06:30:51Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/1885/console Started: 2019/08/08 08:32

cmsbuild · 2019-08-08T08:09:06Z

+1
Tested at: 60a3e86
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c31479/1885/summary.html

cmsbuild · 2019-08-08T08:09:09Z

Comparison job queued.

cmsbuild · 2019-08-08T10:17:10Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c31479/1885/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 33
DQMHistoTests: Total histograms compared: 2715989
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2715659
DQMHistoTests: Total skipped: 329
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 32 files compared)
Checked 137 log files, 14 edm output root files, 33 DQM output files

smuzaffar · 2019-08-08T10:30:12Z

+externals

cmsbuild · 2019-08-08T10:30:41Z

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_11_0_X/gcc700 IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

smuzaffar · 2019-08-09T06:48:52Z

@hqucms , this is now available in CMSSW_11_0_X IBs. Note that occxruntime currently does not support PowerPC so that is why it is not available for ppc64le SCRAM ARCH IBs.

hqucms · 2019-08-09T14:46:49Z

@smuzaffar Thank you very much! I will test occxruntime from the IBs and proceed from there.

Add onnxruntime.

005f931

cmsbuild added comparison-pending externals-pending orp-pending pending-signatures tests-pending labels Jul 8, 2019

cmsbuild added tests-started and removed tests-pending labels Jul 8, 2019

slava77 reviewed Jul 8, 2019

View reviewed changes

cmsbuild added tests-rejected and removed tests-started labels Jul 8, 2019

cmsbuild added tests-pending and removed tests-rejected labels Jul 9, 2019

cmsbuild added tests-started and removed tests-pending labels Jul 9, 2019

cmsbuild added tests-rejected and removed tests-started labels Jul 9, 2019

added onnxruntime to cmssw-tool-conf

0a1d082

cmsbuild added tests-approved and removed tests-started labels Aug 7, 2019

cmsbuild added comparison-available and removed comparison-pending labels Aug 8, 2019

cmsbuild added comparison-pending and removed comparison-available tests-approved labels Aug 8, 2019

cmsbuild added tests-pending tests-started and removed tests-pending labels Aug 8, 2019

cmsbuild added tests-approved and removed tests-started labels Aug 8, 2019

cmsbuild added comparison-available and removed comparison-pending labels Aug 8, 2019

smuzaffar merged commit 5a314f4 into cms-sw:IB/CMSSW_11_0_X/gcc700 Aug 8, 2019

cmsbuild added externals-approved fully-signed and removed externals-pending pending-signatures labels Aug 8, 2019

hqucms mentioned this pull request Sep 20, 2019

Accelerate ML inference with ONNX Runtime cms-sw/cmssw#27458

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add onnxruntime #5080

Add onnxruntime #5080

hqucms commented Jul 8, 2019

cmsbuild commented Jul 8, 2019

davidlange6 commented Jul 8, 2019

cmsbuild commented Jul 8, 2019 •

edited

slava77 Jul 8, 2019

hqucms Jul 8, 2019

cmsbuild commented Jul 8, 2019

hqucms commented Jul 9, 2019

smuzaffar commented Jul 9, 2019

cmsbuild commented Jul 9, 2019 •

edited

smuzaffar commented Jul 9, 2019

smuzaffar commented Jul 9, 2019

cmsbuild commented Jul 9, 2019

hqucms commented Jul 9, 2019

cmsbuild commented Jul 10, 2019

cmsbuild commented Aug 8, 2019

smuzaffar commented Aug 8, 2019

cmsbuild commented Aug 8, 2019 •

edited

cmsbuild commented Aug 8, 2019

cmsbuild commented Aug 8, 2019

cmsbuild commented Aug 8, 2019

smuzaffar commented Aug 8, 2019

cmsbuild commented Aug 8, 2019

smuzaffar commented Aug 9, 2019

hqucms commented Aug 9, 2019

Add onnxruntime #5080

Add onnxruntime #5080

Conversation

hqucms commented Jul 8, 2019

cmsbuild commented Jul 8, 2019

davidlange6 commented Jul 8, 2019

cmsbuild commented Jul 8, 2019 • edited

slava77 Jul 8, 2019

Choose a reason for hiding this comment

hqucms Jul 8, 2019

Choose a reason for hiding this comment

cmsbuild commented Jul 8, 2019

hqucms commented Jul 9, 2019

smuzaffar commented Jul 9, 2019

cmsbuild commented Jul 9, 2019 • edited

smuzaffar commented Jul 9, 2019

smuzaffar commented Jul 9, 2019

cmsbuild commented Jul 9, 2019

hqucms commented Jul 9, 2019

cmsbuild commented Jul 10, 2019

cmsbuild commented Aug 8, 2019

smuzaffar commented Aug 8, 2019

cmsbuild commented Aug 8, 2019 • edited

cmsbuild commented Aug 8, 2019

cmsbuild commented Aug 8, 2019

cmsbuild commented Aug 8, 2019

smuzaffar commented Aug 8, 2019

cmsbuild commented Aug 8, 2019

smuzaffar commented Aug 9, 2019

hqucms commented Aug 9, 2019

cmsbuild commented Jul 8, 2019 •

edited

cmsbuild commented Jul 9, 2019 •

edited

cmsbuild commented Aug 8, 2019 •

edited