Update SonicTriton client to Triton version 2.3 #31715

kpedro88 · 2020-10-08T15:44:42Z

PR description:

Triton version 2 redesigns the client API. Most of the redesign is just reorganization and renaming. Important Triton changes:

client-side timeouts now supported
https/ssl now supported (not introduced here, support will come in a followup)
batch size handling changed: models that support triton-style batching have the specified shape modified so that the first entry is the batch size. The handling of shapes in SonicTriton is modified so that the user does not have to worry about this, and can continue to consider batch size and input shape separately, which is conceptually easier.

SonicTriton interface changes:

modelVersion = cms.string() (was cms.int32())
setShape() accessors to change one dimension or all dimensions (rather than non-const ref shape() accessor)
unneeded dims() accessor removed
some other accessors made private (only used by TritonClient)

The cvmfs server image for tests is updated accordingly (see lgray/triton-torchgeo-gat-example#1).

The corresponding external update is cms-sw/cmsdist#6293.

PR validation:

Ran examples in HeterogeneousCore/SonicTriton/test.

cmsbuild · 2020-10-08T15:45:07Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-10-08T15:49:22Z

-code-checks

ERROR: Build errors found during clang-tidy run.

HeterogeneousCore/SonicTriton/interface/triton_utils.h:11:10: error: 'grpc_client.h' file not found [clang-diagnostic-error]
#include "grpc_client.h"
         ^
Suppressed 177 warnings (176 in non-user code, 1 with check filters).
--
HeterogeneousCore/SonicTriton/interface/TritonData.h:15:10: error: 'grpc_client.h' file not found [clang-diagnostic-error]
#include "grpc_client.h"
         ^
Suppressed 518 warnings (517 in non-user code, 1 with check filters).
--
HeterogeneousCore/SonicTriton/interface/TritonData.h:15:10: error: 'grpc_client.h' file not found [clang-diagnostic-error]
#include "grpc_client.h"
         ^
Suppressed 489 warnings (488 in non-user code, 1 with check filters).
--
gmake: *** [config/SCRAM/GMake/Makefile.coderules:128: code-checks] Error 2
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

cmsbuild · 2020-10-08T15:52:29Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmsdist#6293

CMSSW_11_2_X_2020-10-08-1100/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/9813/console Started: 2020/10/08 17:55

kpedro88 · 2020-10-08T15:52:34Z

@smuzaffar @mrodozov what is the right approach to make the code checks work with new headers from an external update?

cmsbuild · 2020-10-08T16:16:24Z

-1

Tested at: feb3396

Build:

I found compilation error when building:


-- Configuring incomplete, errors occurred!
See also "/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc820/external/triton-inference-server/2.3.0-db7bb5/build/CMakeFiles/CMakeOutput.log".
See also "/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc820/external/triton-inference-server/2.3.0-db7bb5/build/CMakeFiles/CMakeError.log".
error: Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.03s2e6 (%build)


RPM build errors:
Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.03s2e6 (%build)

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-db7bb5/9813/summary.html

cmsbuild · 2020-10-08T18:00:38Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmsdist#6293

CMSSW_11_2_X_2020-10-08-1100/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/9819/console Started: 2020/10/08 20:01

cmsbuild · 2020-10-08T19:41:51Z

+1
Tested at: feb3396
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-db7bb5/9819/summary.html
CMSSW: CMSSW_11_2_X_2020-10-08-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-10-08T19:41:56Z

Comparison job queued.

cmsbuild · 2020-10-13T20:43:16Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-10-13T20:48:35Z

-code-checks

ERROR: Build errors found during clang-tidy run.

HeterogeneousCore/SonicTriton/interface/triton_utils.h:11:10: error: 'grpc_client.h' file not found [clang-diagnostic-error]
#include "grpc_client.h"
         ^
Suppressed 177 warnings (176 in non-user code, 1 with check filters).
--
HeterogeneousCore/SonicTriton/interface/TritonData.h:15:10: error: 'grpc_client.h' file not found [clang-diagnostic-error]
#include "grpc_client.h"
         ^
Suppressed 518 warnings (517 in non-user code, 1 with check filters).
--
HeterogeneousCore/SonicTriton/interface/TritonData.h:15:10: error: 'grpc_client.h' file not found [clang-diagnostic-error]
#include "grpc_client.h"
         ^
Suppressed 489 warnings (488 in non-user code, 1 with check filters).
--
gmake: *** [config/SCRAM/GMake/Makefile.coderules:128: code-checks] Error 2
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 2

kpedro88 · 2020-10-13T20:50:33Z

@makortel thanks for the suggestion, my latest commit also includes code-checks and code-format

cmsbuild · 2020-10-13T20:51:01Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmsdist#6297

CMSSW_11_2_X_2020-10-13-1100/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/9923/console Started: 2020/10/13 22:53

cmsbuild · 2020-10-13T22:33:25Z

+1
Tested at: 96a3519
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-452733/9923/summary.html
CMSSW: CMSSW_11_2_X_2020-10-13-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-10-13T22:33:30Z

Comparison job queued.

cmsbuild · 2020-10-14T02:18:40Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-452733/9923/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2543692
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2543667
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
Checked 149 log files, 22 edm output root files, 35 DQM output files

makortel · 2020-10-14T15:45:16Z

+1

kpedro88 · 2020-10-14T17:14:27Z

@silviodonato @qliphy this will need to be force-merged (known problem: bot can't run code checks if CMSSW code uses new headers from an external PR)

please make sure to merge this together with cms-sw/cmsdist#6297

thanks

silviodonato · 2020-10-14T18:24:29Z

merge

kpedro88 added 4 commits October 7, 2020 09:48

update to v2 API and corresponding server version

794db42

change in batch size handling, simplify public interface

a04ed4a

fix bugs

cb57f60

code format

feb3396

cmsbuild added this to the CMSSW_11_2_X milestone Oct 8, 2020

cmsbuild added code-checks-pending comparison-pending heterogeneous-pending orp-pending pending-signatures tests-pending labels Oct 8, 2020

This was referenced Oct 8, 2020

Triton v2.3 cms-sw/cmsdist#6293

Merged

Update SonicTriton to Triton 2.X and v2 API fastmachinelearning/SonicCMS#10

Closed

cmsbuild added code-checks-rejected and removed code-checks-pending labels Oct 8, 2020

cmsbuild added requires-external tests-started and removed tests-pending labels Oct 8, 2020

cmsbuild added tests-rejected and removed tests-started labels Oct 8, 2020

cmsbuild added tests-started and removed tests-rejected labels Oct 8, 2020

cmsbuild added code-checks-rejected and removed code-checks-pending labels Oct 13, 2020

cmsbuild added requires-external tests-started and removed tests-pending labels Oct 13, 2020

cmsbuild added tests-approved and removed tests-started labels Oct 13, 2020

cmsbuild added comparison-available and removed comparison-pending labels Oct 14, 2020

makortel mentioned this pull request Oct 14, 2020

Random DQM differences in PR tests #31644

Closed

cmsbuild added fully-signed heterogeneous-approved and removed heterogeneous-pending pending-signatures labels Oct 14, 2020

cmsbuild added orp-approved and removed orp-pending labels Oct 14, 2020

silviodonato merged commit b95457f into cms-sw:master Oct 14, 2020

cmsbuild mentioned this pull request Oct 15, 2020

Add ParticleNetAK4 to JME custom NanoAOD #31714

Merged

mrodozov mentioned this pull request Oct 15, 2020

Add low-pT electrons to MINIAOD, update ID, improve end user experience #31220

Merged

makortel mentioned this pull request Oct 15, 2020

SonicTriton package does not build in slc7_ppc64le #31807

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update SonicTriton client to Triton version 2.3 #31715

Update SonicTriton client to Triton version 2.3 #31715

kpedro88 commented Oct 8, 2020

cmsbuild commented Oct 8, 2020

cmsbuild commented Oct 8, 2020

cmsbuild commented Oct 8, 2020 •

edited

kpedro88 commented Oct 8, 2020

cmsbuild commented Oct 8, 2020

cmsbuild commented Oct 8, 2020 •

edited

cmsbuild commented Oct 8, 2020

cmsbuild commented Oct 8, 2020

cmsbuild commented Oct 13, 2020

cmsbuild commented Oct 13, 2020

kpedro88 commented Oct 13, 2020

cmsbuild commented Oct 13, 2020 •

edited

cmsbuild commented Oct 13, 2020

cmsbuild commented Oct 13, 2020

cmsbuild commented Oct 14, 2020

makortel commented Oct 14, 2020

kpedro88 commented Oct 14, 2020

silviodonato commented Oct 14, 2020

Update SonicTriton client to Triton version 2.3 #31715

Update SonicTriton client to Triton version 2.3 #31715

Conversation

kpedro88 commented Oct 8, 2020

PR description:

PR validation:

cmsbuild commented Oct 8, 2020

cmsbuild commented Oct 8, 2020

cmsbuild commented Oct 8, 2020 • edited

kpedro88 commented Oct 8, 2020

cmsbuild commented Oct 8, 2020

cmsbuild commented Oct 8, 2020 • edited

cmsbuild commented Oct 8, 2020

cmsbuild commented Oct 8, 2020

cmsbuild commented Oct 13, 2020

cmsbuild commented Oct 13, 2020

kpedro88 commented Oct 13, 2020

cmsbuild commented Oct 13, 2020 • edited

cmsbuild commented Oct 13, 2020

cmsbuild commented Oct 13, 2020

cmsbuild commented Oct 14, 2020

makortel commented Oct 14, 2020

kpedro88 commented Oct 14, 2020

silviodonato commented Oct 14, 2020

cmsbuild commented Oct 8, 2020 •

edited

cmsbuild commented Oct 8, 2020 •

edited

cmsbuild commented Oct 13, 2020 •

edited