[Question] trtexec with multi streams

[tp-nan]

Hi,
I built an engine with dynamic batching input, and num_optimization_profiles == 1,

but trtexec support streams=2. How it happend, as multi-contexts can not use the same profile at the same time? Did I miss something?

[ttyio]

Hello @ShiyangZhang , yes you are right, when dynamic shape is enabled, each profile can only assign to single context.
I just check trtexec:

--streams=N Instantiate N engines to use concurrently

So we create 2 engines in your case. thanks

[tp-nan]

thx

[yuekaizhang]

Hi, @ttyio I was wondering if trtexec support --streams for dynamic shape or not? I encounter the same issue #1341 with trt8.4 EA.

[nvpohanh]

@yuekaizhang trtexec does not support multiple optimization profiles yet