Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how to limit OnnxRuntime cpu usage? #3166

Open
leepengcheng opened this issue May 8, 2024 · 1 comment
Open

how to limit OnnxRuntime cpu usage? #3166

leepengcheng opened this issue May 8, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@leepengcheng
Copy link

Description

I have set interOpNumThreads and intraOpNumThreads option,but the cpu usage is 3200%, it's to high.
so how to reduce the cpu usage ,such as 400%.

Code

    val criteria = Criteria.builder
      .optApplication(Application.NLP.TEXT_CLASSIFICATION)
      .optModelUrls(model_path)
      .setTypes(classOf[Longs], classOf[Floats])
      .optTranslator(new BertTranslator())
      .optEngine("OnnxRuntime")
      .optOption("interOpNumThreads", "1")
      .optOption("intraOpNumThreads", "1")
      .optOption("executionMode", "SEQUENTIAL")
      .optOption("optLevel", "NO_OPT")
      .optOption("memoryPatternOptimization", "true")
      .optOption("cpuArenaAllocator", "false")
      .optOption("disablePerSessionThreads", "true")
      .build

Cpu

image

@leepengcheng leepengcheng added the enhancement New feature or request label May 8, 2024
@frankfliu
Copy link
Contributor

If you send NUM_OMP_THREADS=1, it should only use 1 CPU at a time. How many java thread are you using for inference?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants