java+libtorch version runs much slower than the python+torchscript version

[ares89]

First, I train a IDCNN+CRF model with torch 1.4.0,
Then,I transform it to a torchscript model.
Last, I use the torchscript model with java+libtorch.

In same machine(centos 7.4), use same torchscript model, java+libtorch version runs much slower than the python+torchscript version.
To predict almost 35000 samples, java+libtorch use 1250seconds, python+torchscript use only 300seconds.
Did I miss something？

[ares89]

And also,Java+libtorch has 90% cpu load and pytorch+torchscript use only 50% cpu load . CPU is Intel Xeon E5-2680v4 , 56 cores

[ares89]

https://github.com/NervanaSystems/nlp-architect/blob/7f71a16965355d2508e86d0e1d6726842bccb58e/nlp_architect/nn/torch/modules/embedders.py#L157
this is the IDCNN i use

[ares89]

And also, same mechine,

1. both Java+libtorch, IDCNN+CRF run slower than Bilstm+CRF.
2. both pytorch+torchscript, IDCNN+CRF run faster than Bilstm+CRF.

[ares89]

solution is add environment variables. set OMP_NUM_THREADS=2 or some value else will work better.
reference:
1.https://software.intel.com/content/www/us/en/develop/articles/maximize-tensorflow-performance-on-cpu-considerations-and-recommendations-for-inference.html
2.https://github.com/mingfeima/convnet-benchmarks/blob/master/pytorch/run.sh#L16-L25
3.pytorch/pytorch#20156