Custom Build Python Backend Locale Error

[s402621342]

Description
I custom build a python backend, but when using the open function to open a file, the encoding is set to ascii while the LC_ALL is C.UTF-8，and i try to print(_locale.nl_langinfo(_locale.CODESET)), it returns ANSI_X3.4-1968, but when i directly use python3 and print , it returns UTF-8

Triton Information
What version of Triton are you using?

r22.12

Are you using the Triton container or did you build it yourself?

using the Triton container , but when i build it myself, i face the same problem

To Reproduce
Steps to reproduce the behavior.

docker run  it --rm --net=host -v ${PWD}/model_repository:/models nvcr.io/nvidia/tritonserver:22.12-py3 bash 

apt-get update && apt-get install  -y rapidjson-dev libarchive-dev zlib1g-dev

cd /root
wget https://github.com/Kitware/CMake/releases/download/v3.24.3/cmake-3.24.3-linux-x86_64.tar.gz
tar -zvxf cmake-3.24.3-linux-x86_64.tar.gz
ln -sf /root/cmake-3.24.3-linux-x86_64/bin/* /usr/bin/




git clone https://github.com/triton-inference-server/python_backend -b r22.12
cd python_backend
mkdir build && cd build
cmake -DTRITON_ENABLE_GPU=ON -DTRITON_BACKEND_REPO_TAG=r22.12 -DTRITON_COMMON_REPO_TAG=r22.12 -DTRITON_CORE_REPO_TAG=r22.12 -DCMAKE_INSTALL_PREFIX:PATH=`pwd`/install ..
make triton-python-backend-stub


export LC_ALL=C.UTF-8

tritonserver --model-repository=/models


cp triton_python_backend_stub /models/model_a

before cp triton_python_backend_stub, it returns UTF-8 ,but when i change the triton_python_backend_stub it returns ANSI_X3.4-1968

the sample model.py is

import triton_python_backend_utils as pb_utils
import _locale

class TritonPythonModel:

    def initialize(self, args):
        print(_locale.nl_langinfo(_locale.CODESET))

        print('Initialized...')

    def execute(self, requests):

        responses = []

        return responses

Describe the models (framework, inputs, outputs), ideally include the model configuration file (if using an ensemble include the model configuration file for that as well).

Expected behavior
A clear and concise description of what you expected to happen.

open function can use the LC_ALL config , and opening file with UTF-8

[lajiyuan]

same problem

[rmccorm4]

Linking a couple relevant threads, maybe some upstream issue such as NVRTC is changing the locale as a part of the python backend stub build dependencies?

• Encoding change to ANSI_X3.4-1968 on import explosion/spaCy#11909
• nvrtc.nvrtcCompileProgram is changing the preferred encoding from UTF-8 to ANSI_X3.4-1968 NVIDIA/cuda-python#29

@Tabrizian I don't directly see NVRTC used in the build, but I figured you'd be more familiar

---

@s402621342 @lajiyuan as a WAR in the meantime, can you set the encoding on the function call? ex:

f = open('myfile.txt', encoding='UTF-8')

or maybe locale.setlocale may help:

import locale
locale.set_locale(locale.LC_ALL, "C.UTF-8")

---

I tried playing with this locally in a python environment and found some similar behavior when setting locale to C/POSIX, no Triton components included at all:

>>> import locale
>>> locale.getlocale()
('en_US', 'UTF-8')
>>> locale.getpreferredencoding()
'UTF-8'
>>> locale.nl_langinfo(locale.CODESET)
'UTF-8'

>>> locale.setlocale(locale.LC_ALL, "C.UTF-8")
'C.UTF-8'
>>> locale.getlocale()
('en_US', 'UTF-8')
>>> locale.getpreferredencoding()
'UTF-8'
>>> locale.nl_langinfo(locale.CODESET)
'UTF-8'

>>> locale.setlocale(locale.LC_ALL, "C")
'C'
>>> locale.getlocale()
(None, None)
>>> locale.getpreferredencoding()
'UTF-8'
>>> locale.nl_langinfo(locale.CODESET)
'ANSI_X3.4-1968'

>>> locale.setlocale(locale.LC_ALL, "POSIX")
'C'
>>> locale.getlocale()
(None, None)
>>> locale.getpreferredencoding()
'UTF-8'
>>> locale.nl_langinfo(locale.CODESET)
'ANSI_X3.4-1968'

Even if the locale was being changed during the build, I'm not sure why it would only change after copying your build artifact (cp triton_python_backend_stub ...) rather than after the build itself as you describe above.

[s402621342]

Thank you for replying @rmccorm4

I have tried

open('myfile.txt', encoding='UTF-8')

or

import locale
locale.set_locale(locale.LC_ALL, "")

both would work, but I have a lot of python model providing by different person running on the triton server , so I don't think it's a goods idea to check every model and add such code

Python backend require the custom build triton_python_backend_stub to place like this

models
|-- model_a
    |-- 1
    |   |-- model.py
    |-- config.pbtxt
    `-- triton_python_backend_stub

That's the reason why after (cp triton_python_backend_stub ) , the behavior will change

By the way, I have tried building custom python backend on r21.12, and it would not have the same problem, I'm not sure what change between r21.12 and r22.12 cause the difference.

I found the problem when I tried to upgrade the triton server from r21.12 to r22.12 to use the new feature, and when trying to build the triton server, the same problem would happen. The build script like this

./build.py  --enable-logging --enable-stats --enable-tracing --enable-metrics  --endpoint=http --endpoint=grpc --repo-tag=common:r22.12 --repo-tag=core:r22.12  --repo-tag=backend:r22.12  --repo-tag=thirdparty:r22.12  --backend=ensemble --backend=python:r22.12

But the python backend on the nvcr.io/nvidia/tritonserver:22.12-py3 do not have the same problem

I also tried build r22.07 after and not have the same problem, maybe the change is between r22.07 and r22.12

[Tabrizian]

I was able to reproduce this issue locally. The strange thing is that the environment variable is propagated to the Python model so I'm not sure what is the issue here. We are not using nvcc or nvrtc but we do link against CUDA libraries but I don't think it is related to this issue. @rmccorm4 Can you file a ticket? It looks like the problem could've been introduced between r22.07 and r22.12.

[rmccorm4]

Sure, filed DLIS-4568 @Tabrizian

[kimdwkimdw]

@Tabrizian @rmccorm4

it was able to reproduce in 23.02 too.

[Tabrizian]

@nnshah1 Has a fix for this here: triton-inference-server/python_backend#260

Looks like it is due to a bug in pybind11.