python setup.py build error

[jkl375]

When I use the image jd-train-cn-north-1-inner.jcr.service.jdcloud.com/xllm-ai:xllm-a2-aarch64-v4 and execute python setup.py build to compile, error occurs as follow:

[21/141] cd /export/home/xllm/xllm/core/framework/tokenizer/tokenizers && /usr/bin/cmake -E env CARGO_TARGET_DIR=/export/home/xllm/build/cmake.linux-aarch64-cpython-311/xllm/core/framework/tokenizer/tokenizers /root/.cargo/bin/cargo build --target aarch64-unknown-linux-gnu --release
FAILED: xllm/core/framework/tokenizer/tokenizers/aarch64-unknown-linux-gnu/release/librust_tokenizers.so /export/home/xllm/build/cmake.linux-aarch64-cpython-311/xllm/core/framework/tokenizer/tokenizers/aarch64-unknown-linux-gnu/release/librust_tokenizers.so 
cd /export/home/xllm/xllm/core/framework/tokenizer/tokenizers && /usr/bin/cmake -E env CARGO_TARGET_DIR=/export/home/xllm/build/cmake.linux-aarch64-cpython-311/xllm/core/framework/tokenizer/tokenizers /root/.cargo/bin/cargo build --target aarch64-unknown-linux-gnu --release
   Compiling rust_tokenizers v0.21.0 (/export/home/xllm/xllm/core/framework/tokenizer/tokenizers)
error: implicit autoref creates a reference to the dereference of a raw pointer
   --> src/lib.rs:152:20
    |
152 |         *out_len = (*handle).decode_str.len();
    |                    ^^------^^^^^^^^^^^^^^^^^^
    |                      |
    |                      this raw pointer has type `*mut TokenizerWrapper`
    |
    = note: creating a reference requires the pointer target to be valid and imposes aliasing requirements
note: autoref is being applied to this expression, resulting in: `&std::string::String`
   --> src/lib.rs:152:20
    |
152 |         *out_len = (*handle).decode_str.len();
    |                    ^^^^^^^^^^^^^^^^^^^^
note: method calls to `len` require a reference
   --> /rustc/29483883eed69d5fb4db01964cdf2af4d86e9cb2/library/alloc/src/string.rs:1854:5
    = note: `#[deny(dangerous_implicit_autorefs)]` on by default
help: try using a raw pointer method instead; or if this reference is intentional, make it explicit
    |
152 |         *out_len = (&(*handle).decode_str).len();
    |                    ++                    +

error: implicit autoref creates a reference to the dereference of a raw pointer
   --> src/lib.rs:185:20
    |
185 |         *out_len = (*handle).id_to_token_result.len();
    |                    ^^------^^^^^^^^^^^^^^^^^^^^^^^^^^
    |                      |
    |                      this raw pointer has type `*mut TokenizerWrapper`
    |
    = note: creating a reference requires the pointer target to be valid and imposes aliasing requirements
note: autoref is being applied to this expression, resulting in: `&std::string::String`
   --> src/lib.rs:185:20
    |
185 |         *out_len = (*handle).id_to_token_result.len();
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: method calls to `len` require a reference
   --> /rustc/29483883eed69d5fb4db01964cdf2af4d86e9cb2/library/alloc/src/string.rs:1854:5
help: try using a raw pointer method instead; or if this reference is intentional, make it explicit
    |
185 |         *out_len = (&(*handle).id_to_token_result).len();
    |                    ++                            +

error: could not compile `rust_tokenizers` (lib) due to 2 previous errors
ninja: build stopped: subcommand failed.

Traceback (most recent call last):
  File "/export/home/xllm/setup.py", line 422, in <module>
    setup(
  File "/usr/local/lib/python3.11/site-packages/setuptools/__init__.py", line 115, in setup
    return distutils.core.setup(**attrs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 186, in setup
    return run_commands(dist)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
    dist.run_commands()
  File "/usr/local/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
    self.run_command(cmd)
  File "/usr/local/lib/python3.11/site-packages/setuptools/dist.py", line 1102, in run_command
    super().run_command(command)
  File "/usr/local/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.11/site-packages/setuptools/_distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/usr/local/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python3.11/site-packages/setuptools/dist.py", line 1102, in run_command
    super().run_command(command)
  File "/usr/local/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
    cmd_obj.run()
  File "/export/home/xllm/setup.py", line 243, in run
    self.build_extension(ext)
  File "/export/home/xllm/setup.py", line 304, in build_extension
    subprocess.check_call(["cmake", "--build", ".", "--verbose"] + build_args, cwd=cmake_dir)
  File "/usr/lib64/python3.11/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--verbose', '--config', 'Release', '-j192', '--target', 'xllm', 'xllm']' returned non-zero exit status 1.

[yq33victor]

@jkl375 this commit fix the error 45682de