[Question] why the latest snapshot build is in January?

[vmiheer]

After installing latest snapshot build from releases, I was trying to run https://github.com/llvm/torch-mlir/blob/main/test/python/fx_importer/sparse_test.py but it was giving me error.
I was wondering why there are no newer snapshot builds in releases?
cc: @stellaraccident because they seem to have release the last build.
Also does it make sense to pin numpy at 1.26 in https://github.com/llvm/torch-mlir/blob/main/build-requirements.txt? (I got error while running above example with in tree build with suggested cmake options that two numpy versions are used in two different projects.

[stellaraccident]

We don't do release builds in this repo anymore. Was a spy chain security thing.

A community member has been running then in a sibling repo with more security: https://github.com/llvm/torch-mlir-release

We need to update docs and haven't. Many of the users are leveraging torch-mlir as a library not from a community binary.

[stellaraccident]

I can't really comment on the rest without details of the errors/issues. In general for most of that stuff, we just want what torch brings with it.

[vmiheer]

Thanks @stellaraccident, I can now reuse pre-built wheels.

I can't really comment on the rest without details of the errors/issues. In general for most of that stuff, we just want what torch brings with it.

In meanwhile, I was building torch-mlir from source.

my torch-mlir install script:

[[ -f ~/commons.sh ]] && source ~/commons.sh
declare -f enable_pyenv &>/dev/null && enable_pyenv
pyenv activate torch_mlir
TOP_DIR=$(pwd)/build
mkdir -p build && cd build
if [[ ! -d torch-mlir ]]; then
  git clone --single-branch https://github.com/llvm/torch-mlir
fi
cd torch-mlir
git submodule update --init --progress --depth=1

python -m pip install --upgrade pip
# Install latest PyTorch nightlies and build requirements.
python -m pip install -r requirements.txt
python -m pip install -r torchvision-requirements.txt
# torchvision seem to be compile against numpy 1.X
# python -m pip install numpy==1.26 <<< SEE HERE
mkdir -p build
if [[ ! -f build/build.ninja ]]; then
  cmake -GNinja -Bbuild \
    -DCMAKE_BUILD_TYPE=Debug \
    -DPython3_FIND_VIRTUALENV=ONLY \
    -DLLVM_ENABLE_PROJECTS=mlir \
    -DLLVM_EXTERNAL_PROJECTS="torch-mlir" \
    -DLLVM_EXTERNAL_TORCH_MLIR_SOURCE_DIR="$PWD" \
    -DMLIR_ENABLE_BINDINGS_PYTHON=ON \
    -DLLVM_TARGETS_TO_BUILD=host \
    -DCMAKE_C_COMPILER_LAUNCHER=ccache \
    -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
    -DCMAKE_EXE_LINKER_FLAGS_INIT="-fuse-ld=lld" \
    -DCMAKE_MODULE_LINKER_FLAGS_INIT="-fuse-ld=lld" \
    -DCMAKE_SHARED_LINKER_FLAGS_INIT="-fuse-ld=lld" \
    externals/llvm-project/llvm
fi
export PYTHONPATH=`pwd`/build/tools/torch-mlir/python_packages/torch_mlir:\
`pwd`/test/python/fx_importer:$PYTHONPATH
export PATH=`pwd`/build/bin:$PATH
cd $TOP_DIR/..

When I run:

python test/python/fx_importer/sparse_test.py

I get error:

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0b1 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled against NumPy 2.0.

If you are a user of the module, the easiest solution will be to
either downgrade NumPy or update the failing module (if available).

Things are okay when I explicitly install numpy 1.x as suggested. But it would be great to have it pinned in build-requirements.txt.

detailed error:

test_sparse_sum --------------- /torch-mlir/test/python/fx_importer/sparse_test.py:215: UserWarning: Sparse CSR tensor support is in beta state. If you miss a functionality in the sparse tensor support, please submit a feature request to https://github.com/pytorch/pytorch/issues. (Triggered internally at ../aten/src/ATen/SparseCsrTensorImpl.cpp:53.) sparse_input = dense_input.to_sparse_csr()

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0b1 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled against NumPy 2.0.

If you are a user of the module, the easiest solution will be to
either downgrade NumPy or update the failing module (if available).

Traceback (most recent call last): File "/torch-mlir/test/python/fx_importer/sparse_test.py", line 192, in
@run
File "/torch-mlir/test/python/fx_importer/sparse_test.py", line 188, in run
f()
File "/torch-mlir/test/python/fx_importer/sparse_test.py", line 216, in test_sparse_sum
m = export_and_import(net, sparse_input)
File "/torch-mlir/test/python/fx_importer/sparse_test.py", line 131, in export_and_import
prog = sparse_export(f, args, kwargs)
File "/torch-mlir/test/python/fx_importer/sparse_test.py", line 107, in sparse_export
prog = torch.export.export(f, dargs, kwargs)
File "/.pyenv/versions/torch_mlir/lib/python3.11/site-packages/torch/export/init.py", line 174, in export
return _export(
File "/.pyenv/versions/torch_mlir/lib/python3.11/site-packages/torch/export/_trace.py", line 659, in wrapper
ep = fn(*args, **kwargs)
File "/.pyenv/versions/torch_mlir/lib/python3.11/site-packages/torch/export/exported_program.py", line 84, in wrapper
return fn(*args, **kwargs)
File "/.pyenv/versions/torch_mlir/lib/python3.11/site-packages/torch/export/_trace.py", line 908, in _export
gm_torch_level = _export_to_torch_ir(
File "/.pyenv/versions/torch_mlir/lib/python3.11/site-packages/torch/export/_trace.py", line 349, in _export_to_torch_ir
gm_torch_level, _ = torch._dynamo.export(
File "/.pyenv/versions/torch_mlir/lib/python3.11/site-packages/torch/_dynamo/eval_frame.py", line 1221, in inner
opt_f = optimize_assert(
File "/.pyenv/versions/torch_mlir/lib/python3.11/site-packages/torch/_dynamo/eval_frame.py", line 1380, in optimize_assert
backend = get_compiler_fn(backend)
File "/.pyenv/versions/torch_mlir/lib/python3.11/site-packages/torch/_dynamo/eval_frame.py", line 512, in get_compiler_fn
from .repro.after_dynamo import wrap_backend_debug
File "/.pyenv/versions/torch_mlir/lib/python3.11/site-packages/torch/_dynamo/repro/after_dynamo.py", line 15, in
from torch._dynamo.debug_utils import (
File "/.pyenv/versions/torch_mlir/lib/python3.11/site-packages/torch/_dynamo/debug_utils.py", line 511, in
_device_or_default = _mk_defaulter(torch.device("cpu"))
/.pyenv/versions/torch_mlir/lib/python3.11/site-packages/torch/_dynamo/debug_utils.py:511: UserWarning: Failed to initialize NumPy: module compiled against ABI version 0x1000009 but this version of numpy is 0x2000000 (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:84.)
_device_or_default = _mk_defaulter(torch.device("cpu"))
#sparse = #sparse_tensor.encoding<{ map = (d0, d1) -> (d0 : dense, d1 : compressed), posWidth = 64, crdWidth = 64 }>
module {
func.func @main(%arg0: !torch.vtensor<[64,64],f32,#sparse>) -> !torch.vtensor<[],f32> {
%none = torch.constant.none
%0 = torch.aten.sum %arg0, %none : !torch.vtensor<[64,64],f32,#sparse>, !torch.none -> !torch.vtensor<[],f32>
return %0 : !torch.vtensor<[],f32>
}
}

Traceback (most recent call last):
File "/torch-mlir/test/python/fx_importer/sparse_test.py", line 192, in
@run
^^^
File "/torch-mlir/test/python/fx_importer/sparse_test.py", line 188, in run
f()
File "/torch-mlir/test/python/fx_importer/sparse_test.py", line 221, in test_sparse_sum
res2 = sparse_jit(net, sparse_input)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/torch-mlir/test/python/fx_importer/sparse_test.py", line 172, in sparse_jit
xargs.append(a.values().numpy())
^^^^^^^^^^^^^^^^^^
RuntimeError: Numpy is not available