No module named 'torch_mlir'

[maxyanghu]

Hello!

I am just trying to go through build instructions in the README. I built Pytorch Frontend in docker container following the instructions and then installed IREE via pip3 install. But When I try to run the e2e test targeting the IREE backend I got the following error:

root@0e8aafd709c6:/src/mlir-npcomp# python frontends/pytorch/e2e_testing/torchscript/main.py --config=iree
Traceback (most recent call last):
  File "frontends/pytorch/e2e_testing/torchscript/main.py", line 9, in <module>
    from torch_mlir.torchscript.e2e_test.framework import run_tests
ModuleNotFoundError: No module named 'torch_mlir'

Could anyone help me? I think I didn't miss any instructions in the README...

[byronyi]

You need to set the env variables in .env.

[maxyanghu]

You need to set the env variables in .env.

Hey byronyi,

Could you elaborate a little? my .env file is like this rn:

[maxyanghu]

And Do I need to run this 'Vanilla NumPy-only build' step:

If I do, I don't think the ninja command works because there is no ninja file generated from the configuration step, I got this error:

[silvasean]

Hi @hunbssfy , can you try using tools/torchscript_e2e_test.sh -- I recently changed the invocation method to that from a raw python invocation.

I fixed the docs here: c289d83

[maxyanghu]

Hi @hunbssfy , can you try using tools/torchscript_e2e_test.sh -- I recently changed the invocation method to that from a raw python invocation.

I fixed the docs here: c289d83

Hello @silvasean, I tried using tools/torchscript_e2e_test.sh but it still gives the same error:

[maxyanghu]

Relly need some help here to get this test running. Anyone could help me a little?

[stellaraccident]

It looks like the PYTHONPATH env variable is not set (just having it in the .env file doesn't do anything for your shell). Try exporting that in a way that is convenient.

(Sorry for delays - several of us who work on this are on vacation and afk)

[maxyanghu]

It looks like the PYTHONPATH env variable is not set (just having it in the .env file doesn't do anything for your shell). Try exporting that in a way that is convenient.

(Sorry for delays - several of us who work on this are on vacation and afk)

Hello stellaraccident:

Thanks for the reply! I think my PYTHONPATH env variable is set to the directory containing the built python package (torch-mlir), here is it:

And here is what's under the directory of /build/npcomp/python/. Am I setting the correct PYTHONPATH?

[maxyanghu]

Hi stellaraccident,

Well I added torch_mlir and mlir 's import path to the sys.path in main.py manually under e2e_testing, and it worked. Here are the test results I got:

Is this normal? And what does Expectedly Failed mean?
And the strange thing is I still cannot import torch_mlir or mlir without explicitly adding their import path to sys.path...

[stellaraccident]

Without seeing any details (like what is in the directory), it appears that at least the torch part of the project didn't build: the missing import is just a python file that should be symlinked into the build tree -- that's not even the native library, so no funny business: just python.

Look closely at the cmake output and make sure it is enabling the torch integration when it builds.

[stellaraccident]

Sorry to be pedantic, but did you export the PYTHONPATH or just set it? It looks like python isn't seeing the env var.

[maxyanghu]

You were right I didn't export the PYTHONPATH. Thanks!

About the test results though are those 'Expectedly Failed' tests supposed to fail at the moment?

[stellaraccident]

https://llvm.org/docs/TestingGuide.html#constraining-test-execution

It is a standard way in llvm to Mark a test that is known to be failing or not yet supported in some configuration.

[silvasean]

Hi @hunbssfy , I'm really sorry about the difficulty here. I recently ran into this exact same PYTHONPATH export issue when hooking up the CI and addressed it in ef118eb

Just running torchscript_e2e_test.sh should now "just work" even if you don't have your PYTHONPATH exported.

[maxyanghu]

@silvasean Thanks for the correction! It works now.