Skip to content
This repository has been archived by the owner on Mar 14, 2021. It is now read-only.

Cannot import name 'Agent' from 'agent' #8

Closed
Krutonium opened this issue Aug 31, 2019 · 31 comments
Closed

Cannot import name 'Agent' from 'agent' #8

Krutonium opened this issue Aug 31, 2019 · 31 comments

Comments

@Krutonium
Copy link

 krutonium@krutonium-pc  ~/craftassist   master  python3 ./python/craftassist/craftassist_agent.py
Traceback (most recent call last):
  File "./python/craftassist/craftassist_agent.py", line 22, in <module>
    from agent import Agent
ImportError: cannot import name 'Agent' from 'agent' (/home/krutonium/.local/lib/python3.7/site-packages/agent.py)

Am I being boneheaded or am I doing somthing wrong?

@jonbaer
Copy link

jonbaer commented Aug 31, 2019

You need to build the C++ client first so you end up with a file like python/agent.cpython-37m-darwin.so (on OSX) ... from the root directory you could cd into client dir and cmake . && make (you are basically missing the lib python is "talking" to)

@Krutonium
Copy link
Author

Krutonium commented Aug 31, 2019

I cloned and ran make as the readme says, the server itself starts up fine, indicating building worked afaik. Of note, I am running Arch Linux.

Edit: I just re-read it I think I get what you mean

 krutonium@krutonium-pc  ~/craftassist/client   master  make
[  7%] Building CXX object CMakeFiles/agent.dir/src/agent.cpp.o
/home/krutonium/craftassist/client/src/agent.cpp: In member function ‘bool Agent::setHeldItem(pybind11::object)’:
/home/krutonium/craftassist/client/src/agent.cpp:203:16: error: catching polymorphic type ‘class pybind11::cast_error’ by value [-Werror=catch-value=]
  203 |   } catch (py::cast_error) {
      |                ^~~~~~~~~~
/home/krutonium/craftassist/client/src/agent.cpp: In member function ‘int Agent::craft(pybind11::object)’:
/home/krutonium/craftassist/client/src/agent.cpp:269:16: error: catching polymorphic type ‘class pybind11::cast_error’ by value [-Werror=catch-value=]
  269 |   } catch (py::cast_error) {
      |                ^~~~~~~~~~
cc1plus: all warnings being treated as errors
make[2]: *** [CMakeFiles/agent.dir/build.make:63: CMakeFiles/agent.dir/src/agent.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:79: CMakeFiles/agent.dir/all] Error 2
make: *** [Makefile:84: all] Error 2

@jonbaer
Copy link

jonbaer commented Aug 31, 2019

Looks similar to issue @ #5 ... either way obviously the agent did not build correctly so could not be imported (on the py side). Maybe dig around the pybind11 side if it's an issue on Arch Linux, https://github.com/pybind/pybind11/

@Krutonium
Copy link
Author

Krutonium commented Sep 1, 2019

I asked someone who uses pybind, they said as far as they know it's not them, and pointed at this https://stackoverflow.com/questions/2522299/c-catch-blocks-catch-exception-by-value-or-reference/2522311#2522311

@drothermel
Copy link
Contributor

It does look pretty similar to issue #5, what version of gcc are you using so that we can try to repro?

@Krutonium
Copy link
Author

Krutonium commented Sep 4, 2019

9.1.0, but it also failed when I tried to use 7.4.1.

You can fix the first couple errors and run into some entirely different (memory related?) errors if you do this in agent.cpp

Replace:

  } catch (py::cast_error) {

with

  } catch (py::cast_error const&) {

@Krutonium
Copy link
Author

Okay so:

Make the change above, then git submodule update --remote client/lib/rapidjson to fix Tencent/rapidjson#1323 . After that, it seems to be broken at ld

[ 38%] Linking CXX shared module ../python/agent.cpython-37m-x86_64-linux-gnu.so
/usr/bin/ld: cannot find -lgflags
collect2: error: ld returned 1 exit status
make[3]: *** [CMakeFiles/agent.dir/build.make:249: ../python/agent.cpython-37m-x86_64-linux-gnu.so] Error 1
make[3]: Leaving directory '/home/krutonium/craftassist/client'
make[2]: *** [CMakeFiles/Makefile2:79: CMakeFiles/agent.dir/all] Error 2
make[2]: Leaving directory '/home/krutonium/craftassist/client'
make[1]: *** [Makefile:84: all] Error 2
make[1]: Leaving directory '/home/krutonium/craftassist/client'
make: *** [Makefile:20: client] Error 2

@drothermel
Copy link
Contributor

Thanks for the additional information! We just tried again from scratch to install the repo using the specified gcc on ubuntu and succeeded (as have others, on ubuntu and mac). So it seems likely that is specific to Arch Linux, which we don't currently support.

However, we appreciate you continuing to solve the errors that are arising on your system and hope it will be helpful for others who run into similar ones!

@Krutonium
Copy link
Author

Krutonium commented Sep 6, 2019

I mean, these errors should probably be resolved regardless, since the longer you wait the harder it will be to make this run on newer systems. Ubuntu releases every 6 months, remember, and is going to eventually update gcc.

Edit: Realized I needed to install the Python Development Files. Flips table

@Krutonium
Copy link
Author

Krutonium commented Sep 6, 2019

Also worth noting that on my Arch system and my Solus system, neither one can install the python requirements. Any tips?

Edit: Previous edit was intended to go here.

@Krutonium
Copy link
Author

Krutonium commented Sep 6, 2019

(craft) krutonium@desktop-solus ~/craftassist $ python ./python/craftassist/craftassist_agent.py
Traceback (most recent call last):
  File "./python/craftassist/craftassist_agent.py", line 24, in <module>
    from vision.subcomponent_classifier import SubComponentClassifier
  File "/home/krutonium/craftassist/python/craftassist/vision/subcomponent_classifier.py", line 17, in <module>
    from semseg_models import SemSegWrapper
  File "/home/krutonium/craftassist/python/craftassist/vision/semantic_segmentation/semseg_models.py", line 6, in <module>
    import torch
ModuleNotFoundError: No module named 'torch'

(Everything now compiles, but it still doesn't run)

@drothermel
Copy link
Contributor

Progress, great! Yeah, if you can repro the problem on another system we'll keep trying to repro as well.

I actually think you're pretty close, if you're using conda, try:
conda install pytorch torchvision -c pytorch
Otherwise with pip try:
pip3 install torch torchvision

From there, if it still doesn't work (depending on the error) I might suggest a path that uses anaconda + conda environments.

Separately, we just updated our README to make this part easier, thanks for the feedback!

@drothermel
Copy link
Contributor

My wording was imprecise, what I meant was "great, thanks for repro'ing on another system. Because you've demonstrated its a broader problem, its a supported issue and we'll keep working with you to fix it. Thank you for testing our project and finding and surfacing ways that we can improve it!"

Did you get a chance to try the pip/conda install commands?

@Krutonium
Copy link
Author

I haven't had a chance to sit down and play with it again yet, no. Soon though!

@kavyasrinet
Copy link
Contributor

Thanks so much for keeping us updated!
As @drothermel said above, we'd be happy to work with you to fix the issues and add a section to our README once we resolve it.

@Krutonium
Copy link
Author

I can't get Conda to agree with my system, so ignoring that since it's still using the base env which whatever, I'm still unable to install all pacakges. https://paste.ubuntu.com/p/fYwN2kPvhv/

@drothermel
Copy link
Contributor

Ah, perhaps I'm missing something, my understanding was that as of your "(Everything now compiles, but it still doesn't run)" comment the only failure when you ran the bot was that ModuleNotFoundError: No module named 'torch'.

Is the paste above from trying to start from scratch with conda? And if so, in your old setup where everything else was working, would you mind trying pip3 install torch torchvision and seeing if that fixes the ModuleNotFoundError?

Then we can work through starting from scratch with conda if thats what your paste is from! Otherwise, what commands did you run to produce that paste?

@Krutonium
Copy link
Author

Krutonium commented Sep 20, 2019

Otherwise, what commands did you run to produce that paste?

I copied the script in the readme into a .sh file and ran it, and it output that, after install conda (which annoyingly doesn't want to init no matter how many times I init it)

I'm going to try installing torch manually in just a moment

Edit: That worked, but the client is back at "No module named 'agent'"

https://paste.ubuntu.com/p/mtXrrjHzXN/

@drothermel
Copy link
Contributor

We were eventually able to repro this issue and have a fix. We're in the process of pushing the fix but if you would like to try it out before the push:

First, update the requirements.txt file to contain the following:

awscli==1.15.7
black==18.9b0
botocore==1.10.7
cython==0.29.14
decorator==4.3.0
docopt==0.6.2
docutils==0.14
flake8==3.5.0
flake8-mypy==17.8.0
ipdb==0.11
ipython-genutils==0.2.0
ipython==6.3.1
joblib==0.11
matplotlib==2.2.2
mypy-extensions==0.4.1
mypy==0.701
numpy==1.14.2
python-Levenshtein==0.12.0
python-dateutil==2.7.2
s3transfer==0.1.13
scikit-learn==0.19.1
scipy==1.1.0
sentry-sdk==0.7.9
six==1.11.0
snowballstemmer==1.2.1
spacy==2.0.12
tqdm==4.23.4
word2number==1.1
https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz

Then, after making sure you have the dependencies listed in the "Dependencies" section, use conda to create a new env preloaded with the conda install deps and activate the env:

conda create -n minecraft_env python==3.7.4 pip numpy scikit-learn==0.19.1 pytorch torchvision -c conda-forge -c pytorch
conda activate minecraft_env

Then, install the rest of the deps with pip:

while read requirement; do echo $requirement; pip install $requirement; done < requirements.txt

The versions specified above, installed in this order should work and solve the no module named agent issue (but let us know if this is not the case)!

@kavyasrinet
Copy link
Contributor

Hi @Krutonium , can you please confirm if the above instructions helped ?

@Krutonium
Copy link
Author

Sorry, I completely missed the notification - I'll give it a shot shortly.

@Krutonium
Copy link
Author

Krutonium commented Jan 11, 2020

Traceback (most recent call last):
  File "./python/craftassist/craftassist_agent.py", line 22, in <module>
    from agent import Agent
ModuleNotFoundError: No module named 'agent'

I started with a completely fresh clone and this is where it finished.

@kavyasrinet
Copy link
Contributor

This looks like the build didn't go through.
Tagging @drothermel to take a look.

@drothermel
Copy link
Contributor

Hi @Krutonium, we just did a push and I stepped through the new instructions in the README start to finish to ensure that they worked for me from a clean conda env and build.

Would you please pull and then specifically do the instructions under "Python Requirements: Using a Conda Environment" through "Checking out the repo" (the curl and tar commands to get the model data files) and let me know if you're still getting the above error?

If you're still getting the error, would you please also paste the output of a conda list for your environment and that of ls python/?

@kavyasrinet
Copy link
Contributor

Hi @Krutonium ,
Just checking in again :)
Could you please confirm if the instructions @drothermel posted above solves your problem.

@Krutonium
Copy link
Author

So this is where I am at now

https://asciinema.org/a/nFExrEMeYhRr9sRI1TmFdPkXX

It would seem the requirements are no longer functional, specifically

ERROR: boto3 1.11.13 has requirement botocore<1.15.0,>=1.14.13, but you'll have botocore 1.10.7 which is incompatible.
ERROR: boto3 1.11.13 has requirement s3transfer<0.4.0,>=0.3.0, but you'll have s3transfer 0.1.13 which is incompatible.

Also, the readme seems to be out of order, since you have to clone the repo before you have access to requirements.txt

It also says to cd minecraft which as far as I can tell isn't a directory that actually exists?

@snyxan
Copy link
Member

snyxan commented Feb 10, 2020

Hi @Krutonium ,

Glad to help here!

So this is where I am at now

https://asciinema.org/a/nFExrEMeYhRr9sRI1TmFdPkXX

It would seem the requirements are no longer functional, specifically

ERROR: boto3 1.11.13 has requirement botocore<1.15.0,>=1.14.13, but you'll have botocore 1.10.7 which is incompatible.
ERROR: boto3 1.11.13 has requirement s3transfer<0.4.0,>=0.3.0, but you'll have s3transfer 0.1.13 which is incompatible.

In the video, it looks like some objects are skipped because your git lfs is not properly installed. e.g.:

Entering 'cuberite/lib/sqlite'                                                                                                                                                                                   
Skipping object checkout, Git LFS is not installed.

Try run git lfs install before cloning the repo.

Also, the readme seems to be out of order, since you have to clone the repo before you have access to requirements.txt

You are right. You can do pip install -r requirements.txt after cloning our repo.

It also says to cd minecraft which as far as I can tell isn't a directory that actually exists?

It should be cd craftassist, which is the name of our repo. Sorry for the confusion!

@Krutonium
Copy link
Author

Krutonium commented Feb 11, 2020

https://asciinema.org/a/Yp1M9qcJ987lnAXwX7AFgY7fm

How does this look?

@snyxan
Copy link
Member

snyxan commented Feb 18, 2020

https://asciinema.org/a/Yp1M9qcJ987lnAXwX7AFgY7fm

How does this look?

Hi @Krutonium , it seems like an anaconda linker issue. Please refer pytorch/pytorch#16683 (comment) to see if it solves your problem.

Related issues:
pytorch/pytorch#16683 (comment)
ContinuumIO/anaconda-issues#11152

@kavyasrinet
Copy link
Contributor

Hi @Krutonium !
Checking in if @snyxan 's comment above helps fix the issue.

@kavyasrinet
Copy link
Contributor

Hi @Krutonium !
We will be closing this issue for now.
We'll also be sending out a heads-up over email but we have now migrated the craftassist codebase to be here: https://github.com/facebookresearch/droidlet and we will be maintaining this repo going forward. We ask that you please switch to the droidlet repo going forward.
Please feel free to reopen the issue under the droidlet repo if you still need help.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants