Win7 32-bit, when I install mars, it reports file not found

[eagleLiu82]

Environment：win7 32-bit+python3.7.0
Execute：pip install pymars
Error Message：

Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\ADMINI~1\AppData\Local\Temp\pip-install-h838j__8\pymars\set
up.py", line 42, in <module>
        with open(os.path.join(repo_root, 'requirements-extra.txt'), 'r') as f:
    FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\ADMINI~1
\\AppData\\Local\\Temp\\pip-install-h838j__8\\pymars\\requirements-extra.txt'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\ADMINI~1
\AppData\Local\Temp\pip-install-h838j__8\pymars\

[qinxuye]

Could you please translate your issue into English? Thanks in advance.

[wjsi]

Due to limitations by pyarrow, we do not support 32-bit Windows, please use 64-bit version instead. What's more, only threaded mode or client mode is supported in Windows.

[eagleLiu82]

Due to limitations by pyarrow, we do not support 32-bit Windows, please use 64-bit version instead. What's more, only threaded mode or client mode is supported in Windows.

suse 10 64bit, when I use pip install pymars, it reports "Import Error:no module named Cython". I think this project should tell us what should prepare.

[qinxuye]

Mars has pre-compiled wheel files listed at pypi: https://pypi.org/project/pymars/#files, if the Python version are one of 2.7, 3.5, 3.6 and 3.7, usually, pip install pymars will work. In other situations, the source code will be fetched and Cython would be necessary to do compilation work.

[eagleLiu82]

Mars has pre-compiled wheel files listed at pypi: https://pypi.org/project/pymars/#files, if the Python version are one of 2.7, 3.5, 3.6 and 3.7, usually, pip install pymars will work. In other situations, the source code will be fetched and Cython would be necessary to do compilation work.

suse 10 64bit,python3.6
when I use source code to install pymars, it reports " with open(os.path.join(repo_root, 'requirements-extra.txt'), 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/root/pymars-0.1.0b1/requirements-extra.txt'"
but requirements-extra.txt is not in source code tar file.

[qinxuye]

Yeah, this must be a problem, we should have forgotten to add the requirements-extra.txt to the MANIFEST.in. I will fix this soon later.

Could you please try to install directly from Github with:

pip install git+https://github.com/mars-project/mars.git

see if it can work.

[eagleLiu82]

Yeah, this must be a problem, we should have forgotten to add the requirements-extra.txt to the MANIFEST.in. I will fix this soon later.

Could you please try to install directly from Github with:

pip install git+https://github.com/mars-project/mars.git

see if it can work.

yes, use pip install git+https://github.com/mars-project/mars.git seems ok.
it report "Successfully built pymars
distributed 1.21.8 requires msgpack, which is not installed.
Installing collected packages: pyarrow, pymars
Successfully installed pyarrow-0.11.1 pymars-0.1.0b1
"
but when I excute"import mars.tensor as mt" in python console, it reports " from .core import ValueType, Serializable, SerializableMetaclass, AttributeAsDict,
ModuleNotFoundError: No module named 'mars.serialize.core'"

[qinxuye]

ModuleNotFoundError: No module named 'mars.serialize.core'"

Can you find the serialize directory in the location where Mars installed?

[eagleLiu82]

yes, serialize directory exist in where I install mars.

[qinxuye]

yes, serialize directory exist in where I install mars.

Is there any .so files exist in the serialize directory?

[eagleLiu82]

I can't find any .so files in serialize directory.

[qinxuye]

I can't find any .so files in serialize directory.

Could you please list the files in that directory?

[eagleLiu82]

"core.c" "init.py" "pbserializer.c" "pycache"
"dataserializer.py" "jsonserializer.c" "protos"

[qinxuye]

"core.c" "init.py" "pbserializer.c" "pycache"
"dataserializer.py" "jsonserializer.c" "protos"

Could you please try to remove the installed one by either pip uninstall pymars or delete the directory directly, and reinstall from Github again?

[eagleLiu82]

yes, I reinstall it, but it reports the same Error message.
"File "/home/root/pymars-0.1.0b1/mars/serialize/init.py", line 18, in
from .core import ValueType, Serializable, SerializableMetaclass, AttributeAsDict,
ModuleNotFoundError: No module named 'mars.serialize.core'"

[qinxuye]

yes, I reinstall it, but it reports the same Error message.
"File "/home/root/pymars-0.1.0b1/mars/serialize/init.py", line 18, in
from .core import ValueType, Serializable, SerializableMetaclass, AttributeAsDict,
ModuleNotFoundError: No module named 'mars.serialize.core'"

Seems that the c file not compiled into .so, never encounter that before. What's the version of Python and Cython?

[eagleLiu82]

yes, I reinstall it, but it reports the same Error message.
"File "/home/root/pymars-0.1.0b1/mars/serialize/init.py", line 18, in
from .core import ValueType, Serializable, SerializableMetaclass, AttributeAsDict,
ModuleNotFoundError: No module named 'mars.serialize.core'"

Seems that the c file not compiled into .so, never encounter that before. What's the version of Python and Cython?

Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56)
[GCC 7.2.0] on linux

[qinxuye]

yes, I reinstall it, but it reports the same Error message.
"File "/home/root/pymars-0.1.0b1/mars/serialize/init.py", line 18, in
from .core import ValueType, Serializable, SerializableMetaclass, AttributeAsDict,
ModuleNotFoundError: No module named 'mars.serialize.core'"

Seems that the c file not compiled into .so, never encounter that before. What's the version of Python and Cython?

Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56)
[GCC 7.2.0] on linux

And how about the Cython version? thanks.

[wjsi]

Installing by pip install git+https://github.com/mars-project/mars.git will first clone the repo to a temp dir and then start building wheel. Therefore it is strange for me to see the error occurred in a home path without *.pyx.

[eagleLiu82]

Cython version 0.28.2

[qinxuye]

Cython version 0.28.2

Sorry to bother you again, could you please git clone the code to some place, enter the mars diretory, and then run

python setup.py build_ext -i

Check if any .so file exists in the serialize directory.

[eagleLiu82]

.so file exists.
but when I excute "import mars.tensor as mt" in python console, it reports Error message.
File "/home/root/anaconda3/lib/python3.6/site-packages/mars/serialize/protos/value_pb2.py", line 22, in
serialized_pb=_b('\n!mars/serialize/protos/value.proto"\xa6\x04\n\x05Value\x12\x11\n\x07is_null\x18\x01 \x01(\x08H\x00\x12\x0b\n\x01\x62\x18\x02 \x01(\x08H\x00\x12\x0b\n\x01i\x18\x03 \x01(\x03H\x00\x12\x0b\n\x01\x66\x18\x04 \x01(\x02H\x00\x12\x0b\n\x01s\x18\x05 \x01(\x0cH\x00\x12\x0b\n\x01u\x18\x06 \x01(\tH\x00\x12\x1b\n\x04list\x18\x07 \x01(\x0b\x32\x0b.Value.ListH\x00\x12\x1b\n\x04\x64ict\x18\x08 \x01(\x0b\x32\x0b.Value.DictH\x00\x12\x1d\n\x05slice\x18\t \x01(\x0b\x32\x0c.Value.SliceH\x00\x12\r\n\x03\x61rr\x18\n \x01(\x0cH\x00\x12\x0f\n\x05\x64type\x18\x0b \x01(\x0cH\x00\x12\x19\n\x03key\x18\x0c \x01(\x0b\x32\n.Value.KeyH\x00\x12\x14\n\ndatetime64\x18\r \x01(\x0cH\x00\x12\x15\n\x0btimedelta64\x18\x0e \x01(\x0cH\x00\x1a/\n\x04List\x12\x10\n\x08is_tuple\x18\x01 \x01(\x08\x12\x15\n\x05value\x18\x02 \x03(\x0b\x32\x06.Value\x1an\n\x05Slice\x12\x0f\n\x07is_null\x18\x04 \x01(\x08\x12\x13\n\tstart_val\x18\x01 \x01(\x03H\x00\x12\x12\n\x08stop_val\x18\x02 \x01(\x03H\x01\x12\x12\n\x08step_val\x18\x03 \x01(\x03H\x02\x42\x07\n\x05startB\x06\n\x04stopB\x06\n\x04step\x1a>\n\x04\x44ict\x12\x19\n\x04keys\x18\x01 \x01(\x0b\x32\x0b.Value.List\x12\x1b\n\x06values\x18\x02 \x01(\x0b\x32\x0b.Value.List\x1a\x1e\n\x03Key\x12\x0b\n\x03key\x18\x01 \x01(\t\x12\n\n\x02id\x18\x02 \x01(\tB\x07\n\x05valueb\x06proto3')
TypeError: init() got an unexpected keyword argument 'serialized_options'

[qinxuye]

.so file exists.
but when I excute "import mars.tensor as mt" in python console, it reports Error message.
File "/home/root/anaconda3/lib/python3.6/site-packages/mars/serialize/protos/value_pb2.py", line 22, in
serialized_pb=_b('\n!mars/serialize/protos/value.proto"\xa6\x04\n\x05Value\x12\x11\n\x07is_null\x18\x01 \x01(\x08H\x00\x12\x0b\n\x01\x62\x18\x02 \x01(\x08H\x00\x12\x0b\n\x01i\x18\x03 \x01(\x03H\x00\x12\x0b\n\x01\x66\x18\x04 \x01(\x02H\x00\x12\x0b\n\x01s\x18\x05 \x01(\x0cH\x00\x12\x0b\n\x01u\x18\x06 \x01(\tH\x00\x12\x1b\n\x04list\x18\x07 \x01(\x0b\x32\x0b.Value.ListH\x00\x12\x1b\n\x04\x64ict\x18\x08 \x01(\x0b\x32\x0b.Value.DictH\x00\x12\x1d\n\x05slice\x18\t \x01(\x0b\x32\x0c.Value.SliceH\x00\x12\r\n\x03\x61rr\x18\n \x01(\x0cH\x00\x12\x0f\n\x05\x64type\x18\x0b \x01(\x0cH\x00\x12\x19\n\x03key\x18\x0c \x01(\x0b\x32\n.Value.KeyH\x00\x12\x14\n\ndatetime64\x18\r \x01(\x0cH\x00\x12\x15\n\x0btimedelta64\x18\x0e \x01(\x0cH\x00\x1a/\n\x04List\x12\x10\n\x08is_tuple\x18\x01 \x01(\x08\x12\x15\n\x05value\x18\x02 \x03(\x0b\x32\x06.Value\x1an\n\x05Slice\x12\x0f\n\x07is_null\x18\x04 \x01(\x08\x12\x13\n\tstart_val\x18\x01 \x01(\x03H\x00\x12\x12\n\x08stop_val\x18\x02 \x01(\x03H\x01\x12\x12\n\x08step_val\x18\x03 \x01(\x03H\x02\x42\x07\n\x05startB\x06\n\x04stopB\x06\n\x04step\x1a>\n\x04\x44ict\x12\x19\n\x04keys\x18\x01 \x01(\x0b\x32\x0b.Value.List\x12\x1b\n\x06values\x18\x02 \x01(\x0b\x32\x0b.Value.List\x1a\x1e\n\x03Key\x12\x0b\n\x03key\x18\x01 \x01(\t\x12\n\n\x02id\x18\x02 \x01(\tB\x07\n\x05valueb\x06proto3')
TypeError: init() got an unexpected keyword argument 'serialized_options'

Seems protobuf is old, try conda update protobuf.

[eagleLiu82]

I have update protobuf, but also report error message.
“ File "mars/serialize/pbserializer.pyx", line 33, in init mars.serialize.pbserializer
File "/home/root/anaconda3/lib/python3.6/site-packages/mars/serialize/protos/value_pb2.py", line 22, in
serialized_pb=_b('\n!mars/serialize/protos/value.proto"\xa6\x04\n\x05Value\x12\x11\n\x07is_null\x18\x01 \x01(\x08H\x00\x12\x0b\n\x01\x62\x18\x02 \x01(\x08H\x00\x12\x0b\n\x01i\x18\x03 \x01(\x03H\x00\x12\x0b\n\x01\x66\x18\x04 \x01(\x02H\x00\x12\x0b\n\x01s\x18\x05 \x01(\x0cH\x00\x12\x0b\n\x01u\x18\x06 \x01(\tH\x00\x12\x1b\n\x04list\x18\x07 \x01(\x0b\x32\x0b.Value.ListH\x00\x12\x1b\n\x04\x64ict\x18\x08 \x01(\x0b\x32\x0b.Value.DictH\x00\x12\x1d\n\x05slice\x18\t \x01(\x0b\x32\x0c.Value.SliceH\x00\x12\r\n\x03\x61rr\x18\n \x01(\x0cH\x00\x12\x0f\n\x05\x64type\x18\x0b \x01(\x0cH\x00\x12\x19\n\x03key\x18\x0c \x01(\x0b\x32\n.Value.KeyH\x00\x12\x14\n\ndatetime64\x18\r \x01(\x0cH\x00\x12\x15\n\x0btimedelta64\x18\x0e \x01(\x0cH\x00\x1a/\n\x04List\x12\x10\n\x08is_tuple\x18\x01 \x01(\x08\x12\x15\n\x05value\x18\x02 \x03(\x0b\x32\x06.Value\x1an\n\x05Slice\x12\x0f\n\x07is_null\x18\x04 \x01(\x08\x12\x13\n\tstart_val\x18\x01 \x01(\x03H\x00\x12\x12\n\x08stop_val\x18\x02 \x01(\x03H\x01\x12\x12\n\x08step_val\x18\x03 \x01(\x03H\x02\x42\x07\n\x05startB\x06\n\x04stopB\x06\n\x04step\x1a>\n\x04\x44ict\x12\x19\n\x04keys\x18\x01 \x01(\x0b\x32\x0b.Value.List\x12\x1b\n\x06values\x18\x02 \x01(\x0b\x32\x0b.Value.List\x1a\x1e\n\x03Key\x12\x0b\n\x03key\x18\x01 \x01(\t\x12\n\n\x02id\x18\x02 \x01(\tB\x07\n\x05valueb\x06proto3')
TypeError: init() got an unexpected keyword argument 'serialized_options'
”

[qinxuye]

I have update protobuf, but also report error message.

Seems protobuf should be at least 3.4.0, please check what the protobuf version yours is.

[wjsi]

serialized_options is introduced in protobuf 3.6.x, therefore you need check the version of protobuf you installed.

[eagleLiu82]

protobuf 3.6.1 is ok.