getting theano working out of the box with mingwpy (on python3.4, then friends)

[stonebig]

Steps:

• include a theano wheel,
• auto-create theano-cache.bat , theano-nose.bat, theano-test.bat
• passing the basic test
• integrate it back in default build
• include a theano test in winpython-checker
• lasagne wheel (may be square, we'll see)
• add .theanorc for [blas] ldflags = -lopenblaspy

[hiccup7]

OpenBLAS v0.2.14 is several times slower than MKL BLAS on use cases important to Theano users. The OpenBLAS team has committed some changes, but has not yet been able to resolve bugs. Since WinPython already uses MKL BLAS in NumPy+MKL, could WinPython use MKL BLAS with Theano?

[stonebig]

MKL is not free, Theano can only use MKL via the numpy dll compiled by Christoph Gohlke.
Can you point me the OpenBLAS open issues you are speaking of ?

[hiccup7]

I agree. Would the Theano team need to change their code to support this? Or can you just change WinPython build options to make Theano use MKL in the NumPy DLL?

[stonebig]

These questions are rather for Theano team.

[carlkl]

I guess you need VisualC and the Intel Fortran compiler to use MKL from Theano.
Which use cases are several times slower compared to MKL? Is this true for the latest development version?

[hiccup7]

See https://github.com/xianyi/OpenBLAS/issues
I tested #530 & #532 on the latest develop branch, and they are still several times slower compared to MKL.

[stonebig]

Until I can get Theano working with 'basic' mingwpy, these are just theorical issues to me.

For the moment Theano documentation is written for Winpython+TDM_GCC + Theano setup.py, and I don't know really know how to make it work with Winpython+mingwpy+Theano wheel+ my building method.

(help/procedure welcomed)

with Previous example, I get this:

Problem occurred during compilation with the command line below:
D:\result_tests\winpython-3.4.3.amd64_build1\python-3.4.3.amd64\..\tools\mingw32\bin\g++.exe -shared -g -O3 -fno-math-errno -Wno-unused-label -Wno-unused-variable -Wno-write-strings -march=westmere -mmmx -mno-3dnow -msse -msse2 -msse3 -mssse3 -mno-sse4a -mcx16 -msahf -mno-movbe -mno-aes -mno-sha -mno-pclmul -mpopcnt -mno-abm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-bmi2 -mno-tbm -mno-avx -mno-avx2 -msse4.2 -msse4.1 -mno-lzcnt -mno-rtm -mno-hle -mno-rdrnd -mno-f16c -mno-fsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mno-xsave -mno-xsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=3072 -mtune=westmere -D NPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -DMS_WIN64 -ID:\result_tests\winpython-3.4.3.amd64_build1\python-3.4.3.amd64\lib\site-packages\numpy\core\include -ID:\result_tests\winpython-3.4.3.amd64_build1\python-3.4.3.amd64\include -o C:\Users\famille\AppData\Local\Theano\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_37_Stepping_2_GenuineIntel-3.4.3-64\tmpz5scbbls\m192d61d4f9962ed45f35313cca04a163.pyd C:\Users\famille\AppData\Local\Theano\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_37_Stepping_2_GenuineIntel-3.4.3-64\tmpz5scbbls\mod.cpp -LD:\result_tests\winpython-3.4.3.amd64_build1\python-3.4.3.amd64\libs -LD:\result_tests\winpython-3.4.3.amd64_build1\python-3.4.3.amd64 -lpython34
Traceback (most recent call last):

  File "<ipython-input-1-3071dfac33ef>", line 6, in <module>
    f = function([x, y], z)

  File "D:\result_tests\winpython-3.4.3.amd64_build1\python-3.4.3.amd64\lib\site-packages\theano\compile\function.py", line 266, in function
    profile=profile)

  File "D:\result_tests\winpython-3.4.3.amd64_build1\python-3.4.3.amd64\lib\site-packages\theano\compile\pfunc.py", line 511, in pfunc
    on_unused_input=on_unused_input)

  File "D:\result_tests\winpython-3.4.3.amd64_build1\python-3.4.3.amd64\lib\site-packages\theano\compile\function_module.py", line 1466, in orig_function
    defaults)

  File "D:\result_tests\winpython-3.4.3.amd64_build1\python-3.4.3.amd64\lib\site-packages\theano\compile\function_module.py", line 1324, in create
    input_storage=input_storage_lists)

  File "D:\result_tests\winpython-3.4.3.amd64_build1\python-3.4.3.amd64\lib\site-packages\theano\gof\link.py", line 519, in make_thunk
    output_storage=output_storage)[:3]

  File "D:\result_tests\winpython-3.4.3.amd64_build1\python-3.4.3.amd64\lib\site-packages\theano\gof\vm.py", line 897, in make_all
    no_recycling))

  File "D:\result_tests\winpython-3.4.3.amd64_build1\python-3.4.3.amd64\lib\site-packages\theano\gof\op.py", line 1002, in make_thunk
    compute_map, no_recycling)

  File "D:\result_tests\winpython-3.4.3.amd64_build1\python-3.4.3.amd64\lib\site-packages\theano\gof\op.py", line 739, in make_thunk
    output_storage=node_output_storage)

  File "D:\result_tests\winpython-3.4.3.amd64_build1\python-3.4.3.amd64\lib\site-packages\theano\gof\cc.py", line 1073, in make_thunk
    keep_lock=keep_lock)

  File "D:\result_tests\winpython-3.4.3.amd64_build1\python-3.4.3.amd64\lib\site-packages\theano\gof\cc.py", line 1015, in __compile__
    keep_lock=keep_lock)

  File "D:\result_tests\winpython-3.4.3.amd64_build1\python-3.4.3.amd64\lib\site-packages\theano\gof\cc.py", line 1442, in cthunk_factory
    key=key, lnk=self, keep_lock=keep_lock)

  File "D:\result_tests\winpython-3.4.3.amd64_build1\python-3.4.3.amd64\lib\site-packages\theano\gof\cmodule.py", line 1076, in module_from_key
    module = lnk.compile_cmodule(location)

  File "D:\result_tests\winpython-3.4.3.amd64_build1\python-3.4.3.amd64\lib\site-packages\theano\gof\cc.py", line 1354, in compile_cmodule
    preargs=preargs)

  File "D:\result_tests\winpython-3.4.3.amd64_build1\python-3.4.3.amd64\lib\site-packages\theano\gof\cmodule.py", line 2010, in compile_str
    (status, compile_stderr.replace('\n', '. ')))

Exception: ('The following error happened while compiling the node', Elemwise{add,no_inplace}(x, y), '\n', "Compilation failed (return status=1): D:/result_tests/winpython-3.4.3.amd64_build1/tools/mingw32/bin/../lib/gcc/x86_64-w64-mingw32/4.9.2/libstdc++.a(basic_file.o):(.text$_ZN12_GLOBAL__N_1L6xwriteEiPKcx+0x29): undefined reference to `write'\r. D:/result_tests/winpython-3.4.3.amd64_build1/tools/mingw32/bin/../lib/gcc/x86_64-w64-mingw32/4.9.2/libstdc++.a(basic_file.o):(.text$_ZNSt12__basic_fileIcE8sys_openEiSt13_Ios_Openmode+0x36): undefined reference to `fdopen'\r. D:/result_tests/winpython-3.4.3.amd64_build1/tools/mingw32/bin/../lib/gcc/x86_64-w64-mingw32/4.9.2/../../../../x86_64-w64-mingw32/bin/ld.exe: D:/result_tests/winpython-3.4.3.amd64_build1/tools/mingw32/bin/../lib/gcc/x86_64-w64-mingw32/4.9.2/libstdc++.a(basic_file.o): bad reloc address 0x36 in section `.text$_ZNSt12__basic_fileIcE8sys_openEiSt13_Ios_Openmode'\r. collect2.exe: error: ld returned 1 exit status\r. ", '[Elemwise{add,no_inplace}(x, y)]')

[stonebig]

Could this be the problem ?

http://stackoverflow.com/questions/4070015/undefined-reference-when-linking-c-code-in-linux

The functions open, read, write, close, etc. have underscores in front of their names on Windows, but they do not on Linux

[stonebig]

seeeing this:
Theano/Theano#2733

[stonebig]

@hiccup7
I now understand your GEMM concern:
http://petewarden.com/2015/04/20/why-gemm-is-at-the-heart-of-deep-learning/

[carlkl]

theano is working with plain python + mingwpy + openblas when using the openblas compiled numpy. In this case theano is using the blas information from numpy.distutils. checked with "python check_blas.py" in heano/misc.
Obviously this fails using numpy-MKL as in WinPython. I have to learn how to setup ${HOME}/.theanorc to make this work in WinPython.

[stonebig]

with plain python 3.4 on windows ? (all post I read on Theano are still using winpython2.7.9.x )