Mac port #100
Mac port #100
Conversation
This allows in-place builds in a directory with spaces in its path to compile. Out-of-place builds in paths with spaces would require changes to the Makefile too, but this is unfeasible due to the lack of support for spaces in the built-in make commands (such as "include", "wildcard", etc.).
|
Hi @jpauwels, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution! TTYL, MSBOT; |
|
@jpauwels, Thanks for signing the contribution license agreement so quickly! Actual humans will now validate the agreement and then evaluate the PR. |
…ent missing symbols during linking
… prevent double defined symbols during linking
jpauwels
force-pushed
the
jpauwels-mac-port
branch
from
5978652
to
1a2b111
Compare
|
I already tested the changes on Windows myself (see AppVeyor CI) and after a minor change, everything now compiles under Windows, OS X and Linux (still CPU-only though and based on f628e52). |
|
is this going to be merged? i'd like to use CNTK on my mac! |
|
Any update on this issue? |
|
As long as we don't have infrastructure to automatically run nightly tests on OSX, this is extremely hard to do. |
|
Thanks for finally acknowledging the existence of this pull request, it's only nine months old. If CI testing is the only problem, I made a Travis configuration file too as you can see in my first message. At the time I wrote it, Linux compilation was in a perpetual broken state as well, so your nightly tests weren't working anyway. This PR is probably too old now to be salvaged and I have sort of moved on to competing deep learning toolkits. I might be persuaded to try another mac port starting from the current head if there's enough interest and a chance it might be merged. |
|
We use Jenkins, not Travis, on our own infrastructure. In the near future we will move Jenkins to Azure. At that point it will be a matter of Azure having OSX VMs. So it will probably take a long time. |
|
Travis is free for open-source projects. So why not use it in addition to your own Jenkins? I can provide you with the configuration file. |
|
Free Travis for OSX related build and test sounds interesting. Incorporating that into our current Jenkins based flow is not something we have discussed or planned up till now. We'd need to figure out GPU support, etc... We look into viability in our next sprint. Some OSX users have been doing CNTK inside Docker containers (no GPU). |
Apple allows VMs of it's OSX on Apple Hardware Only. Unless MS gets Mac Hardware to run OSX for Azure, there's probably no other way to expect a VM anytime soon. |
|
@jpauwels For whatever it's worth I'm personally trying to get CNTK CPU only running on macOS again. Right now I'm not worrying about OpenCV or Python support but I plan on getting to them. I just finished getting the key dependencies built on macOS (e.g. C++ 4.8.4, Intel's MKL, OpenMPI, Zlib, LibZip, Protobuf and Boost) and am now going to use your changes as a guide to changing the latest version of CNTK and seeing if I can get things building. |
build-macos.sh - Some minor changes to get CustomMKL to build on the mac. I should probably combine this with build-linux.sh. headers.txt - Had to add dnn headers needed by the newer version of MKL I have to use because nobody knows how to download the mac version of the old version that CNTK is currently using For the next two files I'm just lifting things out of microsoft#100 MakeFile - Making the file output so or dylib depending on platform configure - Had to change bash ! header to deal with the fact that Apple ships bash 3 and we need bash 4. This change in ! let's us install bash 4 on the mac in a way that shouldn't mess with linux (I tested it and it seemed fine). Needed to change 'stat' to use -c on the mac not -f which isn't supported.
A huge debt of gratitude to microsoft#100 from which I stole shamelessly. build-macos.sh - Make it use environment bash to match other changes we made to .sh files. Also change the TAR file name to include macOS not Linux Makefile - Added -mmacosx-version-min=10.7 and -D_XOPEN_SOURCE=700 so we can support wcscasecmp and wcsncasecmp functions. But kept -D_XOPEN_SOURCE =600 (as it was before) for everyone else. File.cpp - limits.h has a different location on Apple than Linux so I had to fix the path with a macro. Platform.h - We needded to add strings.h and wchar.h. The changes to mach_time & snprintf are taken straight from PR 100 in CNTK. All files not explicitly called out had some combination of the following three changes: 1.Changed malloc.h to malloc/malloc.h 2. Removing ifndef __unix__ and replacing with ifdef _WIN32 3. Added __APPLE__ to #ifdef __unix__
|
We are very interested in having Mac support for CNTK now that CNTK 2.0 has been released. Travis build would be fine for us. If anyone is willing to help, please! |
|
This pull request has been too old to merge with the master. Closing it now. As I mentioned, we are still interested in MAC support, and if someone could work on it, we will make sure to handle the PR as soon as possible. |
These change made it possible for me to build CNTK on OS X 10.11.3. Just a single commit deals with Mac-specific changes, the others correct non-standard linking behaviour that depends on luck and compiler implementation details (I'm surprised CNTK even links on Windows and Linux).
I based this PR on f628e52 because the Linux build is currently broken in master. I'll rebase this PR as soon as this is fixed.
I could only test the CPU-only version because I don't have an Nvidia GPU. On the other hand, I set up CI to test the changes with GCC 4.9 and 5.3 both on Linux and Mac, so I'm pretty confident they don't break anything (see Travis CI). Now I'd like someone to verify on a Windows machine.
In order to build on a Mac, you need an
mpic++that is built with GCC, which doesn't come standard with OS X. If you have homebrew, you can do:brew install gcc --without-multiliband thenbrew install --cc=gcc openmpi. You also need Intel MKL, because ACML is not available for Mac, but the former is freely available too (since recently).