C++ compilation issue on MacOSX with the new VTK (6 or 7)

[binarybottle]

When make is run within mindboggle/surface_cpp_tools/bin after running cmake, I get an odd error, where it is trying to access an old (10.7 vs. 10.11) SDK:

Lutchi:bin arno$ make
[ 7%] Built target FsSurfaceReader
[ 15%] Built target GradientComputer
[ 23%] Built target MeshAnalyser
[ 30%] Built target PointAreaComputer
[ 38%] Built target Overlap
[ 46%] Built target TravelDepth
[ 53%] Built target VtkFileEditor
make[2]: *** No rule to make target /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk/System/Library/Frameworks/AGL.framework', needed bytravel_depth/TravelDepthMain'. Stop.
make[1]: *** [travel_depth/CMakeFiles/TravelDepthMain.dir/all] Error 2
make: *** [all] Error 2

[binarybottle]

I was able to circumvent this by adding the following above the project() line in the top CMakeLists.txt file (from https://cmake.org/Bug/view.php?id=14695):

SET(MACOSX_DEPLOYMENT_TARGET ${DARWIN_MAJOR_SDK_VERSION}.${DARWIN_MINOR_SDK_VERSION})
SET(CMAKE_CXX_FLAGS "--sysroot ${CMAKE_OSX_SYSROOT} ${CMAKE_CXX_FLAGS}")
MESSAGE("Setting MACOSX_DEPLOYMENT_TARGET to '${MACOSX_DEPLOYMENT_TARGET}'.")
#SET(CMAKE_OSX_DEPLOYMENT_TARGET "")
#SET(CMAKE_OSX_SYSROOT /)

However, this generated a new error:

Scanning dependencies of target FsSurfaceReader
[ 3%] Building CXX object CMakeFiles/FsSurfaceReader.dir/FsSurfaceReader.cpp.o
In file included from /software/install/mindboggle/surface_cpp_tools/FsSurfaceReader.cpp:13:
In file included from /software/install/mindboggle/surface_cpp_tools/FsSurfaceReader.h:16:
In file included from /usr/local/include/vtk-6.0/vtkPolyData.h:58:
In file included from /usr/local/include/vtk-6.0/vtkPointSet.h:30:
In file included from /usr/local/include/vtk-6.0/vtkDataSet.h:41:
In file included from /usr/local/include/vtk-6.0/vtkDataObject.h:36:
In file included from /usr/local/include/vtk-6.0/vtkObject.h:42:
In file included from /usr/local/include/vtk-6.0/vtkObjectBase.h:44:
In file included from /usr/local/include/vtk-6.0/vtkIndent.h:25:
In file included from /usr/local/include/vtk-6.0/vtkSystemIncludes.h:36:
In file included from /usr/local/include/vtk-6.0/vtkIOStream.h:33:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/iostream:37:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:23:10: fatal error:
'unistd.h' file not found
#include <unistd.h>
^
1 error generated.
make[2]: *** [CMakeFiles/FsSurfaceReader.dir/FsSurfaceReader.cpp.o] Error 1
make[1]: *** [CMakeFiles/FsSurfaceReader.dir/all] Error 2
make: *** [all] Error 2

[binarybottle]

I get a new error now on MacOSX that I can't figure out:

Scanning dependencies of target FsSurfaceReader
[ 3%] Building CXX object CMakeFiles/FsSurfaceReader.dir/FsSurfaceReader.cpp.o
In file included from /software/mindboggle/surface_cpp_tools/FsSurfaceReader.cpp:13:
In file included from /software/mindboggle/surface_cpp_tools/FsSurfaceReader.h:16:
In file included from /usr/local/include/vtk-6.0/vtkPolyData.h:58:
In file included from /usr/local/include/vtk-6.0/vtkPointSet.h:30:
In file included from /usr/local/include/vtk-6.0/vtkDataSet.h:41:
In file included from /usr/local/include/vtk-6.0/vtkDataObject.h:36:
In file included from /usr/local/include/vtk-6.0/vtkObject.h:42:
In file included from /usr/local/include/vtk-6.0/vtkObjectBase.h:44:
In file included from /usr/local/include/vtk-6.0/vtkIndent.h:25:
In file included from /usr/local/include/vtk-6.0/vtkSystemIncludes.h:36:
In file included from /usr/local/include/vtk-6.0/vtkIOStream.h:33:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/iostream:37:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:23:10: fatal error:
'unistd.h' file not found
#include <unistd.h>
^
1 error generated.
make[2]: *** [CMakeFiles/FsSurfaceReader.dir/FsSurfaceReader.cpp.o] Error 1
make[1]: *** [CMakeFiles/FsSurfaceReader.dir/all] Error 2
make: *** [all] Error 2

[bcipolli]

Do you have Xcode installed? This answer says one issue is if it is not installed:
https://stackoverflow.com/questions/29357618/clang-fatal-error-unistd-h-file-not-found

Another finds finds the issue with XCode installed (but reports things work with gcc):
https://stackoverflow.com/questions/33793281/unistd-h-file-not-found-clang-and-os-x

[binarybottle]

Thank you, Ben. I just got an answer on the vtkusers mailing list:

David Gobbi wrote:

...you're building a binary package to share with others. I strongly advise against using these:

SET(CMAKE_OSX_DEPLOYMENT_TARGET "")
SET(CMAKE_OSX_SYSROOT /)

The resulting binary will only work with your own version of OS X or later. Your best bet is to name a reasonably old deployment target (e.g. 10.7) and SDK (e.g. 10.9).

CMAKE_OSX_DEPLOYMENT_TARGET:STRING=10.7
CMAKE_OSX_SYSROOT:STRING=macosx10.9

These will provide a binary package that will work on 10.7 or later.

What is the minimum supported OS X that the anaconda VTK supports? Ideally, you should use that version as your deployment target.

As for AGL, it might be missing from some SDKs because it is obsolete (modern programs should not be using it). However, it is in my 10.9 SDK, so it will probably also be in yours.

...

Using the macosx10.11 SDK is fine as long as it allows you to set this desired deployment target. So if you are able to compile with the following settings, then the result should run on OS X 10.7:

CMAKE_OSX_DEPLOYMENT_TARGET:STRING=10.7
CMAKE_OSX_SYSROOT:STRING=macosx10.11

I'm not sure how far back you can set the deployment target with this SDK, but if the 10.7 target doesn't compile, you can try 10.8 or 10.9.

If if gives errors for the AGL.framework, then make sure that your code has no calls to Carbon APIs.

[binarybottle]

David Thompson wrote to the vtkusers mailing list:

FWIW, I just upgraded my Xcode to 7.2 and projects which had previously been configured by CMake > suddenly started failing with this error (unistd.h file not found) because the MacOSX10.9 SDK directory was removed: only the 10.11 SDK is provided with Xcode 7.2. You can still set the deployment target to > 10.9 but I have not tried anything earlier.

[binarybottle]

David Gobbi wrote to the vtkusers mailing list:

I think that I've found out where the AGL link is coming from. It isn't via Carbon, it looks like it was linked directly into the VTK that you're using. And it looks like a bug within CMake itself.

CMake has a module called FindOpenGL.cmake that VTK uses to find OpenGL. It looks for both the OpenGL.framework and for the obsolete AGL.framework (which is only for Carbon, and is deprecated by Apple). If it finds both (e.g. with an older SDK and XCode 6 or earlier), then it uses both, even though AGL.framework is really not needed for any modern software.

This means that a VTK that is built with (for example) Xcode 6 and its SDKs becomes linked to AGL, because AGL.framework is present in those older SDKs. It's a private link, though, so I'm surprised that > it would cause the error. But you are building with Xcode 7 and you are seeing an error... that much is indisputable.

I suggest that you enable verbose error reporting so that you can see exactly what is being linked when the error occurs:

export VERBOSE=1
make
...

Use "unset VERBOSE" to turn off the verbosity.

[binarybottle]

Exchange on vtkusers@vtk.org mailing list with David Gobbi and others (Jan. 6-8, 2016):

me to David, vtkusers@vtk.org
Jan 6
Thank you all so much for your kind help. Unfortunately, I am still unable to compile the C++ code in my project on MacOSX, even after setting the following in my top-level CMakeLists.txt file:
SET(CMAKE_OSX_DEPLOYMENT_TARGET "10.7")

When I type:
$ export VERBOSE=1
$ make

I get the following even though I don't have /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk/ but instead have /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/ on my computer:

...
[ 57%] Building CXX object travel_depth/CMakeFiles/TravelDepthMain.dir/TravelDepthMain.cpp.o
…
make[2]: *** No rule to make target /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk/System/Library/Frameworks/AGL.framework', needed bytravel_depth/TravelDepthMain'. Stop.

make[1]: *** [travel_depth/CMakeFiles/TravelDepthMain.dir/all] Error 2

…

David Gobbi to me, vtkusers@vtk.org
Jan 6
Look for the following line in your CMakeCache.txt:
OPENGL_glu_LIBRARY:FILEPATH=/System/Library/Frameworks/AGL.framework
Change it to the following and then recompile:
OPENGL_glu_LIBRARY:FILEPATH=/System/Library/Frameworks/OpenGL.framework
That should keep the problematic AGL library out of your build.
…
Also see this related patch that was added to cmake itself two weeks ago:
https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=08580be2
…

me to David, vtkusers@vtk.org
Jan 6
There was no instance of "AGL" in CMakeCache.txt, but I replaced every instance of "AGL" with OpenGL" in all of the build.make files. Without cmake, but with make I got the following error:

make[2]: *** No rule to make target /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk/System/Library/Frameworks/OpenGL.framework', needed bytravel_depth/TravelDepthMain'. Stop.

It still calls 10.7 even though I have 10.11 installed on my Mac.
…

David Gobbi to me, vtkusers@vtk.org
Jan 6
It could be a stale dependency. Remove everything after the "INTERNAL cache entries" comment in your CMakeCache.txt, and also search your cache to make sure that 10.7 does not appear anywhere.
Also do an "ls -l /System/Library/Frameworks/" to make sure that the OpenGL framework isn't a link to a non-existent 10.7 SDK (though I don't see how that could happen).
…

me to David, vtkusers@vtk.org
Jan 6
When I clear the bin directory, then run "cmake .." and "make" I don't find any instance of 10.7 mentioned in any file, but still get "AGL.framework" in every subdirectory's build.make file. The CMakeCache.txt file contains the following line:

CMAKE_OSX_SYSROOT:PATH=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk

And yet I am still receiving the following error:
…
make[2]: *** No rule to make target /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk/System/Library/Frameworks/AGL.framework', needed bytravel_depth/TravelDepthMain'. Stop.
…

David Gobbi to me, vtkusers@vtk.org
Jan 6
Are you still building against the Anaconda python? The build has to be picking up the AGL and the 10.7 SDK from somewhere, and if it isn't in your CMakeCache.txt, then it must be coming from somewhere.
…

me to David, vtkusers@vtk.org
Jan 7
Yes, it should be building from miniconda2. However, I can't find any instance of "AGL" or of "10.7" anywhere in the code base (only within the bin directory after running cmake). I am baffled and stuck.
…

David Gobbi to me, vtkusers@vtk.org
Jan 7
You can try replacing FindOpenGL.cmake with the latest one from the cmake master branch:
https://cmake.org/gitweb?p=cmake.git;a=blob_plain;f=Modules/FindOpenGL.cmake
On my machine, for example, the file is located here:
/Applications/CMake.app/Contents/share/cmake-3.4/Modules/FindOpenGL.cmake
That might help with AGL.
…

me to David, vtkusers@vtk.org
Jan 7
That was an excellent suggestion, but alas, even after replacing that file with the newer one from the link you shared I am still getting the same error!
…

David Gobbi to me, vtkusers@vtk.org
Jan 7
There's also this patch to vtkOpenGL.cmake that you can try:
https://gitlab.kitware.com/vtk/vtk/commit/4a1fbefe
…

me to David, vtkusers@vtk.org
Jan 7
Thank you for the suggestion, David. I placed the vtkOpenGL.cmake file in
/Users/arno/Software/install/miniconda2/pkgs/cmake-3.3.1-0/share/cmake-3.3/Modules but still receive the same error.
…

David Gobbi to me, vtkusers@vtk.org
Jan 7
My fault, I should have mentioned that it needs to be in /CMake/vtkOpenGL.cmake in order be used during the VTK build.
…

me to David, vtkusers@vtk.org
Jan 7
If that is the same directory that FindOpenGL.cmake is in, then the path I sent should be correct, yes? I installed cmake using Anaconda: "conda install --yes cmake"
…

David Gobbi to me, vtkusers@vtk.org
Jan 7
It is not the same directory. FindOpenGL.cmake is part of cmake, but vtkOpenGL.cmake is part of VTK.
…

me to David, vtkusers@vtk.org
Jan 7
Thank you! I am afraid I still don't know where it goes, given the conda installation of vtk doesn't have a CMake subdirectory (miniconda2/pkgs/vtk-6.3.0-py27_1/). Any suggestions how to proceed?
…

David Gobbi to me, vtkusers@vtk.org
Jan 7
I apologize for that... I had forgotten that you were using a binary distribution of VTK rather than building VTK from source. My recent thinking was that the only binary piece you were using from Anaconda was python itself, I didn't go back and re-read our entire conversation.

This means that all of my recent advice to you has been totally irrelevant. (It would be very relevant to the Anaconda folks if they plan to put out a new vtk package soon... they could save people like yourself a lot of difficulty.)

If you're using a binary vtk, then you'll have to dig into the cmake exports for VTK and perform some surgical changes to them. Somewhere in your Anaconda VTK, there should be a directory called the following:

cmake/vtk-6.3

Once you find that directory, "cd" into it, and it should be full of cmake files. These cmake files list the dependencies for your VTK package. If you edit these files to remove all references to "AGL" and the problematic SDK, then your compile difficulties should go away. For example, use this to find all the files that reference AGL:

find . -name "*.cmake" -exec fgrep -l AGL {} +

But it might be easier if you find a Mac with OS X 10.9 and Xcode 6.1, and then re-do your build there.
…

me to David, vtkusers@vtk.org
Jan 7
Thank you again, David.

The find command turned up one file within the miniconda2/lib/cmake/vtk-6.3 directory: "VTKTargets.cmake"
Within this file, I commented out lines with AGL and 10.7 (see below), but I still get the same error!

@rno: Removed AGL and SDK 10.7 as per David Gobbi's suggestion:

find . -name "*.cmake" -exec fgrep -l AGL {} +

set_target_properties(vtkRenderingOpenGL PROPERTIES
INTERFACE_LINK_LIBRARIES "vtkRenderingCore;-framework Cocoa"
)
#set_target_properties(vtkRenderingOpenGL PROPERTIES

INTERFACE_LINK_LIBRARIES "vtkRenderingCore;/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk/System/Library/Frameworks/AGL.framework;/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk/System/Library/Frameworks/OpenGL.framework;-framework Cocoa"

#)

Create imported target vtkRenderingContextOpenGL

add_library(vtkRenderingContextOpenGL SHARED IMPORTED)
set_target_properties(vtkRenderingContextOpenGL PROPERTIES
INTERFACE_LINK_LIBRARIES "vtkRenderingContext2D;vtkRenderingOpenGL"
)

Create imported target vtkgl2ps

add_library(vtkgl2ps SHARED IMPORTED)

@rno: Removed AGL and SDK 10.7 as per David Gobbi's suggestion:

find . -name "*.cmake" -exec fgrep -l AGL {} +

set_target_properties(vtkgl2ps PROPERTIES
INTERFACE_LINK_LIBRARIES "vtkzlib;vtkpng;"
)
#set_target_properties(vtkgl2ps PROPERTIES

INTERFACE_LINK_LIBRARIES "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk/System/Library/Frameworks/AGL.framework;/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.7.sdk/System/Library/Frameworks/OpenGL.framework;m;vtkzlib;vtkpng;m"

#)
…

David Gobbi to me, vtkusers@vtk.org
Jan 8
The fact that a specific SDK has been hard-coded into all those dependency paths is very, very ugly. The Anaconda folks should know better than that.

You can't get away with just commenting out those lines. Instead, I recommend that you do the following search and replace within each of those files:

1. replace 10.7.sdk with an SDK that you actually have on your system
2. replace AGL.framework with OpenGL.framework

However, the fact that you got the same error after commenting out those lines, instead of getting a different error, suggests that your build did not pick up the change. After changing the files, did you erase your build directory and do a full rebuild?
…

me to David, vtkusers@vtk.org
Jan 8
I made the edits you recommended (AGL to OpenGL and 10.7 to 10.11) in all miniconda2 files and rebuilt my mindboggle package, and still get the same error! Should I have done something more with miniconda2's cmake or vtk?
…

David Gobbi to me, vtkusers@vtk.org
Jan 8
I'm afraid you're on your own at this point. I have no real experience with Anaconda, all my experience is purely with CMake and VTK, and I'm not going to go so far as to install miniconda on my own system to investigate.
…

me to David, vtkusers@vtk.org
Jan 8
Thank you for all of your help, David! I really appreciate it. I have written to the anaconda forum and will tell you what I learn...

[binarybottle]

I am not able to install Mindboggle on MacOSX 10.11 with VTK 6 yet, and I don't wish to revert to the deprecated VTK 5, so for now I will work on the Linux installation, which doesn't have the same problems.

[binarybottle]

I updated Mindboggle to run on Python 3 with VTK 7, hoping that these changes would overcome the VTK 5/6 issues we're getting on MacOSX, but alas I get the following error:

Standard error:
dyld: Library not loaded: @rpath/./libvtkDomainsChemistryOpenGL2-7.0.1.dylib
Referenced from: /software/mindboggle/vtk_cpp_tools/bin/travel_depth/TravelDepthMain
Reason: image not found
Return code: -5
Interface CommandLine failed to run.
Interface Function failed to run.