Problem with installing on macOS Catalina

[AneesHl]

when I run

env CC=gcc sudo python3 setup.py install

I get the following error:

running install
running bdist_egg
running egg_info
writing disco_dop.egg-info/PKG-INFO
writing dependency_links to disco_dop.egg-info/dependency_links.txt
writing entry points to disco_dop.egg-info/entry_points.txt
writing requirements to disco_dop.egg-info/requires.txt
writing top-level names to disco_dop.egg-info/top_level.txt
reading manifest file 'disco_dop.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '.c' under directory 'discodop'
warning: no previously-included files matching '.pyx' found under directory 'discodop'
warning: no previously-included files matching '.pxi' found under directory 'discodop'
warning: no previously-included files matching '.pxd' found under directory 'discodop'
warning: no previously-included files matching '*' found under directory 'web/grammars'
writing manifest file 'disco_dop.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.9-x86_64/egg
running install_lib
running build_py
running build_ext
building 'discodop._fragments' extension
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch x86_64 -g -Idiscodop -I./discodop -I/Library/Frameworks/Python.framework/Versions/3.8/include/python3.8 -c discodop/_fragments.cpp -o build/temp.macosx-10.9-x86_64-3.8/discodop/_fragments.o -Isparsepp/ -Wno-strict-prototypes -Wno-unused-function -Wno-unreachable-code -Wno-sign-compare -D__STDC_LIMIT_MACROS -O3 -march=native -DNDEBUG
In file included from discodop/_fragments.cpp:664:
In file included from discodop/_containers.h:11:
In file included from discodop/../sparsepp/sparsepp/spp.h:65:
discodop/../sparsepp/sparsepp/spp_utils.h:87:17: fatal error: 'tr1/unordered_map' file not found
#include <tr1/unordered_map>
^~~~~~~~~~~~~~~~~~~
1 error generated.
error: command 'gcc' failed with exit status 1

I found this hint online, but I don't know how to implement it or if it'll actually work:

As of macOS 10.12 Sierra, the default is now libc++ and libstdc++ is deprecated. libstdc++ is quite old, v4.2.1, and predates C++11 (hence the tr1 headers). If you're going to be using this code long-term, it'd be worth the time to at least make it C++11 compliant (i.e. #include <unordered_map>)

Update: Xcode 10 no longer allows building against libstdc++. Either update your codebase to use standard C++11 headers, or use Xcode 9 if that's really not an option.

Any help would be much appreciated. Thnx

[andreasvc]

Can you check whether this fixes your issue: greg7mdp/sparsepp#40

[AneesHl]

unfortunately not... The Problem starts already at the "<tr1" part:

thank you for the answer, regardless.

[andreasvc]

I have updated sparsepp to the latest version now. Use git pull && git submodule update --remote sparsepp to get it. If that doesn't solve the problem, it's an issue that should be reported to sparsepp.

[AneesHl]

Now I'm getting a different error:

==========
...
running build_ext
building 'discodop._fragments' extension
creating build/temp.macosx-10.7-x86_64-3.7
creating build/temp.macosx-10.7-x86_64-3.7/discodop
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/anees123/opt/anaconda3/include -arch x86_64 -I/Users/anees123/opt/anaconda3/include -arch x86_64 -Idiscodop -I./discodop -I/Users/anees123/opt/anaconda3/include/python3.7m -c discodop/_fragments.cpp -o build/temp.macosx-10.7-x86_64-3.7/discodop/_fragments.o -Isparsepp/ -Wno-strict-prototypes -Wno-unused-function -Wno-unreachable-code -Wno-sign-compare -D__STDC_LIMIT_MACROS -O3 -march=native -DNDEBUG
clang: warning: include path for libstdc++ headers not found; pass '-stdlib=libc++' on the command line to use the libc++ standard library instead [-Wstdlibcxx-not-found]
discodop/_fragments.cpp:654:10: fatal error: 'ios' file not found
#include "ios"
^~~~~
1 error generated.
error: command 'gcc' failed with exit status 1

==========
thanks again

[andreasvc]

Well this is progress :)
I did some googling on the warning, and it seems like you might be able to fix it by adding the option '-stdlib=libc++' in setup.py to the compile and link arguments:

cython/cython#2694 (comment)

Does that work?

[andreasvc]

The latest commit should be a proper fix.

[AneesHl]

thanks for your answers.
Unfortunately both solutions didn't fix the problem. I'm getting the first error again:

=========

...
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/anees123/opt/anaconda3/include -arch x86_64 -I/Users/anees123/opt/anaconda3/include -arch x86_64 -Idiscodop -I./discodop -I/Users/anees123/opt/anaconda3/include/python3.7m -c discodop/_fragments.cpp -o build/temp.macosx-10.9-x86_64-3.7/discodop/_fragments.o -Isparsepp/ -Wno-strict-prototypes -Wno-unused-function -Wno-unreachable-code -Wno-sign-compare -D__STDC_LIMIT_MACROS -O3 -march=native -DNDEBUG
In file included from discodop/_fragments.cpp:664:
In file included from discodop/_containers.h:11:
In file included from discodop/../sparsepp/sparsepp/spp.h:65:
discodop/../sparsepp/sparsepp/spp_utils.h:87:17: fatal error: 'tr1/functional' file not found
#include <tr1/functional>
^~~~~~~~~~~~~~~~
1 error generated.
error: command 'gcc' failed with exit status 1

=========

[AneesHl]

the guys from sparsepp gave a solution to the original error:

replace #if //__has_feature(cxx_noexcept) with #if 1

but now I'm getting this similar error but not in sparsepp:

[andreasvc]

I guess it's the same issue again. Looking at btree_config.h, you could define CPP_BTREE_CXX11 by passing -DCPP_BTREE_CXX11 as compile argument in setup.py, or replace that condition with 1 as in the suggested solution for sparsepp.

[AneesHl]

Thank you!

Adding '-DCPP_BTREE_CXX11' to the compile arguments in setup.py fixed the problem. ;)

[nschneid]

I encountered this issue as well and it took me awhile to find this bug. Could you please add an explanation or link from the Mac section of the README? Thanks

[andreasvc]

Thanks for the reminder. I committed a fix; however, I do not have a mac to test with. Could you confirm for me that this fixes the issue? Thanks in advance.

[AneesHl]

Thanks for the reminder. I committed a fix; however, I do not have a mac to test with. Could you confirm for me that this fixes the issue? Thanks in advance.

Hi,
no your commit doesn't fix the problem yet, because I've had to do a second change to get the installation to work.
The second one (as I wrote above) is to change line 83 of 'disco-dop/sparsepp/sparsepp/spp_utils.h' from
#if __has_feature(cxx_noexcept) to #if 1. Although I'm not sure whether this would cause other problems on other platforms, on macOS Ventura it works (with some warnings).

Let me know if you need more details.

warnings.log

[nschneid]

I had to make both changes on macOS Monterey.

[andreasvc]

@AneesHl @nschneid thanks for testing.
I forgot about my own suggestion at greg7mdp/sparsepp#88 (comment)
I have updated the version of sparsepp and added a define option.
Use git pull && git submodule update --remote sparsepp to get the updated version.

[nschneid]

Thanks. @BrettRey is trying to install on his M1 MacBook and is running into the following issue with the roaringbitmap library (in requirements.txt)—any suggestions? https://stackoverflow.com/questions/65966969/why-does-march-native-not-work-on-apple-m1/66333485

[nschneid]

I see there is also https://github.com/Ezibenroc/PyRoaringBitMap in case that helps

[AneesHl]

@AneesHl @nschneid thanks for testing.
I forgot about my own suggestion at greg7mdp/sparsepp#88 (comment)
I have updated the version of sparsepp and added a define option.
Use git pull && git submodule update --remote sparsepp to get the updated version.

I can now confirm that installation on my Intel-mac now works (aside from the warnings as in the log file attached to my previous comment.)

Unfortunately, I don't have an M1-mac to try to reproduce the issue that @BrettRey is having.

[andreasvc]

@nschneid feel free to open an issue in the roaringbitmap repo. Perhaps python3 setup.py install --with-mtune would solve that issue.

[BrettRey]

$ sudo /opt/homebrew/bin/pip3 install --user --with-mtune roaringbitmap

Usage:   
  pip3 install [options] <requirement specifier> [package-index-options] ...
  pip3 install [options] -r <requirements file> [package-index-options] ...
  pip3 install [options] [-e] <vcs project url> ...
  pip3 install [options] [-e] <local project path> ...
  pip3 install [options] <archive url/path> ...

no such option: --with-mtune

[andreasvc]

Yes, pip has no such option, I was referring to setup.py of roaringbitmap. If you encounter further issues, let's discuss them in an issue on the roaringbitmap repo.

[nschneid]

Sorry I'm confused - do you mean https://github.com/RoaringBitmap/RoaringBitmap? I don't see setup.py there.

[andreasvc]

I mean https://github.com/andreasvc/roaringbitmap which is listed in requirements.txt. The others are implementations of the same data structure with different APIs.