Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3.6-3.8 cannot be installed on Mac OS 11.0.1: "implicit declaration of function 'sendfile' is invalid in C99" #1737

Closed
nightzjp opened this issue Nov 20, 2020 · 33 comments
Labels
by-design the reported behavior is as intended third-party the problem is in third-party software

Comments

@nightzjp
Copy link

Hi,Morning:
First of all, thank you very much for thinking about and maintaining pyenv. We found problems installing pypypy3 via pyenv, but we were also able to find solutions we would like to share with you.

Cordial greetings
jianping zhang

environment

  • macOS 11.0.1
  • pyenv 1.2.20

describe

  • Error in installing Python on MacOS and pyenv

`BUILD FAILED (OS X 11.0.1 using python-build 20180424)

Inspect or clean up the working tree at /tmp/python-build.20201120091233.71993
Results logged to /tmp/python-build.20201120091233.71993.log

Last 10 log lines:
./Modules/posixmodule.c:8210:15: error: implicit declaration of function 'sendfile' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
ret = sendfile(in, out, offset, &sbytes, &sf, flags);
^
./Modules/posixmodule.c:10432:5: warning: code will never be executed [-Wunreachable-code]
Py_FatalError("abort() called from Python code didn't abort!");
^~~~~~~~~~~~~
1 warning and 1 error generated.
make: *** [Modules/posixmodule.o] Error 1
make: *** Waiting for unfinished jobs....
1 warning generated`

@EthanYan6
Copy link

❯ pyenv install 3.6.8
python-build: use openssl@1.1 from homebrew
python-build: use readline from homebrew
Downloading Python-3.6.8.tar.xz...
-> https://www.python.org/ftp/python/3.6.8/Python-3.6.8.tar.xz
Installing Python-3.6.8...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk

BUILD FAILED (OS X 11.0.1 using python-build 20180424)

Inspect or clean up the working tree at /var/folders/bb/0bxy1cy57rb_pxdkvlyst3nh0000gn/T/python-build.20201120103846.92208
Results logged to /var/folders/bb/0bxy1cy57rb_pxdkvlyst3nh0000gn/T/python-build.20201120103846.92208.log

Last 10 log lines:
struct sf_hdtr sf;
^
./Modules/posixmodule.c:8210:15: error: implicit declaration of function 'sendfile' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
ret = sendfile(in, out, offset, &sbytes, &sf, flags);
^
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -I. -I./Include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl@1.1/include -I/Users/ethanyan/.pyenv/versions/3.6.8/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl@1.1/include -I/Users/ethanyan/.pyenv/versions/3.6.8/include -I/usr/local/opt/openssl@1.1/include -DPy_BUILD_CORE -c ./Modules/_codecsmodule.c -o Modules/_codecsmodule.o
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -I. -I./Include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl@1.1/include -I/Users/ethanyan/.pyenv/versions/3.6.8/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl@1.1/include -I/Users/ethanyan/.pyenv/versions/3.6.8/include -I/usr/local/opt/openssl@1.1/include -DPy_BUILD_CORE -c ./Modules/_weakref.c -o Modules/_weakref.o
2 errors generated.
make: *** [Modules/posixmodule.o] Error 1
make: *** Waiting for unfinished jobs....

@0xSeanll
Copy link

0xSeanll commented Nov 22, 2020

I solved this issue by the method provided by this blog.

Step 1 Align command-line tools

Open Xcode-beta.app
Go to Preference > Locations
Select the right version of command-line tools

Step 2 Install python

In this case, I installed 3.8.0. If you want to install a different version, you will need to change the version in the following command.
CFLAGS="-I$(brew --prefix openssl)/include -I$(brew --prefix bzip2)/include -I$(brew --prefix readline)/include -I$(xcrun --show-sdk-path)/usr/include" LDFLAGS="-L$(brew --prefix openssl)/lib -L$(brew --prefix readline)/lib -L$(brew --prefix zlib)/lib -L$(brew --prefix bzip2)/lib" pyenv install --patch 3.8.0 < <(curl -sSL https://github.com/python/cpython/commit/8ea6353.patch\?full_index\=1)

@gel-bcegielka
Copy link

gel-bcegielka commented Jan 22, 2021

Hit the exact error:

./Modules/posixmodule.c:8210:15: error: implicit declaration of function 'sendfile' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
ret = sendfile(in, out, offset, &sbytes, &sf, flags);

when trying to install python 3.6.12

QDH for me which worked was to turn off the check:

> CFLAGS=-Wno-implicit-function-declaration pyenv install 3.6.12
Installed Python-3.6.12 to /Users/myuser/.pyenv/versions/3.6.12

@mackinleysmith
Copy link

  1. install python 3.6.0, you can change the version
  CFLAGS="-I$(brew --prefix openssl)/include -I$(brew --prefix bzip2)/include -I$(brew --prefix readline)/include -I$(xcrun --show-sdk-path)/usr/include" LDFLAGS="-L$(brew --prefix openssl)/lib -L$(brew --prefix readline)/lib -L$(brew --prefix zlib)/lib -L$(brew --prefix bzip2)/lib" pyenv install --patch 3.6.0 < <(curl -sSL https://github.com/python/cpython/commit/8ea6353.patch\?full_index\=1)

In case it's useful to anyone else, here's how to do this in fish shell:

CFLAGS="-I"(brew --prefix openssl)"/include -I"(brew --prefix bzip2)"/include -I"(brew --prefix readline)"/include -I"(xcrun --show-sdk-path)"/usr/include" LDFLAGS="-L"(brew --prefix openssl)"/lib -L"(brew --prefix readline)"/lib -L"(brew --prefix zlib)"/lib -L"(brew --prefix bzip2)"/lib" pyenv install --patch 3.6.8 < (curl -sSL https://github.com/python/cpython/commit/8ea6353.patch\?full_index\=1 | psub)

@aavisee
Copy link

aavisee commented Feb 18, 2021

@gawbul

This worked for me on macOS BigSur (11.2) for Python 3.6.12:

brew install bzip2 lbzip2 lzlib openssl zlib
LDFLAGS="-L/usr/local/opt/bzip2/lib -L/usr/local/opt/zlib/lib -L/usr/local/opt/openssl@1.1/lib" CFLAGS="-I/usr/local/opt/bzip2/include -I/usr/local/opt/zlib/include -I/usr/local/opt/openssl@1.1/include -I$(xcrun --show-sdk-path)/usr/include -Wno-implicit-function-declaration" pyenv install 3.6.12

For me this bit of code only works from 3.5.3 and up. Anything older than Python 3.5.2 doesn't work on my Mac.

@lonniev
Copy link

lonniev commented Feb 25, 2021

For the people here who were able to get pyenv to install versions of python on macOS Big Sur, were you, after that, then able to get pip within that installed python to install python modules that involve compiling C/C++ code?

I can get the installs of python to complete but the pip module installer then has similar C/C++ compilation errors.

@phrfpeixoto
Copy link

Is there a way of fixing this without having such a massive command to install a new peenv? Like a way to patch the binary itself?

@native-api
Copy link
Member

native-api commented Mar 9, 2021

Python versions without python/cpython#21113 and its backports are incompatible with MacOS 11 and cannot be built on it.

Releases with the fix are: 3.7.8+, 3.8.4+, 3.9.0+

Any earlier versions can't be built.

@wrp
Copy link

wrp commented Mar 22, 2021

Note that if you are applying modifications to CFLAGS or using a patch (or indeed probably if you are just doing a straight pyenv install!), you probably will need to ensure that CONFIG_SITE is not set in your environment. So perhaps you need to do:

CONFIG_SITE= CFLAGS="-I$(brew --prefix openssl)/include -I$(brew --prefix bzip2)/include -I$(brew --prefix readline
)/include -I$(xcrun --show-sdk-path)/usr/include" LDFLAGS="-L$(brew --prefix openssl)/lib -L$(brew --prefix readline)/lib -L$(brew --prefix zlib)/lib -L$(brew --prefix bzip2)/l
ib" pyenv install --patch 3.8.1 < <(curl -sSL https://github.com/python/cpython/commit/8ea6353.patch\?full_index\=1)

@purifyyourhome
Copy link

From the blog post in this comment and the above comments, got Python 3.6.13 to build on macOS 11.3! The following homebrew packages were installed first:

brew install bzip2 openssl@1.1 readline xz

CFLAGS="-I$(brew --prefix openssl)/include -I$(brew --prefix bzip2)/include -I$(brew --prefix readline)/include -I$(xcrun --show-sdk-path)/usr/include" LDFLAGS="-L$(brew --prefix openssl)/lib -L$(brew --prefix readline)/lib -L$(brew --prefix zlib)/lib -L$(brew --prefix bzip2)/lib" pyenv install --patch 3.6.13 < <(curl -sSL https://github.com/python/cpython/commit/8ea6353.patch\?full_index\=1)

@native-api
Copy link
Member

native-api commented May 6, 2021

#1737 (comment):

Python versions without python/cpython#21113 and its backports are incompatible with MacOS 11 and cannot be built on it.

Releases with the fix are: 3.7.8+, 3.8.4+, 3.9.0+

Any earlier versions can't be built.

@native-api native-api added the by-design the reported behavior is as intended label May 6, 2021
@pyenv pyenv deleted a comment from DanielFNG Apr 10, 2022
@pyenv pyenv deleted a comment from fhughes90 Apr 10, 2022
@pyenv pyenv deleted a comment from ako-amboss Apr 10, 2022
@pyenv pyenv deleted a comment from nshine Apr 10, 2022
@pyenv pyenv deleted a comment from jpmckinney Apr 10, 2022
@pyenv pyenv deleted a comment from belinskayaA Apr 10, 2022
@pyenv pyenv deleted a comment from rr1000 Apr 10, 2022
@native-api native-api changed the title Python 3.6 cannot be installed on Mac OS 11.0.1: "implicit declaration of function 'sendfile' is invalid in C99" Python 3.6-3.8 cannot be installed on Mac OS 11.0.1: "implicit declaration of function 'sendfile' is invalid in C99" Apr 29, 2022
@Sangdol
Copy link

Sangdol commented Jun 7, 2022

For anyone who found this now, since the patch is merged, the proper fix first step to try would be to upgrade pyenv:

brew upgrade pyenv

@brdacost
Copy link

brdacost commented Jun 8, 2022

For anyone who found this now, since the patch is merged, the proper fix would be to upgrade pyenv:

brew upgrade pyenv

I have tried just upgrading the pyenv version and the error still occurs.

@Sangdol
Copy link

Sangdol commented Jun 8, 2022

@brdacost okay, it seems I was too optimistic. I've rephrased my sentence.

@native-api
Copy link
Member

@Sangdol actually, as of this writing, users have only provided patches for 3.6.15 (the last 3.6 release) that backport MacOS 11 support.
For other version lines, you need to install a recent enough release which has official support.

We ought to document such "extended support" cases in the README -- but I'm not sure where exactly users would expect to find them.

@brdacost
Copy link

brdacost commented Jun 8, 2022

@Sangdol no probs, sometimes is good to be optimistic hehe. The rephrased sentence is still a great advice tho'

@Sangdol actually, as of this writing, users have only provided patches for 3.6.15 (the last 3.6 release) that backport MacOS 11 support. For other version lines, you need to install a recent enough release which has official support.

We ought to document such "extended support" cases in the README -- but I'm not sure where exactly users would expect to find them.

@native-api, I added these lines of code to my .zshrc and tryied installing the version 3.6.15. It actually worked! :D
Thanks a lot

@F28L
Copy link

F28L commented Jun 27, 2022

I had lots of trouble installing any 3.x version except for 3.13 on Mac OS X 12.4
Monterrey.

Try this, works for some, but not me:
https://stackoverflow.com/questions/71577626/issues-installing-python-3-8-10-on-macos-12-3-monterey
brew install gcc && CC=which gcc-11 pyenv install 3.7.5

Apparently, Apple released a particular update in OS 12+ that rendered Python < 3.13 faulty (https://developer.apple.com/forums/thread/703557, idk all the details).

I was able to install using pyenv install 3.13 but other commands like pyenv install 3.9 or pyenv install 3.7.5 failed.

Before installing 3.13 I also reinstalled my command line tools and tried updating Xcode
https://apple.stackexchange.com/questions/93573/how-to-reinstall-xcode-command-line-tools
https://stackoverflow.com/questions/34617452/how-to-update-xcode-from-command-line

@native-api
Copy link
Member

We ought to document such "extended support" cases in the README -- but I'm not sure where exactly users would expect to find them.

FYI, we've added the corresponding section to the README since.

@Ancientshield
Copy link

Ancientshield commented Oct 14, 2022

LDFLAGS="-L/usr/local/opt/bzip2/lib -L/usr/local/opt/zlib/lib -L/usr/local/opt/openssl@1.1/lib" CFLAGS="-I/usr/local/opt/bzip2/include -I/usr/local/opt/zlib/include -I/usr/local/opt/openssl@1.1/include -I$(xcrun --show-sdk-path)/usr/include -Wno-implicit-function-declaration" pyenv install 3.7.0
python-build: use openssl@1.1 from homebrew
python-build: use readline from homebrew
Downloading Python-3.7.0.tar.xz...
-> https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tar.xz
Installing Python-3.7.0...
python-build: use tcl-tk from homebrew
python-build: use readline from homebrew
python-build: use zlib from xcode sdk

BUILD FAILED (OS X 12.6 using python-build 2.3.5-2-g03a5d653)

Inspect or clean up the working tree at /var/folders/hn/q_4hj7_n71g_jtk0m52snjvh0000gp/T/python-build.20221014150623.9173
Results logged to /var/folders/hn/q_4hj7_n71g_jtk0m52snjvh0000gp/T/python-build.20221014150623.9173.log

Last 10 log lines:
checking for --with-cxx-main=<compiler>... no
checking for clang++... no
configure:

  By default, distutils will build C++ extension modules with "clang++".
  If this is not intended, then set CXX on the configure command line.

checking for the platform triplet based on compiler characteristics... darwin
configure: error: internal configure error for the platform triplet, please file a bug report
make: *** No targets specified and no makefile found.  Stop.

@zjohnsilver
Copy link

@Ancientshield Did you find a solution?

@zjohnsilver
Copy link

macOS 12.6

I was able to install python 3.7.7 using the .pkg downloaded from the official python website.

@native-api
Copy link
Member

Now, we can backport the patches to support MacOS 11+ to 3.7.0+ and 3.8.0+ if enough people need that.

But 3.6 is EOL so I don't think we're going to touch anything but the last release, 3.6.15 (which already has those patches). Interested parties can provide a PR or commission us to handle specific version(s) for them.

@scirelli
Copy link

I have the same issue as @Ancientshield . Same error.
I am not able to install 3.7.3, I was able to install 3.7.13

@StillLearning89
Copy link

StillLearning89 commented Jan 30, 2023

❯ pyenv install 3.6.8 python-build: use openssl@1.1 from homebrew python-build: use readline from homebrew Downloading Python-3.6.8.tar.xz... -> https://www.python.org/ftp/python/3.6.8/Python-3.6.8.tar.xz Installing Python-3.6.8... python-build: use readline from homebrew python-build: use zlib from xcode sdk

BUILD FAILED (OS X 11.0.1 using python-build 20180424)

Inspect or clean up the working tree at /var/folders/bb/0bxy1cy57rb_pxdkvlyst3nh0000gn/T/python-build.20201120103846.92208 Results logged to /var/folders/bb/0bxy1cy57rb_pxdkvlyst3nh0000gn/T/python-build.20201120103846.92208.log

Last 10 log lines: struct sf_hdtr sf; ^ ./Modules/posixmodule.c:8210:15: error: implicit declaration of function 'sendfile' is invalid in C99 [-Werror,-Wimplicit-function-declaration] ret = sendfile(in, out, offset, &sbytes, &sf, flags); ^ clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -I. -I./Include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl@1.1/include -I/Users/ethanyan/.pyenv/versions/3.6.8/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl@1.1/include -I/Users/ethanyan/.pyenv/versions/3.6.8/include -I/usr/local/opt/openssl@1.1/include -DPy_BUILD_CORE -c ./Modules/_codecsmodule.c -o Modules/_codecsmodule.o clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -I. -I./Include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl@1.1/include -I/Users/ethanyan/.pyenv/versions/3.6.8/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl@1.1/include -I/Users/ethanyan/.pyenv/versions/3.6.8/include -I/usr/local/opt/openssl@1.1/include -DPy_BUILD_CORE -c ./Modules/_weakref.c -o Modules/_weakref.o 2 errors generated. make: *** [Modules/posixmodule.o] Error 1 make: *** Waiting for unfinished jobs....

Hi @EthanYan6 ,
Please let me know if you were able to fix this issue for mac M1.

@StillLearning89
Copy link

StillLearning89 commented Feb 8, 2023

Hi @native-api ,
I'm still facing issues with python 3.6.8 installation on Mac M1. Please let me know if any patch is available for this. Please find the latest error -

~ % CFLAGS="-I$(brew --prefix openssl)/include -I$(brew --prefix bzip2)/include -I$(brew --prefix readline)/include -I$(xcrun --show-sdk-path)/usr/include" LDFLAGS="-L$(brew --prefix openssl)/lib -L$(brew --prefix readline)/lib -L$(brew --prefix zlib)/lib -L$(brew --prefix bzip2)/lib" pyenv86 install --patch 3.6.8 < <(curl -sSL https://github.com/python/cpython/commit/8ea6353.patch\?full_index\=1)
Error: No available formula with the name "openssl@1.0". Did you mean openssl@1.1 or openssl@3?
Downloading openssl-1.1.0j.tar.gz...
-> https://www.openssl.org/source/old/1.1.0/openssl-1.1.0j.tar.gz
Installing openssl-1.1.0j...
Installed openssl-1.1.0j to /Users/lokesh.raizada/.pyenv/versions/3.6.8
python-build: use readline from homebrew
Downloading Python-3.6.8.tar.xz...
-> https://www.python.org/ftp/python/3.6.8/Python-3.6.8.tar.xz
Installing Python-3.6.8...
patching file Misc/NEWS.d/next/macOS/2020-06-24-13-51-57.bpo-41100.mcHdc5.rst
patching file configure
Hunk #1 succeeded at 3375 (offset -51 lines).
patching file configure.ac
Hunk #1 succeeded at 495 (offset -15 lines).
python-build: use readline from homebrew
python-build: use zlib from xcode sdk

BUILD FAILED (OS X 12.6.1 using python-build 20180424)

Inspect or clean up the working tree at /var/folders/41/4c6czpkx6c18sfcfj0lzjkd00000gq/T/python-build.20230208104408.53599
Results logged to /var/folders/41/4c6czpkx6c18sfcfj0lzjkd00000gq/T/python-build.20230208104408.53599.log

Last 10 log lines:
checking for --with-cxx-main=<compiler>... no
checking for clang++... no
configure:

  By default, distutils will build C++ extension modules with "clang++".
  If this is not intended, then set CXX on the configure command line.

checking for the platform triplet based on compiler characteristics... darwin
configure: error: internal configure error for the platform triplet, please file a bug report
make: *** No targets specified and no makefile found.  Stop.

@native-api
Copy link
Member

@StillLearning-HTTPStatus200 Only 3.6.15 has the patch to support MacOS 11+ and XCode 13.3. 3.6 is EOL so we have no plans for anything beyond that from our side.

@StillLearning89
Copy link

@StillLearning-HTTPStatus200 Only 3.6.15 has the patch to support MacOS 11+ and XCode 13.3. 3.6 is EOL so we have no plans for anything beyond that from our side.

Thank you for the response @native-api
As I see the link has support for - 2.7.18 : MacOS 10.15+ and Apple Silicon.

Is there any support/patch available for python 3.6.8 for Mac with M1 chips as well? :(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
by-design the reported behavior is as intended third-party the problem is in third-party software
Projects
None yet
Development

No branches or pull requests