pyport.h FreeBSD/Mac OS X "fix" causes errors in C++ compilation #55119

bertjwregeer · 2011-01-14T21:21:38Z

BPO	10910
Nosy	@brettcannon, @birkenfeld, @barry-scott, @gpshead, @ronaldoussoren, @larryhastings, @benjaminp, @ned-deily, @meadori, @mihaic, @bertjwregeer, @koobs, @DimitryAndric
Files	issue10910.txt test-boost-python.tar.gz test-boost-python.revised.tar.gz: revised simplified test case issue10910-workaround.txt issue10910-fix-versionchecks-1.diff: Correct FreeBSD version checks for enabling ctype UTF-8 workaround

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ned-deily'
closed_at = <Date 2016-08-15.07:35:14.711>
created_at = <Date 2011-01-14.21:21:37.822>
labels = ['OS-mac', 'build']
title = 'pyport.h FreeBSD/Mac OS X "fix" causes errors in C++ compilation'
updated_at = <Date 2016-08-15.07:35:14.709>
user = 'https://github.com/bertjwregeer'

bugs.python.org fields:

activity = <Date 2016-08-15.07:35:14.709>
actor = 'ned.deily'
assignee = 'ned.deily'
closed = True
closed_date = <Date 2016-08-15.07:35:14.711>
closer = 'ned.deily'
components = ['Build', 'macOS']
creation = <Date 2011-01-14.21:21:37.822>
creator = 'X-Istence'
dependencies = []
files = ['21917', '22703', '24428', '30805', '42359']
hgrepos = []
issue_num = 10910
keywords = ['patch', 'needs review']
message_count = 24.0
messages = ['126299', '130477', '133949', '134095', '135414', '140455', '140665', '140718', '140719', '152166', '152169', '152201', '152329', '152340', '152660', '163783', '192420', '192449', '192450', '262828', '272199', '272234', '272727', '272729']
nosy_count = 16.0
nosy_names = ['brett.cannon', 'georg.brandl', 'barry-scott', 'gregory.p.smith', 'ronaldoussoren', 'larry', 'benjamin.peterson', 'ned.deily', 'meador.inge', 'mihaic', 'X-Istence', 'python-dev', 'ndousis', 'Chen.Huang', 'koobs', 'dim']
pr_nums = []
priority = None
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue10910'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.6']

bertjwregeer · 2011-01-14T21:21:37Z

I was recently attempting to get Botan (http://botan.randombit.net) working with Python 2.6.6 on FreeBSD when it failed to compile, I filled a bug with Botan (http://bugs.randombit.net/show_bug.cgi?id=135) and first thought it was a compiler issue, and then thought it might be a Boost.Python issue.

However after further digging, see comment 3 (http://bugs.randombit.net/show_bug.cgi?id=135#c3) I figured out it had to do with the fix that went in to pyport.h for issues with isspace/toupper/et al. on FreeBSD's libc and also Mac OS X.

As can be seen in http://svn.python.org/view/python/trunk/Include/pyport.h?r1=36519&r2=36793 a change was added to override the original definitions with a new one that used the wide character support. It was modified again in http://svn.python.org/view/python/trunk/Include/pyport.h?r1=77585&r2=80178 to also include Mac OS X for http://bugs.python.org/issue7072.

The issue here is that if this file is included in any C++ code that then also includes <locale> or any of other functions that might pull in localefwd.h the defines will cause actual C++ code functions to be overwritten with invalid data, and the C++ compiler will throw errors such as:

/usr/include/c++/4.2/bits/localefwd.h:58:34: error: macro "isspace" passed 2
arguments, but takes just 1

Putting an #if 0, #endif around that block of code allows Botan's python module to cleanly compile without issues.

I do apologise that I don't have a simple C++ program that reproduces the problem.

ndousis · 2011-03-09T23:32:31Z

I'm upgrading from Python 2.6.5 to 2.7.1, and I'm getting the error below when compiling my code using Boost 1.45 and gcc 4.2.1 in OSX 10.6.6. The following thread describes similar symptoms related to the ordering of header files and macro definitions:
http://code.google.com/p/unladen-swallow/issues/detail?id=87#c13

The suggested patch to pyport.h (http://codereview.appspot.com/179049/patch/1/2) does not fix my problem. However, prepending

#include "Python.h"

to each source file (e.g., bend.cc below) does fix the problem. Is there a simple way to fix this without adding Python.h to every source file in my codebase?

I've installed Boost and Python from source to non-standard directories, Boost 1.45 by:
cd $(BOOST); ./bootstrap.sh --prefix=$(CURDIR)/$(BOOST) --with-python-root=$(CURDIR)/$(PYTHON_DIR) --with-python-version=2.7 --with-libraries=mpi

and Python 2.7 by:
cd $(PYTHON_SRCDIR) ; ./configure --enable-shared --prefix=$(CURDIR)/$(PYTHON_DIR)
cd $(PYTHON_SRCDIR) ; make -j $(NUMPROC) && make install

Any guidance will be greatly appreciated.

Kind regards,
Nasos

===============
...
darwin.compile.c++ bin/darwin-4.2.1/release/threading-multi/bend.o
In file included from /usr/include/c++/4.2.1/ios:47,
from /usr/include/c++/4.2.1/ostream:45,
from /usr/include/c++/4.2.1/iterator:70,
from boost/boost/next_prior.hpp:15,
from boost/boost/utility.hpp:17,
from boost/boost/python/instance_holder.hpp:10,
from boost/boost/python/object/pointer_holder.hpp:14,
from boost/boost/python/to_python_indirect.hpp:10,
from boost/boost/python/converter/arg_to_python.hpp:10,
from boost/boost/python/call.hpp:15,
from boost/boost/python/object_core.hpp:14,
from boost/boost/python/args.hpp:25,
from boost/boost/python.hpp:11,
from bend.hh:6,
from bend.cc:1:
/usr/include/c++/4.2.1/bits/localefwd.h:58:34: error: macro "isspace" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:70:34: error: macro "isupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:74:34: error: macro "islower" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:78:34: error: macro "isalpha" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:94:34: error: macro "isalnum" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:102:34: error: macro "toupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/localefwd.h:106:34: error: macro "tolower" passed 2 arguments, but takes just 1
In file included from /usr/include/c++/4.2.1/bits/basic_ios.h:44,
from /usr/include/c++/4.2.1/ios:50,
from /usr/include/c++/4.2.1/ostream:45,
from /usr/include/c++/4.2.1/iterator:70,
from boost/boost/next_prior.hpp:15,
from boost/boost/utility.hpp:17,
from boost/boost/python/instance_holder.hpp:10,
from boost/boost/python/object/pointer_holder.hpp:14,
from boost/boost/python/to_python_indirect.hpp:10,
from boost/boost/python/converter/arg_to_python.hpp:10,
from boost/boost/python/call.hpp:15,
from boost/boost/python/object_core.hpp:14,
from boost/boost/python/args.hpp:25,
from boost/boost/python.hpp:11,
from bend.hh:6,
from bend.cc:1:
/usr/include/c++/4.2.1/bits/locale_facets.h:242:53: error: macro "toupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:271:53: error: macro "tolower" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:814:53: error: macro "toupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:847:53: error: macro "tolower" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4611:44: error: macro "isspace" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4629:44: error: macro "isupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4635:44: error: macro "islower" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4641:44: error: macro "isalpha" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4665:44: error: macro "isalnum" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4677:44: error: macro "toupper" passed 2 arguments, but takes just 1
/usr/include/c++/4.2.1/bits/locale_facets.h:4683:44: error: macro "tolower" passed 2 arguments, but takes just 1
In file included from /usr/include/c++/4.2.1/ios:47,
from /usr/include/c++/4.2.1/ostream:45,
from /usr/include/c++/4.2.1/iterator:70,
from boost/boost/next_prior.hpp:15,
from boost/boost/utility.hpp:17,
from boost/boost/python/instance_holder.hpp:10,
from boost/boost/python/object/pointer_holder.hpp:14,
from boost/boost/python/to_python_indirect.hpp:10,
from boost/boost/python/converter/arg_to_python.hpp:10,
from boost/boost/python/call.hpp:15,
from boost/boost/python/object_core.hpp:14,
from boost/boost/python/args.hpp:25,
from boost/boost/python.hpp:11,
from bend.hh:6,
from bend.cc:1:
/usr/include/c++/4.2.1/bits/localefwd.h:58: error: ‘std::isspace’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:58: error: template declaration of ‘bool std::isspace’
/usr/include/c++/4.2.1/bits/localefwd.h:70: error: ‘std::isupper’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:70: error: template declaration of ‘bool std::isupper’
/usr/include/c++/4.2.1/bits/localefwd.h:74: error: ‘std::islower’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:74: error: template declaration of ‘bool std::islower’
/usr/include/c++/4.2.1/bits/localefwd.h:78: error: ‘std::isalpha’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:78: error: template declaration of ‘bool std::isalpha’
/usr/include/c++/4.2.1/bits/localefwd.h:94: error: ‘std::isalnum’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:94: error: template declaration of ‘bool std::isalnum’
/usr/include/c++/4.2.1/bits/localefwd.h:102: error: ‘std::toupper’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:102: error: template declaration of ‘_CharT std::toupper’
/usr/include/c++/4.2.1/bits/localefwd.h:106: error: ‘std::tolower’ declared as an ‘inline’ variable
/usr/include/c++/4.2.1/bits/localefwd.h:106: error: template declaration of ‘_CharT std::tolower’
In file included from /usr/include/c++/4.2.1/bits/basic_ios.h:44,
from /usr/include/c++/4.2.1/ios:50,
from /usr/include/c++/4.2.1/ostream:45,
from /usr/include/c++/4.2.1/iterator:70,
from boost/boost/next_prior.hpp:15,
from boost/boost/utility.hpp:17,
from boost/boost/python/instance_holder.hpp:10,
from boost/boost/python/object/pointer_holder.hpp:14,
from boost/boost/python/to_python_indirect.hpp:10,
from boost/boost/python/converter/arg_to_python.hpp:10,
from boost/boost/python/call.hpp:15,
from boost/boost/python/object_core.hpp:14,
from boost/boost/python/args.hpp:25,
from boost/boost/python.hpp:11,
from bend.hh:6,
from bend.cc:1:
/usr/include/c++/4.2.1/bits/locale_facets.h:227: error: ‘btowc’ is not a type
/usr/include/c++/4.2.1/bits/locale_facets.h:242: error: expected ‘;’ before ‘const’
/usr/include/c++/4.2.1/bits/locale_facets.h:255: error: expected ;' before ‘char_type’ /usr/include/c++/4.2.1/bits/locale_facets.h:256: error: ‘btowc’ is not a type /usr/include/c++/4.2.1/bits/locale_facets.h:271: error: expected ‘;’ before ‘const’ /usr/include/c++/4.2.1/bits/locale_facets.h:287: error: expected ;' before ‘char_type’
/usr/include/c++/4.2.1/bits/locale_facets.h: In member function ‘_CharT std::__ctype_abstract_base<_CharT>::towupper(int ()(_CharT)) const’:
/usr/include/c++/4.2.1/bits/locale_facets.h:228: error: ‘__c’ was not declared in this scope
/usr/include/c++/4.2.1/bits/locale_facets.h: In member function ‘_CharT std::__ctype_abstract_base<_CharT>::towlower(int ()(_CharT)) const’:
/usr/include/c++/4.2.1/bits/locale_facets.h:257: error: ‘__c’ was not declared in this scope
/usr/include/c++/4.2.1/bits/locale_facets.h: At global scope:
/usr/include/c++/4.2.1/bits/locale_facets.h:797: error: ‘btowc’ is not a type
/usr/include/c++/4.2.1/bits/locale_facets.h:814: error: expected ‘;’ before ‘const’
/usr/include/c++/4.2.1/bits/locale_facets.h:829: error: expected ;' before ‘char_type’ /usr/include/c++/4.2.1/bits/locale_facets.h:830: error: ‘btowc’ is not a type /usr/include/c++/4.2.1/bits/locale_facets.h:847: error: expected ‘;’ before ‘const’ /usr/include/c++/4.2.1/bits/locale_facets.h:866: error: expected ;' before ‘char_type’
/usr/include/c++/4.2.1/bits/locale_facets.h: In member function ‘char std::ctype::towupper(int ()(char)) const’:
/usr/include/c++/4.2.1/bits/locale_facets.h:798: error: ‘__c’ was not declared in this scope
/usr/include/c++/4.2.1/bits/locale_facets.h: In member function ‘char std::ctype::towlower(int ()(char)) const’:
/usr/include/c++/4.2.1/bits/locale_facets.h:831: error: ‘__c’ was not declared in this scope
In file included from /usr/include/c++/4.2.1/bits/basic_ios.h:44,
from /usr/include/c++/4.2.1/ios:50,
from /usr/include/c++/4.2.1/ostream:45,
from /usr/include/c++/4.2.1/iterator:70,
from boost/boost/next_prior.hpp:15,
from boost/boost/utility.hpp:17,
from boost/boost/python/instance_holder.hpp:10,
from boost/boost/python/object/pointer_holder.hpp:14,
from boost/boost/python/to_python_indirect.hpp:10,
from boost/boost/python/converter/arg_to_python.hpp:10,
from boost/boost/python/call.hpp:15,
from boost/boost/python/object_core.hpp:14,
from boost/boost/python/args.hpp:25,
from boost/boost/python.hpp:11,
from bend.hh:6,
from bend.cc:1:
/usr/include/c++/4.2.1/bits/locale_facets.h: At global scope:
/usr/include/c++/4.2.1/bits/locale_facets.h:4611: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4629: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4635: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4641: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4665: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4677: error: function definition does not declare parameters
/usr/include/c++/4.2.1/bits/locale_facets.h:4683: error: function definition does not declare parameters

"g++"  -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -g -dynamic -no-cpp-precomp -gdwarf-2 -fPIC -Wstrict-aliasing=2 -ffast-math -fno-strict-aliasing -DBOOST_PYTHON_MAX_ARITY=30 -D_REENTRANT  -I"boost" -I"python/include/python2.7" -c -o "bin/darwin-4.2.1/release/threading-multi/Dbend.o" "bend.cc"

...failed darwin.compile.c++ bin/darwin-4.2.1/release/threading-multi/bend.o...

meadori · 2011-04-18T01:20:47Z

This doesn't look like it has anything to to with the 'ctypes' library
component (http://docs.python.org/library/ctypes). So I am removing
'ctypes' from the 'Components' selection. Please correct me if I am
wrong ...

ndousis · 2011-04-19T20:16:20Z

Meador et al,

Thanks for your attention to this issue. Just to clarify, I can eliminate
the compile errors by prepending

#include "Python.h"

to any source file with

#include<boost/python.hpp>

Also, my code compiles in Linux, with and without the Python.h header:

$ uname -a
Linux nxx.xx.com 2.6.18-128.1.14.el5 #1 SMP Wed Jun 17 06:38:05 EDT 2009
x86_64 x86_64 x86_64 GNU/Linux

with the same flags as before:

cd $(BOOST); ./bootstrap.sh --prefix=$(CURDIR)/$(BOOST)
--with-python-root=$(CURDIR)/$(PYTHON_DIR) --with-python-version=2.7
--with-libraries=mpi

cd $(PYTHON_SRCDIR) ; ./configure --enable-shared
--prefix=$(CURDIR)/$(PYTHON_DIR)
cd $(PYTHON_SRCDIR) ; make -j $(NUMPROC) && make install

Thanks again,
Nasos

ronaldoussoren · 2011-05-07T08:13:12Z

The problem is that pyport tries to replace the ctypes definion of the isascii(ch) and related functions by a replacement that works better with UTF-8 on FreeBSD and OSX. That replacement is incompatible with the localfwd.h header.

The attached patch is a very crude hack that should fix the issue for this. This assumes that Python itself won't be compiled with C++ though, and might cause problems in C++ programs that use locale.h instead of localfwd.h.

As such I won't apply the patch just yet, the patch needs review from someone that's more experienced w.r.t. C++.

ndousis · 2011-07-15T18:48:37Z

Ronald,

Thanks much for your help and insight. I tried the patch but unfortunately it didn't work for me-- this might be because I have Python extensions written in C++.

Any other suggestions? Should I wait for the next version of boost.python?

Thanks and regards,
Nasos

ronaldoussoren · 2011-07-19T13:31:20Z

Waiting for a new version of boost probably won't help: this is an incompatibility between a redefinition of ctypes macros in pyport.h and definitions in the C++ header <locale>.

I barely use C++ at this time and don't know how to tweak the headers to avoid this problem. It would probably have been better to not redefine the ctype.h definitions, we should have introduced Py_isascii etc. That requires some invasive changes to the Python codebase though, and I'm not sure if that is worth the trouble.

Could someone attach a sample project that suffers from this issue (with instructions on how to build it)? That would make it easier to debug the problem.

(Removing python 2.6 because that version receives only security patches at this time, adding 3.2 and 3.3 because those use the same code in pyport.h and hence should also be affected by this issue)

ndousis · 2011-07-20T03:47:57Z

Ronald, thanks again. I stripped out as much as I could from the
original code and generated the attached project. In doing so, I
narrowed the problem down to boost/weak_ptr.hpp.

The sample project consists of:

test-boost-python/Makefile
test-boost-python/src
test-boost-python/src/Jamroot
test-boost-python/src/test
test-boost-python/src/test/Jamfile
test-boost-python/src/test/TestBoostPython.cc
test-boost-python/src/test/TestBoostPython.hh
test-boost-python/tools/Makefile

Before building the project (just type "make" in test-boost-python),
you'll need to download Boost and save it to
test-boost-python/tools/boost/boost.tar.bz2; also, the tools/Makefile
automatically downloads Python 2.7.2, but you can try Python 3.2+ in
its place.

Thanks and kind regards,
Nasos

ndousis · 2011-07-20T03:49:00Z

With attachment--

ChenHuang · 2012-01-28T16:00:35Z

I am experiencing the same issue. Googled for a bit but couldn't find much with the error, then changed the search context, come across to this thread.

In file included from /opt/local/include/boost/date_time/gregorian/parsers.hpp:13,
from /opt/local/include/boost/date_time/gregorian/gregorian.hpp:34,
from /opt/local/include/boost/date_time/posix_time/time_formatters.hpp:12,
from /opt/local/include/boost/date_time/posix_time/posix_time.hpp:24,
from /opt/local/include/boost/date_time/local_time/local_time.hpp:11,
from /opt/local/include/boost/date_time.hpp:15,
from /Users/chen/GitRepo/Cpp_EmbeddingPython/Device.h:11,
from /Users/chen/GitRepo/Cpp_EmbeddingPython/Device.cpp:12:
/opt/local/include/boost/date_time/date_parsing.hpp:50:33: error: macro "tolower" passed 2 arguments, but takes just 1
In file included from /Users/chen/GitRepo/Cpp_EmbeddingPython/Device.cpp:12:
/Users/chen/GitRepo/Cpp_EmbeddingPython/Device.h:13:21: error: Devices.h: No such file or directory
In file included from /opt/local/include/boost/date_time/gregorian/parsers.hpp:13,
from /opt/local/include/boost/date_time/gregorian/gregorian.hpp:34,
from /opt/local/include/boost/date_time/posix_time/time_formatters.hpp:12,
from /opt/local/include/boost/date_time/posix_time/posix_time.hpp:24,
from /opt/local/include/boost/date_time/local_time/local_time.hpp:11,
from /opt/local/include/boost/date_time.hpp:15,
from /Users/chen/GitRepo/Cpp_EmbeddingPython/Device.h:11,
from /Users/chen/GitRepo/Cpp_EmbeddingPython/Device.cpp:12:
/opt/local/include/boost/date_time/date_parsing.hpp: In function ‘std::string boost::date_time::convert_to_lower(std::string)’:
/opt/local/include/boost/date_time/date_parsing.hpp:50: error: cannot resolve overloaded function ‘tolower’ based on conversion to type ‘char’
In file included from /Users/chen/GitRepo/Cpp_EmbeddingPython/Device.cpp:12:

ChenHuang · 2012-01-28T16:40:25Z

if I put "#include <cctype>" before boost/date_time, it solves my problem.

And by the way, i can't seem to get to the point where boost/week_ptr.hpp causes the error. I tried to compile Nasos's upload test-boost-python. That codes works on my machine.

ChenHuang · 2012-01-28T23:34:32Z

Nevermind, putting #include <cctype> doesn't help. I have to put <python.h> on top to get away this.

ronaldoussoren · 2012-01-30T16:49:14Z

The only real fix I found is to introduce "Py_" prefixed versions of all definitions in ctypes.h that are used in Python (that is Py_isalnum) and use that throughout the python source tree.

That's a pretty invasive patch though and would probably be off-limit for 2.7 and 3.2.

bertjwregeer · 2012-01-30T17:52:17Z

In my first comment on this bug post I posted what project has issues with this, Botan with Boost.Python on FreeBSD and Mac OS X.

If required I will post how to reproduce this error using that project. If you would prefer a simplified test case, unfortunately I don't have the time to dig into that and find one.

One way to solve it is by including the Python header before any other file, however this is not an option for most 3rd party software that never sees this bug because those functions are not redefined for Linux...

ndousis · 2012-02-05T04:28:15Z

Per Chen Huang's comment, I retested my test case code using several versions of boost and Python 2.7:

boost 1.45: build fails, errors as reported.
boost 1.46: success.
boost 1.47: success.

I've attached a revised test case that automatically downloads and builds boost 1.45.

barry-scott · 2012-06-24T12:44:58Z

I'm happy to review patches or create them for you.

I see a related set of errors on Mac OS X that are down to issues in bytes_methods.py that mirror the pyport.h issues.

In C++ isspace and friends are functions not macros.
You cannot go around defining them.

Here are the lines in python2.7 that cause compilation errors for me.

/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/bytes_methods.h:#undef isspace
/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/bytes_methods.h:#define isspace(c) undefined_isspace(c)

ronaldoussoren · 2013-07-06T12:03:03Z

The bytes_methods.h issue is not present in the default branch, but is still present in the 2.7 branch. I'm not sure why those defines are there, I'm adding Gregory P. Smith to the nosy list because he added the defines and might remember why they were added and if it would be save to remove them (from bytes_methods.h).

That is defines like this:

#undef islower
#define islower(c) undefined_islower(c)

I don't know what to do about the defines in pyport.h, one workaround is disable that entire block for C++ compiler, as in the attached patches (the later one is the same as the first one, but adds a comment that explains why the code is disabled for C++)

gpshead · 2013-07-06T15:08:43Z

I was merely refactoring for PEP-3137, the original version of all that code prior to 2.7 was in Objects/stringobject.c and long predates me.

ronaldoussoren · 2013-07-06T15:11:24Z

I've some more digging to do then, although I expect that nobody knows anymore why the #define's are there. It should be safe to remove them by now, and they are not present in the default branch.

DimitryAndric · 2016-04-03T15:41:48Z

I am a FreeBSD committer, and I recently ran into this issue too, since I am working on an update of libc++ in the FreeBSD base system. As part of this work, we attempt to recompile all ports with the proposed change (see [1]). During such a recompile, we get many errors in ports that include pyport.h in C++ mode, similar to (see [2]):

In file included from scipy/interpolate/src/_interpolate.cpp:4:
In file included from scipy/interpolate/src/interpolate.h:3:
In file included from /usr/include/c++/v1/iostream:38:
In file included from /usr/include/c++/v1/ios:216:
/usr/include/c++/v1/__locale:468:15: error: C++ requires a type specifier for all declarations
    char_type toupper(char_type __c) const
              ^
/usr/local/include/python2.7/pyport.h:731:29: note: expanded from macro 'toupper'
#define toupper(c) towupper(btowc(c))
                            ^

I think Ronald's proposed workaround is fine, since messing with ctype macros is risky in C++ mode. E.g. the libc++ <ctype.h> header explicitly undefines all ctype macros, and replaces them with inline functions.

Additionally, the original issue mentioned in changeset 32950 [3] was fixed in October 2007 for FreeBSD 8.x [4], and subsequently the fix was merged back to FreeBSD 6.x and 7.x [5].

I'm adding an additional diff that corrects the __FreeBSD_version number checks. This will also be submitted to the FreeBSD ports bug tracker.

[1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=208158
[2] http://package18.nyi.freebsd.org/data/headamd64PR208158-default/2016-03-22_18h30m05s/logs/errors/py27-scipy-0.16.1.log
[3] https://hg.python.org/cpython/rev/adfe7d39a049
[4] https://svnweb.freebsd.org/base?view=revision&revision=172619
[5] https://svnweb.freebsd.org/base?view=revision&revision=172929

brettcannon · 2016-08-08T22:17:43Z

What do you think, Ned?

koobs · 2016-08-09T11:05:53Z

@brett / Ned I'm happy to carry the proposed patch in the FreeBSD ports/packages until the next releases if that helps your confidence levels.

python-dev · 2016-08-15T07:24:39Z

New changeset 2f857ac9c7af by Ned Deily in branch '3.5':
Issue bpo-10910: Avoid C++ compilation errors on FreeBSD and OS X.
https://hg.python.org/cpython/rev/2f857ac9c7af

New changeset 27a99a722828 by Ned Deily in branch '3.5':
Issue bpo-10910: Update FreedBSD version checks for the ctype UTF-8 workaround.
https://hg.python.org/cpython/rev/27a99a722828

New changeset 5ca8790f1161 by Ned Deily in branch 'default':
Issue bpo-10910: merge from 3.5
https://hg.python.org/cpython/rev/5ca8790f1161

New changeset e0ec3471cb09 by Ned Deily in branch '2.7':
Issue bpo-10910: Avoid C++ compilation errors on FreeBSD and OS X.
https://hg.python.org/cpython/rev/e0ec3471cb09

New changeset 12a70477db03 by Ned Deily in branch '2.7':
Issue bpo-10910: Update FreedBSD version checks for the ctype UTF-8 workaround.
https://hg.python.org/cpython/rev/12a70477db03

ned-deily · 2016-08-15T07:35:14Z

I have no special insight into this one but, since there seems to be general agreement here that this makes things better for those using C++ on OS X and FreeBSD, I guess it's time to try it. Pushed for release in 2.7.13, 3.5.3, and 3.6.0.

Thanks for the version update, Dimitry. BTW, it would be good if you signed the PSF Contributor Agreement (https://www.python.org/psf/contrib/contrib-form/).

bertjwregeer mannequin added extension-modules C modules in the Modules dir stdlib Python modules in the Lib dir OS-mac topic-unicode topic-ctypes labels Jan 14, 2011

bertjwregeer mannequin assigned ronaldoussoren Jan 14, 2011

bertjwregeer mannequin added build The build process and cross-build labels Jan 14, 2011

meadori added OS-mac and removed extension-modules C modules in the Modules dir stdlib Python modules in the Lib dir OS-mac topic-unicode topic-ctypes labels Apr 18, 2011

brettcannon added the release-blocker label Aug 8, 2016

brettcannon assigned ned-deily and unassigned ronaldoussoren Aug 8, 2016

ned-deily closed this as completed Aug 15, 2016

ned-deily removed the release-blocker label Aug 15, 2016

ezio-melotti transferred this issue from another repository Apr 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pyport.h FreeBSD/Mac OS X "fix" causes errors in C++ compilation #55119

pyport.h FreeBSD/Mac OS X "fix" causes errors in C++ compilation #55119

bertjwregeer mannequin commented Jan 14, 2011

bertjwregeer mannequin commented Jan 14, 2011

ndousis mannequin commented Mar 9, 2011

meadori commented Apr 18, 2011

ndousis mannequin commented Apr 19, 2011

ronaldoussoren commented May 7, 2011

ndousis mannequin commented Jul 15, 2011

ronaldoussoren commented Jul 19, 2011

ndousis mannequin commented Jul 20, 2011

ndousis mannequin commented Jul 20, 2011

ChenHuang mannequin commented Jan 28, 2012

ChenHuang mannequin commented Jan 28, 2012

ChenHuang mannequin commented Jan 28, 2012

ronaldoussoren commented Jan 30, 2012

bertjwregeer mannequin commented Jan 30, 2012

ndousis mannequin commented Feb 5, 2012

barry-scott mannequin commented Jun 24, 2012

ronaldoussoren commented Jul 6, 2013

gpshead commented Jul 6, 2013

ronaldoussoren commented Jul 6, 2013

DimitryAndric mannequin commented Apr 3, 2016

brettcannon commented Aug 8, 2016

koobs commented Aug 9, 2016

python-dev mannequin commented Aug 15, 2016

ned-deily commented Aug 15, 2016

pyport.h FreeBSD/Mac OS X "fix" causes errors in C++ compilation #55119

pyport.h FreeBSD/Mac OS X "fix" causes errors in C++ compilation #55119

Comments

bertjwregeer mannequin commented Jan 14, 2011

bertjwregeer mannequin commented Jan 14, 2011

ndousis mannequin commented Mar 9, 2011

meadori commented Apr 18, 2011

ndousis mannequin commented Apr 19, 2011

ronaldoussoren commented May 7, 2011

ndousis mannequin commented Jul 15, 2011

ronaldoussoren commented Jul 19, 2011

ndousis mannequin commented Jul 20, 2011

ndousis mannequin commented Jul 20, 2011

ChenHuang mannequin commented Jan 28, 2012

ChenHuang mannequin commented Jan 28, 2012

ChenHuang mannequin commented Jan 28, 2012

ronaldoussoren commented Jan 30, 2012

bertjwregeer mannequin commented Jan 30, 2012

ndousis mannequin commented Feb 5, 2012

barry-scott mannequin commented Jun 24, 2012

ronaldoussoren commented Jul 6, 2013

gpshead commented Jul 6, 2013

ronaldoussoren commented Jul 6, 2013

DimitryAndric mannequin commented Apr 3, 2016

brettcannon commented Aug 8, 2016

koobs commented Aug 9, 2016

python-dev mannequin commented Aug 15, 2016

ned-deily commented Aug 15, 2016