compile error on HP-UX 11.22 ia64 - 'mbstate_t' is used as a type, but has not been defined as a type

[srid]

BPO	5999
Nosy	@loewis, @terryjreedy, @vstinner, @merwok, @michael-o
Files	config.log pyconfig.h wchar.h: /usr/include/wchar.h _mbstate_t.h: #include <sys/_mbstate_t.h> test.c.preprocessed.txt: preprocessed output of test program using mbstate_t test.c_XOPEN_SOURCE_500_preprocessed.txt: preprocessed with -D_XOPEN_SOURCE=500 stdsyms.patch: Workaround for faulty extension detection logic vendor's include file

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-08-22.09:42:38.142>
created_at = <Date 2009-05-11.21:24:33.018>
labels = ['build', 'expert-unicode']
title = "compile error on HP-UX 11.22 ia64 - 'mbstate_t' is used\tas a type, but has not been defined as a type"
updated_at = <Date 2018-08-22.09:42:38.141>
user = 'https://bugs.python.org/srid'

bugs.python.org fields:

activity = <Date 2018-08-22.09:42:38.141>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2018-08-22.09:42:38.142>
closer = 'vstinner'
components = ['Build', 'Unicode']
creation = <Date 2009-05-11.21:24:33.018>
creator = 'srid'
dependencies = []
files = ['14397', '14398', '14400', '14431', '14432', '14433', '22656']
hgrepos = []
issue_num = 5999
keywords = ['patch']
message_count = 20.0
messages = ['87588', '87649', '87653', '89890', '89906', '90030', '90033', '139222', '139307', '139309', '139313', '140344', '140352', '140353', '140354', '140356', '140361', '140426', '220449', '323738']
nosy_count = 7.0
nosy_names = ['loewis', 'terry.reedy', 'vstinner', 'eric.araujo', 'srid', 'jschneid', 'michael-o']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue5999'
versions = ['Python 3.2']

[srid]

cc +DD64 -Ae -D_REENTRANT +Z -c -DNDEBUG -O -I. -IInclude -I./Include
-DPy_BUILD_CORE -o Modules/python.o ./Modules/python.c
Error 419: "./Modules/python.c", line 34 # 'mbstate_t' is used as a
type, but has not been defined as a type.
mbstate_t mbs;
^^^^^^^^^
make: *** [Modules/python.o] Error 2

!
!
!

When I get time, I need to create a patch to fix this.. but if somebody
else already figured out what the issue is, that'd be great too.

[srid]

Adding Martin to the nosy list as this is related to his change:
http://svn.python.org/view/python/branches/py3k/Modules/python.c?view=annotate#l17

[loewis]

I have two questions:

1. does your system provide the mbstate_t type?
2. if so, what header file needs to be included?

Also, please confirm a few things:
a. configure has detected that your system has mbrtowc
b. configure's analysis is correct, i.e. your system has mbrtowc indeed.

[srid]

Hello Martin,

My apologies for responding so late.

[Martin] Also, please confirm a few things: a. configure has detected
that your system has mbrtowc (...)

# Yes, as can be seen in the attached config.log

[Martin] (...) b. configure's analysis is correct, i.e. your system has
mbrtowc indeed.

Yes, mbrtowc is present as the following C programs compiles and

runs successfully:

  #include <wchar.h>
  int main()
  {
    printf("Init\n");
    mbrtowc(NULL, "", 1, NULL);
  }

[Martin] does your system provide the mbstate_t type? if so, what
header file needs to be included?

mbstate_t seems to exist in /usr/include/wchar.h .. however,
including <wchar.h> does not seem to work:

bash-2.04$ cc +DD64 -Ae -D_REENTRANT +Z --version              
cc: HP aC++/ANSI C B3910B A.05.55 [Dec 04 2003]
bash-2.04$ cat test.c
#include <wchar.h>
int main()
{
  mbstate_t foo;
  printf("Init\n");
  mbrtowc(NULL, "", 1, NULL);
}
bash-2.04$ cc test.c -o test                     
Error 419: "test.c", line 6 # 'mbstate_t' is used as a type, but has 
not been defined as a type.
      mbstate_t foo;
      ^^^^^^^^^     
bash-2.04$ aCC test.c -o test
Error 403: "test.c", line 8 # Undeclared variable 'mbrtowc'. Perhaps 
'mbtowc' as in "int mbtowc(wchar_t *,const char *,unsigned long)" ["/
usr/include/stdlib.h", line 169] was intended.
      mbrtowc(NULL, "", 1, NULL);
      ^^^^^^^   

cf. http://www.mail-archive.com/lftp-devel@uniyar.ac.ru/msg00602.html

[loewis]

mbstate_t seems to exist in /usr/include/wchar.h

I can't infer that from the copy of wchar.h that you provided. I
see that mbstate_t* is used, but I fail to find any definition
of mbstate_t.

I see that sys/_mbstate_t.h is included. This inclusion is conditional
on _INCLUDE__STDC_A1_SOURCE, so a first check should be done whether
this is defined. In addition, it would be interesting to know what
_mbstate_t.h contains.

bash-2.04$ cat test.c
#include <wchar.h>
int main()
{
mbstate_t foo;
printf("Init\n");
mbrtowc(NULL, "", 1, NULL);
}

It's best to focus on this example. Produce preprocessor output for
it, and attach that to the bug report.

This is standard C, AFAICT, so if the compiler fails to compile
it, something is wrong with the compiler, or you are using it incorrectly.

As a wild guess, try defining _XOPEN_SOURCE to 500, i.e.
-D_XOPEN_SOURCE=500.

cf. http://www.mail-archive.com/lftp-devel@uniyar.ac.ru/msg00602.html

I don't think this is relevant. Somehow, they managed to #define
mbstate_t to int, breaking the header - this should not happen here.

[srid]

I've attached the files you requested.

As a wild guess, try defining _XOPEN_SOURCE to 500,
i.e. -D_XOPEN_SOURCE=500.

Yes, this works .. the file compiles. I tried modifying the value of
_XOPEN_SOURCE to 500 in pyconfig.h (currently set to 600). While fixes
the mbstate_t error .. it results in in-numerous other errors.

[loewis]

Yes, this works .. the file compiles. I tried modifying the value of
_XOPEN_SOURCE to 500 in pyconfig.h (currently set to 600).

That's an HPUX bug then, please report it to HP. They should assume
that any feature present in XPG 5 is also present in any later version.

While fixes
the mbstate_t error .. it results in in-numerous other errors.

Well, we absolutely need mbstate_t, so we need to resolve all these
other errors.

Please be prepared for this becoming a many-months project. I don't
know what you need the HP-UX port for - maybe it would be best to hire
somebody who knows HP-UX, C, and Python.

[terryjreedy]

Sridhar, is there still a problem with current 3.2/3?

If you are no longer working on this, I think we should close as languishing/postponed.

[srid]

On 2011-06-26, at 2:04 PM, Terry J. Reedy wrote:

Terry J. Reedy <tjreedy@udel.edu> added the comment:

Sridhar, is there still a problem with current 3.2/3?

If you are no longer working on this, I think we should close as languishing/postponed.

I am not working on this yet (its low prio), but will try building 3.2 on HP-UX once 3.2 final is released.

[merwok]

« Python 3.2 was released on February 20th, 2011. » (from python.org)

[srid]

On 2011-06-27, at 9:11 AM, Éric Araujo wrote:

Éric Araujo <merwok@netwok.org> added the comment:

« Python 3.2 was released on February 20th, 2011. » (from python.org)

My mistake; I meant to say 3.2.1 final.

[jschneid]

Martin - <sys/_mbstate_t.h> provides a definition for mbstate_t only (at least on HP/UX 11i V2.0). I can verify that the problem still exists for Python 3.2.1. I am working on a workaround for this issue, and I will attach a patch once I get it to build.

[jschneid]

I got it to build on HP-UX 11. However, there are a lot of compiler warnings about type mismatches, the _ctypes, _multiprocessing and termios modules failed to build, and "make test" died after not finding a usable "binascii" module.

To get it to build, I did the following:

1. Applied the patch I attached to bpo-12561
2. Created a directory sys, and copied /usr/include/sys/stdsyms.h into it.
3. Did "chmod 644" on sys/stdsyms.h and applied the patch stdsyms.patch that I've attached to this issue to it.
4. Ran configure with the argument "CPPFLAGS=-I."

At this point, make ran to completion, and produced a python binary. However, "make test" dies within seconds of starting up.

[loewis]

Jim, the question remains why it fails to compile then. If the type is defined, why does it give an error message "but has not been defined as a type"???

[jschneid]

Martin - sys/mbstate.h is only included if _INCLUDE__STDC_A1_SOURCE is defined. The only way this gets defined in the vendor-provided include files is if _XOPEN_SOURCE is defined and is equal to 500, or __STDC_VERSION_ is defined and is greater than or equal to 199901.

I've attached a patch to broaden the _XOPEN_SOURCE case (as the test should clearly have been >=, not ==). Defining __STDC_VERSION__ to 199901 or greater will also do the job, but it feels more like a hack than just fixing what's broken in the vendor include files.

[loewis]

That's a patch to HP-UX, right? Not one to Python.

[jschneid]

Yes, it is a patch to an HP-provided C compiler system header file. I cannot provide the actual file it patches, due to copyright limitations.

[jschneid]

I am collecting HP/UX compiler workarounds in bpo-12572. I will be adding patches to it as I produce them, including a patch to fix this on HP/UX.

[BreamoreBoy]

Should this be closed in favour of bpo-12572?

[michael-o]

I cannot reproduce this with HP-UX 11.31 and master + 3.7.

I opt to close this one.