Use c99 on the aixtools bot

[skrah]

BPO	40878
Nosy	@skrah, @aixtools, @miss-islington
PRs	bpo-40878: xlc cannot handle C99 extern inline #21887 [3.9] bpo-40878: xlc cannot handle C99 extern inline. (GH-21887) #21891
Files	OpenPGP_0x722BFDB61F396FC2.asc

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/skrah'
closed_at = <Date 2020-08-15.18:43:34.133>
created_at = <Date 2020-06-05.19:34:15.491>
labels = ['build']
title = 'Use c99 on the aixtools bot'
updated_at = <Date 2021-04-26.09:25:22.612>
user = 'https://github.com/skrah'

bugs.python.org fields:

activity = <Date 2021-04-26.09:25:22.612>
actor = 'Michael.Felt'
assignee = 'skrah'
closed = True
closed_date = <Date 2020-08-15.18:43:34.133>
closer = 'skrah'
components = []
creation = <Date 2020-06-05.19:34:15.491>
creator = 'skrah'
dependencies = []
files = ['49986']
hgrepos = []
issue_num = 40878
keywords = ['patch']
message_count = 12.0
messages = ['370779', '371237', '371250', '371251', '371574', '371588', '371617', '371628', '372141', '375466', '375488', '391889']
nosy_count = 3.0
nosy_names = ['skrah', 'Michael.Felt', 'miss-islington']
pr_nums = ['21887', '21891']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue40878'
versions = []

[skrah]

There appears to be an xlc buildbot with libmpdec failures.

libmpdec uses C99 extern inline semantics. From the brief period that I had access to xlc I remember that xlc was quite picky about C99.

Actually all of Python uses C99. So I think xlc_r needs to be invoked as c99_r.

[aixtools]

Actually, iirc - xlc is c99 plus a few extra settings.

The configuration files for xlc are, traditionally, in /etc (the later
versions of the compiler have moved them into /opt/show/where so that
multiple versions of the compiler can be installed.

buildbot@x064:[/home/buildbot]ls -l /etc/*vac*
-rw-r--r--    1 bin      bin           37731 Aug 02 2018  /etc/vac.cfg.53
-rw-r--r--    1 bin      bin           39672 Aug 02 2018  /etc/vac.cfg.61
-rw-r--r--    1 bin      bin           39681 Aug 02 2018  /etc/vac.cfg.71
-rw-r--r--    1 bin      bin           39611 Aug 02 2018  /etc/vac.cfg.72

Herein I see that xlc is "extc99" and some extras:

• -qlanglvl=extc99 C compiler with common extensions, UNIX headers
  xlc:    use        = DEFLT_C
          crt        = /lib/crt0.o
          mcrt       = /lib/mcrt0.o
          gcrt       = /lib/gcrt0.o
          libraries  = -L/usr/vac/lib,-lxlopt,-lxlipa,-lxl,-lc
          proflibs   = -L/lib/profiled,-L/usr/lib/profiled
          options    =
  -qlanglvl=extc99,-qcpluscmt,-qkeyword=inline,-qalias=ansi

I have chosen the xlc_r variant - for thread_safe additions:

• standard c compiler aliased as xlc_r (61 Threads)
  xlc_r:  use        = DEFLT_C
          crt        = /lib/crt0.o
          mcrt       = /lib/mcrt0.o
          gcrt       = /lib/gcrt0.o
          libraries  = -L/usr/vac/lib,-lxlopt,-lxlipa,-lxl,-lpthreads,-lc
          proflibs   = -L/lib/profiled,-L/usr/lib/profiled
          hdlibs     = -L/usr/vac/lib,-lhmd
          options    =
  -qlanglvl=extc99,-qcpluscmt,-qkeyword=inline,-qalias=ansi,-qthreaded,-D_THREAD_SAFE,-D__VACPP_MULTI__

So, what is c99 then? -> focus on it is 'stdc99' rather than 'extc99'

• Strict ANSI compiler, ANSI headers
  c99:    use        = DEFLT_C
          crt        = /lib/crt0.o
          mcrt       = /lib/mcrt0.o
          gcrt       = /lib/gcrt0.o
          libraries  = -L/usr/vac/lib,-lxlopt,-lxlipa,-lxl,-lc
          proflibs   = -L/lib/profiled,-L/usr/lib/profiled
          options    =
  -qlanglvl=stdc99,-D_ANSI_C_SOURCE,-D_ISOC99_SOURCE,-qalias=ansi,-qstrict_induction

Hope this helps clarify that both xlc and c99 are "c99" based.

Regards,

Michael

On 05/06/2020 21:34, Stefan Krah wrote:

New submission from Stefan Krah <stefan@bytereef.org>:

There appears to be an xlc buildbot with libmpdec failures.

libmpdec uses C99 extern inline semantics. From the brief period that I had access to xlc I remember that xlc was quite picky about C99.

Actually all of Python uses C99. So I think xlc_r needs to be invoked as c99_r.

----------
messages: 370779
nosy: Michael.Felt, skrah
priority: normal
severity: normal
status: open
title: Use c99 on the aixtools bot

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue40878\>

---

[skrah]

Thanks for this information. The buildbot, however, fails to compile
C99 extern inline functions. Example:

ld: 0711-317 ERROR: Undefined symbol: .mpd_issnan

I understand the C99 extern inline feature like in this explanation:

https://www.greenend.org.uk/rjk/tech/inline.html

"""
In this example, one of the declarations does not mention inline:

// a declaration not mentioning inline
int max(int a, int b);

// a definition mentioning inline
inline int max(int a, int b) {
  return a > b ? a : b;
}

In either example, the function will be callable from other files.
"""

This is the scheme that libmpdec uses and that compilers understood
at the time I wrote it.

[skrah]

So it would still be interesting to see what happens if you compile
libmpdec with c99_r (emphasis mine):

"""
This command supports all ISO
C99 language features, but does
not support IBM language
extensions. Use this invocation for
*strict* conformance to the C99
standard.
"""

[aixtools]

I'll switch my bot https://buildbot.python.org/all/#/builders/119 to use
c99_r rather than xlc_r.

Test 1129 will b e the first with c99_r (and xlc v13).

On 11/06/2020 00:37, Stefan Krah wrote:

Stefan Krah <stefan@bytereef.org> added the comment:

So it would still be interesting to see what happens if you compile
libmpdec with c99_r (emphasis mine):

"""
This command supports all ISO
C99 language features, but does
not support IBM language
extensions. Use this invocation for
*strict* conformance to the C99
standard.
"""

----------

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue40878\>

---

[skrah]

Thanks!

Ha, it turns out that c99_r has excellent C99 compliance. :)

Variable arguments macro RAISE_SYNTAX_ERROR was invoked with an empty variable argument list.

Totally legit, we should use xlc (at least the front end) more often.

So maybe our code base is not C99 compliant enough and we have to switch back for sanity.

For the _decimal problem at hand, if you give me the compiler identification macro (xlc or something?) I can try the same as for MSVC and use the explicit EXTINLINE definition.

[aixtools]

I’ll switch back to xlc ( even try without the _r ) and look for the macro asap (vacation and occasional travel).

Sent from my iPhone

On 15 Jun 2020, at 21:20, Stefan Krah <report@bugs.python.org> wrote:


Stefan Krah <stefan@bytereef.org> added the comment:

Thanks!

Ha, it turns out that c99_r has excellent C99 compliance. :)

> Variable arguments macro RAISE_SYNTAX_ERROR was invoked with an empty variable argument list.

Totally legit, we should use xlc (at least the front end) more often.

So maybe our code base is not C99 compliant enough and we have to switch back for sanity.

For the _decimal problem at hand, if you give me the compiler identification macro (xlc or something?) I can try the same as for MSVC and use the explicit EXTINLINE definition.

----------

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue40878\>

---

[aixtools]

I switched the "aixtools" bot back to "xlc", and also back to my POWER6
server that runs xlc-v11.

iirc it is xlc-v13 (or maybe even v12) that is having trouble with
_decimal (or is it POWER8). From memory, _decimal was compiling properly
with xlc-v11.

Although - my last runs on the POWER6, by default, were using gcc - so
maybe I thought, incorrectly, that xlc was passing.

Time shall tell.

p.s. - if you make a PR that emulates a rollback, if that is what you
were thinking - I can test that separately/manually.

Michael

On 16/06/2020 09:04, Michael Felt wrote:
> Michael Felt <aixtools@felt.demon.nl> added the comment:
>
> I’ll switch back to xlc ( even try without the _r ) and look for the macro asap (vacation and occasional travel). 
>
> Sent from my iPhone
>
>> On 15 Jun 2020, at 21:20, Stefan Krah <report@bugs.python.org> wrote:
>>
>> 
>> Stefan Krah <stefan@bytereef.org> added the comment:
>>
>> Thanks!
>>
>> Ha, it turns out that c99_r has excellent C99 compliance. :)
>>
>>> Variable arguments macro RAISE_SYNTAX_ERROR was invoked with an empty variable argument list.
>> Totally legit, we should use xlc (at least the front end) more often.
>>
>>
>> So maybe our code base is not C99 compliant enough and we have to switch back for sanity.
>>
>>
>>
>> For the _decimal problem at hand, if you give me the compiler identification macro (__xlc__ or something?) I can try the same as for MSVC and use the explicit EXTINLINE definition.
>>
>> 

---

>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue40878\>
> _______________________________________
>
----------

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue40878\>

---

[aixtools]

I have been doing manual tests - and it seems build is broken for AIX
and xlc.

I shall be working on a git bisect to try and identify what broke it for
xlc-v11. Finding what broke xlc-v13 will require more time.

Note: it is no longer limited to _decimal not being built - builds
(i.e., compile) fail completely atm.

regards,

Michael

On 15/06/2020 18:55, Michael Felt wrote:

Michael Felt <aixtools@felt.demon.nl> added the comment:

I'll switch my bot https://buildbot.python.org/all/#/builders/119 to use
c99_r rather than xlc_r.

Test 1129 will b e the first with c99_r (and xlc v13).

On 11/06/2020 00:37, Stefan Krah wrote:
> Stefan Krah <stefan@bytereef.org> added the comment:
>
> So it would still be interesting to see what happens if you compile
> libmpdec with c99_r (emphasis mine):
>
>
> """
> This command supports all ISO
> C99 language features, but does
> not support IBM language
> extensions. Use this invocation for
> *strict* conformance to the C99
> standard.
> """
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue40878\>
> _______________________________________
>
----------

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue40878\>

---

[skrah]

New changeset 40e700a by Stefan Krah in branch 'master':
bpo-40878: xlc cannot handle C99 extern inline. (GH-21887)
40e700a

[skrah]

New changeset 1864eac by Miss Islington (bot) in branch '3.9':
bpo-40878: xlc cannot handle C99 extern inline. (GH-21891)
1864eac