Update macOS installer builds to use ncurses 6.3

[ned-deily]

BPO	46976
Nosy	@ronaldoussoren, @ned-deily

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ned-deily'
closed_at = None
created_at = <Date 2022-03-10.17:07:20.172>
labels = ['OS-mac', '3.9', '3.10', '3.11']
title = 'Update macOS installer builds to use ncurses 6.3'
updated_at = <Date 2022-03-10.17:53:11.656>
user = 'https://github.com/ned-deily'

bugs.python.org fields:

activity = <Date 2022-03-10.17:53:11.656>
actor = 'ned.deily'
assignee = 'ned.deily'
closed = False
closed_date = None
closer = None
components = ['macOS']
creation = <Date 2022-03-10.17:07:20.172>
creator = 'ned.deily'
dependencies = []
files = []
hgrepos = []
issue_num = 46976
keywords = []
message_count = 2.0
messages = ['414869', '414874']
nosy_count = 2.0
nosy_names = ['ronaldoussoren', 'ned.deily']
pr_nums = []
priority = 'high'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue46976'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

[ned-deily]

The python.org macOS installers include a private copy of the ncurses library; it has not been updated from 5.9 in a long time. The current upstream version is 6.3 and includes bug and security fixes; we should update to it.

[ned-deily]

Assigning to myself as this will require some installer build testing.

[vstinner]

The python.org macOS installers include a private copy of the ncurses library; it has not been updated from 5.9 in a long time. The current upstream version is 6.3 and includes bug and security fixes; we should update to it.

$ grep ncurses Mac/BuildScript/build-installer.py
              url="http://ftp.gnu.org/pub/gnu/ncurses/ncurses-5.9.tar.gz",
                  ("ftp://ftp.invisible-island.net/ncurses//5.9/ncurses-5.9-20120616-patch.sh.bz2",

Ah right, ncurses 5.9 is still used. This version has multiple 3 known security vulnerabilities:

• CVE-2021-39537
• CVE-2019-17595
• CVE-2019-17594

ncurses-5.9-20120616-patch.sh.bz2 is a shell script updating ncurses to 5.9 (20120616).

# Use this script to patch ncurses 5.9 to 5.9 (20120616)
# Run this script inside the ncurses 5.9 source directory et voila! Updated.

[ned-deily]

Sorry, I had planned to do an update prior to the 3.11.0 release but it didn't get done. I will get to it shortly.

[erlend-aasland]

Should we try to get this done for 3.13? Is there anything I can help with? Is the "installer build testing" documented somewhere (I did not find much in the READMEs)?

[ronaldoussoren]

FWIW, I have some code locally that installs ncurses 6.4 + patches for when I work on Python. That build passes the entire test suite with this invocation of configure:

    subprocess.check_call(
        [   
            "./configure",
            "--enable-widec",
            "--without-cxx",
            "--without-cxx-binding",
            "--without-ada",
            "--without-curses-h",
            "--enable-shared",
            "--with-shared",
            "--without-debug",
            "--without-normal",
            "--without-tests",
            "--without-manpages",
            "--without-gpm",
            "--datadir=/usr/share",
            "--sysconfdir=/etc",
            "--sharedstatedir=/usr/com",
            "--with-terminfo-dirs=/usr/share/terminfo",
            "--with-default-terminfo-dir=/usr/share/terminfo",
            "--enable-pc-files",
            f"--prefix={PREFIX}",
            f"CFLAGS=-mmacosx-version-min={DEPLOYMENT_TARGET} -arch arm64 -arch x86_64 -Werror=unguarded-availability-new -g",
            f"LDFLAGS=-mmacosx-version-min={DEPLOYMENT_TARGET} -arch arm64 -arch x86_64 -g",
        ],
        cwd=src_path,
        stdout=subprocess.DEVNULL,
    )

I did have to do some tweaking of a makefile to avoid trying to install termini files in a system location, but that's a trivial change as well.

[ned-deily]

Thanks, I will try that soon.

[cmang]

The python.org macOS installers include a private copy of the ncurses library; it has not been updated from 5.9 in a long time. The current upstream version is 6.3 and includes bug and security fixes; we should update to it.

Hi. I'm sorry to butt in on y'alls issue, but would this update mean that Python for macOS would switch from using the Ncurses 5 ABI to the Ncurses 6 ABI? My understanding is that Ncurses 6 provides a build-time option as to whether it will support ABI version 5 or 6.

There are features I would like to use from the Ncurses 6 ABI, specifically extended color pairs, which have never worked for me in Python (despite curses.has_extended_color_support() returning True across multiple versions, including the current python.org binaries).

If this isn't a good place to ask, then I apologize and will try to put it into a real bug report. Thanks.