Several Dependencies are not detected when building python 3.12 on MacOSX

[i-shenl]

Bug report

Bug description:

When building python 3.12 on MacOSX, the configure script does not detect dependencies such as openssl, libffi, tcltk, ncurses, readline, or bzip2 if they are built from source and installed in custom locations. Using pkgconfig or the corresponding env variables like TCLTK_CFLAGS and TCLTK_LIBS does not help because the script still flags the dependency as missing. For openssl installed in a custom location, the configure still does not detect it even if the --with-openssl and/or --with-openssl-rpath options are used.

The 6 dependencies listed above are ones that the script does not detect. It however is able to detect and use a custom installed zlib. Problem is only observed on MacOSX. Linux is ok.

Dependency versions tested:
openssl (3.0.11, 3.1.4, 3.2.0)
bzip2 (1.0.8)
tcltk (8.6.13)
libffi (3.4.4)
ncurses (6.4)
readline (8.1, 8.2)

Steps to reproduce:

1. build any of the 6 listed dependencies from source and install in a custom location
2. configure python by using either pkgconfig or point the corresponding env vars to the location of the corresponding dependency headers and libraries ie TCLTK_CFLAGS and TCLTK_LIB to location where the custom TclTk is installed

(I needed to do rpath linking so I set this to TCLTK_LIB: TCLTK_LIBS=-Wl,-rpath,/lib )
(for openssl, I set --with-openssl to the location of the custom openssl and set --with-openssl-rpath to auto)

3. build python

Expected Result
Should be able to detect and rlink the dependency

Actual Result
The configure log flags the dependency as missing
ie, in openssl's case it is able to find find it, but it is not able to detect the needed APIs. The same test however works on Linux.

checking for include/openssl/ssl.h in /Users/isleong/openssl-inst... yes
checking whether compiling and linking against OpenSSL works... yes
checking for --with-openssl-rpath... auto
checking whether OpenSSL provides required ssl module APIs... no
checking whether OpenSSL provides required hashlib module APIs... no
...
checking for stdlib extension module _ssl... missing
checking for stdlib extension module _hashlib... missing

CPython versions tested on:

3.12

Operating systems tested on:

macOS

[erlend-aasland]

--with-openssl was removed in 3.11 IIRC; configure does not have that option in 3.12. Please see the devguide for how to build CPython on macOS: https://devguide.python.org/getting-started/setup-building/#install-dependencies

[erlend-aasland]

Also note that there is no variable named TCLTK_LIB; OTOH, there is a variable named TCLTK_LIBS:

$ ./configure --help | grep TCL
  TCLTK_CFLAGS
              C compiler flags for TCLTK, overriding pkg-config
  TCLTK_LIBS  linker flags for TCLTK, overriding pkg-config

[i-shenl]

Thanks for the response. My mistake. Typo on the TCLTK_LIBS in the ticket. I did try TCLTK_LIBS but no luck. And the same with other *_LIBS vars too. In addition to the build requirements, I also followed the notes here: https://discuss.python.org/t/setup-py-removal-python-3-12/17335

If you look inside the 3.12.0 configure script for --with-openssl, it's still listed under "Optional Packages:". No error during configuration if it's used. Also made sure to install autoconf 2.71.

I can't use Homebrew or MacPorts to install the dependencies because the work requirements I have are to build them all from source. Is this way doable?

[erlend-aasland]

Is this way doable?

Yes, it is. Please follow the instructions in the Devguide. If you need help building CPython, it would be better to open a topic on Discourse.