PyConfig.stdlib_dir does not affect sys._stdlib_dir

[Yhg1s]

In Python 3.11, PyConfig gained a stdlib_dir attribute which is, as far as I can tell, supposed to override sys._stdlib_dir (which is used by importlib). No matter what I set it to or what else I set, however, it seems to be ignored. For example:

#include <Python.h>
#include <stdio.h>

extern wchar_t *_Py_GetStdlibDir(void);

int main(int argc, char **argv)
{
    PyStatus status;
    PyConfig config;
    PyConfig_InitPythonConfig(&config);
    PyConfig_SetString(&config, &config.stdlib_dir, L"/my/stdlib/dir");
    Py_InitializeFromConfig(&config);
    if (PyStatus_Exception(status)) {
        Py_ExitStatusException(status);
    }
    PyConfig_Clear(&config);
    fprintf(stderr, "stdlib_dir = '%ls'\n", _Py_GetStdlibDir());
    Py_Finalize();
}

% gcc -Wall $(/python/3.12-opt/bin/python3-config --cflags) -o testprogram testprogram.c $(/python/3.12-opt/bin/python3-config --ldflags) -lpython3.12
# Look ma, no warnings.
% ./testprogram
stdlib_dir = '/python/3.12-opt/lib/python3.12'

It doesn't seem to matter whether PyConfig.stdlib_dir is set to an existing directory either (although for my use-case, it must be a non-existant one; we're getting the stdlib from embedded data, and the incorrect setting of sys._stdlib_dir means importlib's FrozenImporter sets the wrong __file__ attribute on frozen/deepfrozen modules, like os.)

Linked PRs

• gh-106718: Respect PyConfig.stdlib_dir in getpath. #108730

[vstinner]

I wasn't involved in these changes. I only designed the original PyConfig structure.

Maybe @zooba or @ncoghlan know about this?

[zooba]

Looking at getpath.py, it doesn't try to read stdlib_dir out of the config when starting, and I assume never has (there are no checks later on to see whether it's already been set).

It looks like the stdlib_dir attribute was added around the same time I was working on porting getpath.c to Python, so it may just have been a successful merge and nobody noticed that it wasn't being read? That PR took a long time to get in.

I won't be able to get to this in time for 3.12.0rc1, but the change ought to be entirely in Python starting from getpath.py where I linked above, and it's relatively easy to test (the test suite exec's the source file, rather than needing a rebuild)

[yilei]

I'll try to send a PR for this. I'm not familiar with the code so I may need additional pointers later.