Cannot seem to embed Python 3.11 into a 64-bit Windows C++ program

[dannypike]

I cannot get the examples in the C API documentation to work on a 64-bit Windows application. The problem appears to be that the initialization code cannot find a codec for the "utf8" that it chooses to use when I call the Py_InitializeFromConfig() method.

Hopefully this is a simple problem, caused by my ignorance of Python-history. I've built a debug version of python311_d.dll to allow me to step through the source and the problem appears to be that this code fails:

    mod = PyImport_ImportModule("encodings");

Presumably because it cannot find a module called "encodings". Would some kind soul please tell me how I make that module available in the C API and update the documentation?

Thanks,

Dan

Edit:

This is an "isolated" embedding. Here's the code that I am using in my most recent attempt:

      PyPreConfig_InitIsolatedConfig(&preCfg_);
      preCfg_.utf8_mode = 1;
      PyPreConfig_InitIsolatedConfig(&preCfg_);
      PyConfig_InitIsolatedConfig(&cfg_);

      status_ = PyConfig_SetBytesArgv(&cfg_, argc, argv);
      if (PyStatus_Exception(status_)) {
         return false;
      }

      status_ = Py_InitializeFromConfig(&cfg_);
      //PyConfig_Clear(&cfg_);
      if (PyStatus_Exception(status_)) {
         return false;
      }

I've tried quite a few variants on the above, but maybe there's some magic option that I haven't noticed?

[dannypike]

After some more random changes to my code, I have found that the error goes away when the home entry in the PyConfig is set to point to the folder that contains the Lib folder of the distribution. The key is not to set it to the "Lib" folder itself, but to set it to the parent of that folder.

In other words, given this file structure:

C:\git\cpython\Lib\
   __phello__\
      ___init___.py
     ... etc
   |
   .
   |
   encodings\
      ___init___.py
   |
   .
   |
   zoneinfo\
      ___init___.py
   |
   <more .py files>   

then you need to set the configuration's home entry like this:

PyConfig cfg;
cfg.home = (wchar_t *)L"C:\\git\\cpython\\";

and not like this:

PyConfig cfg;
cfg.home = (wchar_t *)L"C:\\git\\cpython\\Lib";

This is probably described somewhere else in the documentation but I think it should be shown in the example, because it certainly has caused this "beginner" a lot of confusion trying to find a bug in his code that does not exist. The problem is that the example is incorrect.

Please let me know if this is, indeed, the right thing to do. In the meantime, I'm going to carry on under the assumption that "no error" is a good thing :)

[zooba]

I don't think it's clearly documented, but the ideal for you is to set module_search_paths to a list of paths that should be on sys.path (run Python normally and look at sys.path for inspiration). You can also add your own files at this point so that anything you're adding into the runtime can be imported.

Setting home basically activates the default CPython search behaviour, which could pull in modules from anywhere. You almost certainly don't want that (better off launching python.exe with a script that calls back into your running app - e.g. with files or a named pipe - if you do want this case).

[dannypike]

Thank you; that's very useful to know.

[vstinner]

Can this issue be closed?

[dannypike]

Yes, thanks.