Cannot import 3rd party module "openai"

[m-from-space]

Following the Renpy documentation, third-party modules can be installed into the python-packages directory.

I installed it using:

pip install --target "mygame/game/python-packages" -U openai

which works flawlessly.

But trying to import the module inside Renpy like:

init python:
    import openai

creates the following error:

ModuleNotFoundError: no module named 'netrc'

Netrc seems to be a standard python module, that for some reason Renpy does not ship with. Can anybody explain this?

When I manually add netrc.py to the python-packages directory, suddenly the module _multibytecodec is missing, which is also considered a standard Python module.

[renpytom]

Not every third party module works with Ren'Py. We ship a limited subset of Python, for game size reasons.

[m-from-space]

Not every third party module works with Ren'Py. We ship a limited subset of Python, for game size reasons.

Thanks Tom, but to be honest, I find that confusing in the year 2023. "Game size" shouldn't really be a big deal when it comes to modules of the standard python library, right? It's like you exclude modules like "random" or "time".

I admire all the work you do, so what can I do to make that work without bothering you any further? :)

[Booplicate]

@m-from-space I believe it is an issue for mobile/web platforms, and it's not just the size - some libs won't work at all. While it's developer's responsibility to manage that, you still don't want to ship dead weight.

That said, questions like this are being raised now and then. You can't cover needs of every game, but you can give a choice. It would be great if developers could choose "full std" build along with "stripped std" version when using the SDK. Or maybe even select the needed libs during building. I once needed to use unittest and I had to add that myself, it's not something you'd usually include in the releases, but it's a must during development.

A lot of very useful and I'd even say mandatory libs are missing. Like multiprocessing (today even games may need to run extra parallel processes) or a bunch of libs required for popular packages like numpy, and to make those work you have to compile renpy yourself which is not ideal for most people.

[m-from-space]

@Booplicate Thanks for your reply. I didn't think of mobile/web platforms to be honest. But I also wonder how much the standard python libraries actually weigh. On my system not very much.

But nevertheless, I would love to just include the missing modules into the Renpy build. But neither appending the sys.path to my local python libraries, nor copying it solved the problem of modules "_multibytecodec" for me. This module is located in the lib_dynload folder as far as I figured it out, but not as a pyc or py file. I also looked into the "renpy-build" project to try to figure out how to build it myself but so far I don't know where to define the libraries that Renpy will include.

Any suggestion on how to include that strange modules would be appreciated. I even asked ChatGPT, but it won't solve my problems for now. ;)

[renpytom]

I've gone ahead and included the full Python standard library, less the modules that require third party libraries (like the dbm wrappers, uuid, sqlite, etc). I also left out the modules that will be removed in Python 3.12.

Multiprocessing is there, as is _multibytecodec, netrc.

I haven't included numpy, but it might get included in the future.

[metanoder]

@renpytom

I am very interested in including numpy in my game in Ren'py. I am hoping this isn't a big ask, but can you provide some general steps to include numpy, if we build ren'py from source ourselves? That would greatly help me until numpy is added in the future.

I've gone ahead and included the full Python standard library, less the modules that require third party libraries (like the dbm wrappers, uuid, sqlite, etc). I also left out the modules that will be removed in Python 3.12.

Multiprocessing is there, as is _multibytecodec, netrc.

I haven't included numpy, but it might get included in the future.

[Gouvernathor]

You can do that : https://www.renpy.org/dev-doc/html/python.html#first-and-third-party-python-modules-and-packages
The example featuring "requests" is a bit obsolete since requests has been included in mainline renpy for some versions now. Maywe we should write numpy instead.

[mal]

numpy isn't pure Python, so those docs won't help. Any docs there might be on the subject are likely in the renpy-build repo.

[metanoder]

Thanks @Gouvernathor @mal

Tried to implement processes from the link Gouvernathor mentioned, but no dice. Had the impression I needed to compile Renpy with Numpy. Will look at the renpy-build repo next.

Looking at the output, numpy needs C-extensions with it? ...yea, would love to know how to do that... I'd have to compile C-extensions with numpy to renpy?
Appreciate any guidance.

[m-from-space]

@metanoder

Looking at the output, numpy needs C-extensions with it? ...yea, would love to know how to do that... I'd have to compile C-extensions with numpy to renpy? Appreciate any guidance.

Since I also want to include sqlite3, which is not shipped with Renpy (although part of the Python standard library), I am looking for answers. This might be the one I guess: https://github.com/renpy/renpy-build/tree/master/extensions - Meaning, prepare those extensions as documented and then build Renpy yourself.

[metanoder]

Thank you for sharing this! I'm gonna give it a try. 🙏🏼🥂