Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
I have a project I am working on. it has close to 200 modules. This is the issue I am running into.
I need to import 132 of the modules into a single module. this single module is python code and not a compiled c extension. the 132 modules I am needing to import are c extensions compiled with Cython.
no matter the arrangement of the imports it always fails at import number 108. It simply spits out an error stating module failed to initialize and the program exits. it's the same error no matter what module I put as the 108th import.
what could be causing this problem? and what would be a solution?
As it turns out there is some kind of a limit. The limit is in Python. I do not know what the limit is. I know there is a limit in place because I removed one of the imports of a pyd right at the beginning of my module import and the error moved onto the next import.
I do not know if it is a limit imposed on a per library bases. or if it is for python as a whole. I am going to import ctypes and see if it is python as a whole.
The code runs fine if i reduce the number of pyd files to 123. as soon as I add the 124th it get an error stating the dll init failed. and it does not matter what extension it is. so it is not a circular import problem.
I am compiling python code into extensions only to gain speed of code execution. so i have the ability to swap out the pyd files for py files. I thought maybe it was because of the number of imports taking place in a single file. so i tried spreading the imports across multiple files.. it still failed on number 124..
I then put into place a lazy import mechanism which only would import files as they are needed. this moved the problem to much later on in the programs execution but it still happened on the 124th file.
I also tried using importlib.import_module and imp.load_dynamic. still the same result.