Polyloader is a python module that extends the Python import statement to enable the discovery and loading of heterogenous source code packages.
Say What? In English this time
import statement is how the Python interpreter finds a module
written in Python and loads it, turning it into variables, executable
functions, constructable classes, and other Python objects, and then
exposes those objects to the currently running program.
import statement has long been extensible so that things other
than Python code could be imported, but this feature has always had two
- It's annoyingly hard to write an importer. (Believe me. Polyloader is one!)
- For filesystem-based modules (which is 99% of them) Python's importer
only understands one loader type per directory. It's not possible to
store code or data written in something other than Python in the same
directory with Python module code and load both via
The former requires a certain degree of abstraction and thought. For the latter, most people ignore the problem and load module configuration files written in JSON or YAML or whatever directly. This is fine, except when you want to write in one of Python's extended languages like Hy or Coconut in a framework like Django, Flask or Glitch.
Polyloader eliminates these limitations.
What's the real problem?
The real problem is that Python's traditional extensions,
.so/.dll files, are hard-coded in Python. In
Python 2, they're in the
_imp builtin; In Python 3, they're defined
in a private section of importlib. Either way, they're not accessible
for modification and extension.
This problem is made harder by the
pkglib module, which is part of
Python's standard library. This module uses
which again only recognizes the usual extensions. Which means you can't
list multilingual modules either; this hampers the development of Django
management commands in a syntax other than Python.
What the solution?
At its heart, the Python import system runs two different internal mechanisms to figure out what the import string (the dotted terms after the word "import") "means." Each mechanism has one or more finders, and the first finder to report "I have a loader that knows what that import string means" wins.
The very last finder is for the filesystem. The solution is to get in front of that finder with one that can handle all the other syntax loaders and knows how to fall back on the last one for those files the last one handles.