Better than the prequel, I promise!
hotload enables exploratory programming with Python by providing a super-fast
feedback loop and continuous program state.
hotload differs from other
reloaders with its focus on
- Minimal dependencies
- Simple execution model (< 200 lines).
Getting started with hotload
Hotload is distributed as a single file.
load.pyin a new folder
- Copy hotload.py in there as well.
Our task is to develop
lib.py. We're going to do this by creating a really
fast feedback loop, where saves trigger re-runs.
# In lib.py x = 3 y = 4 print(x*x + y*y)
lib.py, we're going to use
load.py as a load script. This
script sets up and runs
hotload. We'll use this start:
# In launch.py import hotload hotload.hotload( watch=[ hotload.listfiles(".", ext=".py") ], steps=[ hotload.ClearTerminal(), # (1) hotload.ReloadedPythonModule.from_module_name("lib"), # (2) ] ) # (1) hotload.ClearTerminal() ensures a "dashboard"-like experience when we work, so # that the terminal doesn't keep scrolling down. # (2) then we set up a reloaded python module.
Now, use it! Run the launch script with
$ python launch.py
... and edit lib.py and save! Each save triggers a new reload.
hotload avoids external dependencies, and aims to be portable. Why support
Python 2? Because it's when interacting with old, stale systems with poor
documentation that hotloading really shines.
hotload has been tested on the following systems:
- system-installed Python 3.7.3 on Ubuntu 18.04
- Anaconda Python 3.7.3 on Ubuntu 18.04
- Anaconda Python 3.7.3 on Windows 10
- Anaconda Python 2.7.15 on Windows 10
- Abaqus Python 2.7.4 on Windows 10
Did you get the reloaded experience? Good! Would you like some more? Then, please read on! This is going to cover some limitations with naïve reloading, and how we can overcome them.
Exceptions and interrupts
The balance between liveness and security is fragile. Thus, it's recommended to understand the exception model of hotload.
C-c(Control-c) interrupts the reload loop
- For all other exceptions, the stacktrace is printed and the reload loop continues.
Why aren't recursive dependencies reloaded?
The following setup won't reload
# in lib.py import mymath print(mymath.f(1,2))
# in mymath.py def f(x): return x*x + y*y
To trigger reloads in
mymath.py from the
lib.py entry point, we need a
manual reload. Reloading differs between Python 2 and 3.
# In lib.py, Python 3 from importlib import reload import mymath reload(mymath) print(mymath.f(2,3))
A design note. The first version of this library tried to infer what modules it
would have to reload. It maintained a list of all the modules it had previously
reloaded, and watched those for files. Lots of automation. Lots of complexity.
In the end, I didn't find it useful at all. Instead, I add in reloads in the
modules I'm developing at the top like this. The advantage? It's all dynamic --
I don't have to restart
hotload -- it can just keep on runnning. If you've
chosen to watch a folder, each file change will trigger a reload to the
top-level entry point.
Calling a specific function in a module after reload
Just developing modules by having a reload "be your test" works fine for small modules, and is quite nice to work with. However, I've found it problematic for larger systems. Then I'd like to create an init function in the bottom of the file instead, and call the init function from hotload.
To use this, you can extend
hotload.ReloadedPythonModule and override
pre_reload_hook (for cleanup) or
post_reload_hook (for initialization).
This allows you to let the your reloadable application code stay clean and nice, and you can provide different configurations for running different parts of your code.
See the post-reload-hook example for more details!
How do I persist state across reloads?
- Keep a connection to an external system open under reloads
- Avoid having to re-run time-consuming external calls on each reload
You can use NameError to trigger the computation in the first place:
# in lib.py import time def expensive_computation(): # ... time.sleep(5) return 42 try: constant = constant except NameError: constant = expensive_computation()
This will trigger
expensive_computation() on the first run, and cache the
result on the next reload. Why? Because
reload doesn't flush the module's
namespace. It still has access to everything that was there before! Which can be
Hotload might need a command line interface. Here's an early draft:
find . | grep ".py" | hotload.py --clear-terminal --hotload script --run echo "That's a reload!"
Why not start with the CLI? Because I don't want the CLI to be the only entry point. When you can just start things from code, and have a well documented code entry, the system is extensible. And we don't need to control the initialization.
But a CLI will probably be useful!
Another question: Should the app take parameters to determine which files to watch, or expect a list of files on stdin?
It might be useful to run hotload in the background. Why? Because then we could interact with a hotloaded Python REPL that (at all times) is aware of our code.
Tests might be nice. To be able to ensure that there's no primitive errors across platforms. Challenge: hotload typically operates on the "outer" layer of a code base, so we'd have to build an outer-outer layer.
- entr provides this workflow as a command-line, language-agnostic tool. I use entr all the time on Linux. Limitations: you have to hop out of your Python context, and you'll have to install it on a target environment which supports it.
- Test runners for Python. You can use hotload with a test runner, and have your
lib.pyrun the test runner, thereby avoid having to restart the Python interpreter. Or you might have a test runner with a built in reloader; in which case you might not need this.
- Other Python reloaders like hoh/reloadr.
- Q: Does hotload use polling?
- A: Yes. That means it asks the OS whether all watched files have changed each iteration (by default 144 iterations per second). Using polling is simple, and works across systems. If you need lower response times or less system load, you might want to use something like inotify on Linux. Search for existing Python libraries! Note that these are going to be more complex than hotload.