Repository navigation
9. Memory Conductor
Storefront includes a built-in Memory Conductor (storefront_conductor) that coordinates background tasks, fork() subprocesses, and in-memory cache eviction across all installed KOReader plugins—preventing Kindle's memory watchdog (watchdogd / Linux OOM killer) from terminating KOReader when multiple plugins run simultaneously.
Even if other plugins have never heard of Storefront, the Memory Conductor protects your e-reader automatically as soon as Storefront loads:
-
Background Timer Serialization (
UIManager:scheduleIn): When plugins schedule background timers (onReaderReady,onNetworkConnected,onResume, periodic syncs), the Conductor identifies the calling plugin via stack inspection and checks whether it owns an active foreground window.-
Foreground UI actions, core KOReader timers, micro-timers (
<= 0.25s), and active subprocess polling timers pass through with zero delay. - Background timers are serialized into a single-file lane so two plugins never parse large JSON catalogs or scan chapters at the same time.
-
Foreground UI actions, core KOReader timers, micro-timers (
-
Pre-
fork()Copy-On-Write Stabilization (ffiutil.runInSubProcess): On Linux/Kindle, callingfork()shares LuaJIT's parent memory pages via Copy-On-Write (CoW). If the parent VM runs a garbage collection sweep or mutates tables while a child subprocess is alive, the kernel duplicates those 4 KB pages in RAM. Before any plugin forks a child process, the Conductor:- Reclaims soft caches from idle plugins if available RAM is below the safety watermark.
- Runs
collectgarbage("collect")to zero out parent GC debt beforefork(). - Holds other plugins' background timers until
isSubProcessDone(pid)orterminateSubProcess(pid)completes.
-
Kindle-Aware
/proc/meminfoThresholds: Legacy Kindle kernels (Linux 3.0) lackMemAvailableand report shared tmpfs memory (Shmem) insideCached. Storefront computes true available memory asMemFree + Buffers + max(0, Cached - Shmem):-
Low-Memory Devices (
<= 512 MBRAM / Kindle): Soft watermark45 MB, Critical floor28 MB. -
Standard Devices (
> 512 MBRAM): Soft watermark30 MB, Critical floor18 MB.
-
Low-Memory Devices (
Plugin authors can integrate with the Memory Conductor at two levels without creating a hard dependency on Storefront.
When RAM gets tight, Storefront broadcasts a standard KOReader Event:new("MemoryReclaim", level) to all active WidgetContainer plugins. Any plugin can shed its private cached tables without requireing Storefront:
--- Automatically called by Storefront Memory Conductor when system RAM is tight.
--- @param level string "soft" (pre-task cleanup) or "critical" (below critical RAM floor)
function MyPlugin:onMemoryReclaim(level)
if not self:isMenuOpen() then
self._cached_index = nil
end
if level == "critical" then
self._chapter_buffer = nil
end
return false -- Return false so the event continues to other plugins
endFor background syncs, chapter analyzers, archive jobs, or subprocess workers, plugins can request a priority-managed execution slot via pcall(require, "storefront_conductor"):
local function runWithConductor(opts)
local ok, Conductor = pcall(require, "storefront_conductor")
if ok and Conductor and Conductor.requestSlot then
return Conductor.requestSlot(opts)
end
-- Standalone fallback when Storefront is not installed
opts.run(function() end)
return nil
endQueues or immediately executes a task under Conductor supervision.
| Field | Type | Default | Description |
|---|---|---|---|
plugin |
string |
Auto-detected | Short plugin identifier (e.g. "xray", "sink", "nimbus"). |
task |
string |
"task" |
Descriptive task name for logs. |
priority |
string |
"idle_bg" |
"foreground" (runs immediately, preempts idle work), "interactive_bg" (runs before idle tasks, e.g. dirty progress push on book close), or "idle_bg" (single-lane background queue with cooldown). |
min_free_kb |
number |
Device threshold | Minimum free RAM in KB required before starting. Triggers MemoryReclaim if below this floor. |
subprocess |
boolean |
false |
Set true if the task spawns a child process (ffiutil.runInSubProcess) to run pre-fork() heap stabilization. |
sync |
boolean |
false |
Set true for synchronous functions that don't invoke done() asynchronously. |
run |
function(done) |
Required | Task callback. Async tasks must call done() when finished (a 45s safety lease prevents deadlocks). |
cancel |
function() |
nil |
Called if the Conductor preempts or aborts the task on suspend/low memory. |
on_rejected |
function(reason) |
nil |
Called with "low_memory" or "suspended" if an idle_bg task cannot safely run. |
Example (Async Background Subprocess):
runWithConductor({
plugin = "myplugin",
task = "sync_library",
priority = "idle_bg",
min_free_kb = 40 * 1024, -- Require 40 MB free
subprocess = true,
run = function(done)
self:startBackgroundSync(function(ok, err)
done() -- Release slot for the next plugin
end)
end,
cancel = function()
self:abortBackgroundSync()
end,
on_rejected = function(reason)
logger.info("MyPlugin: deferred sync:", reason)
end,
})Registers a persistent plugin profile for targeted cache eviction and cancellation:
local ok, Conductor = pcall(require, "storefront_conductor")
if ok and Conductor and Conductor.registerPlugin then
Conductor.registerPlugin({
plugin = "myplugin",
is_foreground = function() return self.menu_open == true end,
on_reclaim = function(level)
if not self.menu_open then
self.cache = nil
end
end,
on_cancel_bg = function()
self:abortBackgroundSync()
end,
})
endTriggers an immediate cross-plugin cache reclaim ("soft" or "critical") and garbage collection sweep.
Returns a diagnostic snapshot table:
-
total_kb,available_kb,free_kb,buffers_kb,cached_kb,shmem_kb -
luajit_heap_kb(current LuaJIT VM allocation in KB) -
is_low_memory_device(boolean) -
thresholds({ soft_min_free_kb, critical_min_free_kb }) -
active_subprocesses,queue_depth,is_busy