Private demand-response eligibility scoring under FHE: the utility scores your household on its encrypted smart-meter history and never sees the usage itself, while the utility's model stays its own.
Created with the Niobium FHE Application Design assistant (FHEanna) (v0.13.0).
A utility offers a rebate to households that are good candidates for easing off power during peak hours, and wants to score eligibility without seeing anyone's usage. Each household sends its encrypted 24-hour electricity-usage profile; the utility's model runs on the encrypted data and returns an encrypted eligibility score that only the household can read. Working entirely on the encrypted profile, the model should derive the two things it needs (total daily use, and evening-peak use for 5–9pm) and then score eligibility. Build a small eligibility model and keep the computation shallow.
Your electricity meter is a diary. Its hour-by-hour readings show when you wake up, when you're home, when you cook, when you travel. Utilities want exactly that usage pattern to decide which households to invite into demand-response programs, the ones that pay you a rebate for easing off power during peak-demand moments. Scoring you has meant letting the utility see your usage, and that usage is a minute-by-minute picture of your life.
This application removes the catch. The utility scores your eligibility on your encrypted usage and never sees the usage itself. Your household sends an encrypted copy of its meter history; the utility runs its scoring model directly on that encrypted data and sends back an encrypted result that only you can unlock. You learn whether you qualify. The utility learns nothing at all.
Both sides keep something back. The utility's scoring weights are its own asset and stay on its server. Your usage stays on your device. Encryption is what lets the calculation happen across that gap.
What it does, end to end. From your encrypted hourly readings it works out, still encrypted, the six quantities the model needs, runs the utility's confidential scoring model, and returns an encrypted probability. You unlock it at home and get a simple eligible / not-eligible answer. At no point does your usage, any intermediate quantity, or the score exist in the clear anywhere but your own device.
- Run it: Docker and nothing else
- Evaluation data & features
- Is it practical?
- Comparing to the cleartext model
- Under the hood
- What's next
- Clean up
You need Docker and nothing else: no OpenFHE, no FHE libraries, no compilers installed locally. Everything runs inside one image.
This is a demonstration on synthetic data. The households, the weather and the utility's weights are generated by a seeded script; no real meter reading is involved. The encryption, the protocol and the model form are real, and the numbers you see are really measured, but treat the eligibility answers as a demonstration rather than energy advice. Details under Evaluation data & features.
Set up (one time): install the skill, get the container, then build the app.
1. Install the skill. Fetch the fhe-application-design skill from GitHub into the
repo's .claude/skills/ and .agents/skills/ (both gitignored):
make install-skill2. Get the FHE-dev image (ghcr.io/niobiuminc/fhe-dev:v0.13.0). Pull the prebuilt
image from the GitHub Container Registry (ghcr), or build it from the skill (the first
build clones niobium-client and compiles the instrumented OpenFHE, about an hour the
first time):
# Pull the prebuilt image from ghcr (once published):
docker pull ghcr.io/niobiuminc/fhe-dev:v0.13.0
# Or build it (a) from a fresh clone of the skill repo:
git clone https://github.com/NiobiumInc/niobium-skills
docker build -t ghcr.io/niobiuminc/fhe-dev:v0.13.0 \
niobium-skills/skills/fhe-application-design/environment
# Or build it (b) from the skill installed in step 1:
docker build -t ghcr.io/niobiuminc/fhe-dev:v0.13.0 \
.claude/skills/fhe-application-design/environment3. Build the app's four programs (key generation, encrypt, score, decrypt):
./run-in-container.sh "make build"4. Generate the twin ledgers the run gates on. model/twin.py writes
data/twin_outputs.csv (the faithful-twin predictions) and data/noise_tolerance.txt
(the decision-margin tolerance). Those are the files run_test.sh checks the encrypted output
against, in every mode including a Fog run. They are gitignored (regenerated, not
committed), so a fresh clone must produce them once before the first run:
./run-in-container.sh "python3 model/twin.py"The model files and datasets (model/model.txt, model/derivation.txt,
data/test_inputs.csv, data/test_labels.csv)
are committed, so model/make_model_and_data.py does not need to be re-run; only
twin.py above. (make clean leaves data/ untouched, so this step is one-time
unless you delete the ledgers or change the model.)
1. Run it on the Niobium Fog. The Fog is the accelerated platform this app runs on, and it's the default; a bare run targets it:
./run-in-container.sh "./run_test.sh"- Have an account? Sign in to mint a key:
docker run --rm -it -v "$HOME/.fog":/root/.fog ghcr.io/niobiuminc/fhe-dev:v0.13.0 fog login - New to the Fog? Request access → https://console.niobium.co/request-account
2. Validate locally, no account needed. Run the same encrypted computation on your own machine and check it against the plain result:
./run-in-container.sh "./run_test.sh --cpu" # plain OpenFHE, on your CPU
./run-in-container.sh "./run_test.sh --sim" # the Fog code path, run locally
./run-in-container.sh "./run_test.sh --sim-full" # real math + bit-exact ring-level identity check, local--cpuruns the encrypted circuit directly with OpenFHE on your machine, the quickest correctness check.--simrecords the (hollow) trace the Fog would execute and replays it through a local simulator (fhetch_sim), then twin-compares the result, so you exercise the Fog code path offline. It's the closest thing to a Fog run without an account.--sim-fullis--simbut records real math instead of hollow, adding a bit-exact ring-level check that the replayed trace matches the plain OpenFHE run; run it alongside--simto surface any hollow-recording divergence: the thorough local ground-truth run, still all local with no Fog account.
All three use the four execution targets above. They differ in whose data is scored, and how much of it.
One household encrypts its own 28-day meter history, the utility scores it blind, and that household decrypts a single answer nobody else can read.
./run-in-container.sh "./run_test.sh --cpu"=== your result ===
household scored : 0
eligibility probability : 0.0694
your answer : not eligible
measured truth for you : not eligible
decrypted with your own secret key; the server only ever held ciphertext
About 21 seconds.
HOUSEHOLD picks which row of data/test_inputs.csv to score, 0 to 399,
defaulting to 0:
./run-in-container.sh "HOUSEHOLD=1 ./run_test.sh --cpu"=== your result ===
household scored : 1
eligibility probability : 0.7050
your answer : ELIGIBLE
measured truth for you : eligible
decrypted with your own secret key; the server only ever held ciphertext
The answer is not always right: household 42 scores 0.6366 and is told ELIGIBLE while its measured truth is not eligible. How often that happens is what the quality figures below the result are for.
One household proves the pipeline end to end. NREC scores that many
consecutive households starting at HOUSEHOLD, which gives the
encrypted-vs-twin fidelity gate more than one sample:
./run-in-container.sh "NREC=6 ./run_test.sh --cpu" # households 0-5
./run-in-container.sh "HOUSEHOLD=100 NREC=6 ./run_test.sh --cpu" # households 100-105For a file with several households, HOUSEHOLD picks the row:
./run-in-container.sh "HOUSEHOLD=2 ./run_test.sh --cpu --input my_usage.csv" # third rowThe faithful twin is scored for your series on the fly, so the encryption-fidelity check still runs. Your own data has no measured label, so no truth column appears; the model-quality figures still come from the bundled labeled set.
HOUSEHOLD past the end of the file, a sweep running past the last row, or a row
without exactly 672 values are each rejected before any encryption starts.
Six households take about 95 seconds. To run the whole set under encryption:
./run-in-container.sh "NREC=400 ./run_test.sh --cpu" # all 400, ~96 minutesNote that the model-quality figures printed after every run already cover all 400
households: model/twin.py scores them in the clear, which is why they appear even
on a single-household run. NREC adds encrypted samples to the fidelity gate.
The sweep reuses one key set across its rows to stay fast, and labels itself as doing so in the output. Ways 1, 1b and 3 are the shape a deployment has, where a household holds keys of its own.
Point the run at your own meter data instead of the bundled households. The file
holds 672 comma-separated hourly kWh readings (28 days x 24 hours) on one line,
one household per line. Blank lines and # comments are skipped, and the
extension is not checked, so .csv and .txt both work.
Two ready-made examples ship in data/, each carrying its expected result in the
filename, so you can try this straight from a clone:
./run-in-container.sh "./run_test.sh --cpu --input data/input_ELIGIBLE_p0.866.txt"
./run-in-container.sh "./run_test.sh --cpu --input data/input_NOT-ELIGIBLE_p0.044.csv"=== your result ===
your usage : from your own input
eligibility probability : 0.8663
your answer : ELIGIBLE
decrypted with your own secret key; the server only ever held ciphertext
The first is a heavy evening-peak household with strong air conditioning, the
second is flat and daytime-heavy with none. Their commented header labels every
column d01h00 through d28h23:
# d01h00,d01h01,d01h02,d01h03,d01h04,d01h05,d01h06,d01h07,...
0.47321,0.42153,0.49287,0.81620,0.45843,0.48160,1.20547,1.72411,...
Two separate OS processes talking only over HTTP, the secret key living only on the client, only ciphertext crossing between them:
./run-in-container.sh "python3 harness/demo_two_process.py"This scores one household (household 0), like Way 1 above. DEMO_HOUSEHOLD
picks a different one and DEMO_NREC scores several in a row:
./run-in-container.sh "DEMO_HOUSEHOLD=1 python3 harness/demo_two_process.py"
./run-in-container.sh "DEMO_NREC=3 python3 harness/demo_two_process.py"This stands the household client and the untrusted utility/compute provider
up as separate processes under run_demo/ (a client_home/ that holds sk.bin,
a server_home/ that never does). The client runs dr_keygen and ships only the
crypto context + public/eval keys at setup, while the utility's own two model
layers are staged on the server; then per household the client
encrypts (dr_encrypt), uploads the ciphertext, and downloads and decrypts
(dr_decrypt) the result; the provider runs dr_server --cpu behind
server_guard.sh and logs byte counts only. It runs a negative test first: a
secret key planted in the server home makes the server refuse to start (exit 13).
The run_demo/server_home/ folder is safe to place on an untrusted machine as-is.
Point the client at it with SERVER_URL=http://host:port to run it truly split.
Prefer plain shell? The same split done by hand. There are no environment
variables here: the household is the row index you pass to dr_encrypt, and you
score another household by rerunning the encrypt/score/decrypt steps with a
different one.
./run-in-container.sh '
set -e
./build/dr_keygen client_home # client makes keys; secret key stays here
mkdir -p server_home
cp client_home/{cc,pk,mk,rk}.bin model/model.txt model/derivation.txt server_home/ # public/eval keys + both model layers, no secret key
./build/dr_encrypt client_home data/test_inputs.csv 0 client_home/ct_x.bin # the 0 is the household
cp client_home/ct_x.bin server_home/ # the wire: ciphertext only
./server_guard.sh ./build/dr_server server_home --cpu # utility computes; guard refuses if sk present
cp server_home/ct_result.bin client_home/ # the wire back: still encrypted
./build/dr_decrypt client_home client_home/ct_result.bin # client unlocks the answer
'Synthetic data: proof of concept only. The dataset described here is synthetically generated by a seeded script (
model/make_model_and_data.py) purely to demonstrate the encrypted pipeline. It is not real, personal, or proprietary data: no actual household, smart-meter reading, or utility record is used. For a real deployment, swap in the utility's model and real labeled profiles; the same protocol and guarantees carry over.
Evaluation set: 400 held-out households (data/test_inputs.csv), fit on a
separate 3,000-household training split (data/train_inputs.csv ships the first
100 of that split as a representative sample). Each household is described by its
private 28-day hourly consumption series of 672 kWh values. The pipeline
outputs an eligibility probability (sigmoid), thresholded at 0.5 → eligible /
not eligible.
The label is a measured load impact. A demand-response event is simulated on a hot weekday; the household's Customer Baseline Load is computed 10-in-10 style (the 5–9pm hours averaged over the 10 most recent similar weekdays); the actual event load is subtracted; the household is eligible if the measured reduction clears a kW threshold (base rate ≈ 27% eligible). That is the arithmetic a utility uses for settlement, so the label is produced by the measurement protocol applied to simulated behaviour.
Per-household input (encrypted and sent to the utility):
| Feature | Description | Count |
|---|---|---|
y[0] … y[671] |
Hourly electricity consumption (kWh), 28 days × 24 h | 672 values |
Six quantities the circuit derives from those same 672 values (not extra inputs). The derivation is the CalTRACK/TOWT model form; because its regressors are the calendar and public weather, each quantity is a public linear functional of your encrypted series:
| Derived value | Definition |
|---|---|
cbl_peak |
10-in-10 Customer Baseline Load for the 5–9pm event window |
cool_slope_hi |
TOWT coefficient on the 75–90 °F cooling term (the AC-load proxy) |
peak_towt_design |
TOWT-predicted event-window load at the 95 °F design condition |
daily_mean |
Overall level (mean hourly kWh across the baseline) |
midday_mean |
Daytime-occupancy proxy; daytime-heavy users shed less |
weekend_evening_delta |
Weekend minus weekday evening load |
The confidential model is a ridge logistic regression over those six
quantities. Its weights live only on the utility's server (model/model.txt,
committed here only so the demo runs); the public derivation is separate
(model/derivation.txt). Each household encrypts its own series independently
(672 of 32,768 slots used per record).
Measured on a laptop CPU, scoring one household:
| Time to score one encrypted household | ~13 seconds server compute; ~21 seconds end to end including key generation |
| Memory on the utility's side | ~0.9 GB |
| Data per request | 13 MB up, 1 MB back |
| One-time key setup (per household) | 607 MB |
| Accuracy cost of the encryption | negligible: the encrypted answer matches the ordinary (unencrypted) computation to ~7 decimal places |
| Model quality (measured labels, synthetic households) | ROC-AUC 0.839; recruiting the top 20% hits 60% eligible vs 26.8% at random |
Model quality, and the operating point it is quoted at. A utility does not apply a probability cut to a targeting score. It ranks households and recruits down the list until it hits a budget or a capacity target, so model quality is reported at a recruitment depth. Every figure below is quoted at one depth, the top 20% by score.
Why that depth:
- Decile ranking is the reporting convention for propensity and targeting models, and top-decile lift is the standard published benchmark. Reported values in customer-targeting work generally fall between 2.0× and 3.5×, with ~2.0× a common average (decile analysis, technique comparison).
- Precision holds flat through the first two deciles here and falls from the third (0.600, 0.600, 0.567 …). Top 20% is therefore the deepest depth that still recruits at the full hit rate: it captures twice the eligible households of a top-10% cut at identical precision.
At that operating point, on the 400 held-out households:
| Model | Recruiting at random | |
|---|---|---|
| Households recruited | 80 of 400 | 80 of 400 |
| Precision (hit rate) | 0.600 | 0.268 |
| Lift over random | 2.24× | 1.00× |
| Eligibles captured | 44.9% (48 of 107) | 20% |
Ranking quality independent of any depth is ROC-AUC 0.839.
run_test prints the full recruitment-depth curve so a program with a different
budget can read its own operating point off the table.
Read against the achievable ceiling. Event response is driven by a household's willingness to curtail, which never appears in metered load, so meter data caps how well any model can predict it. A model given perfect knowledge of every physical household parameter, while still blind to willingness, also scores AUC 0.839. The six derived quantities therefore extract essentially everything extractable from the baseline period, and the remaining gap is behavioural.
The households and weather are synthetic (fixed seed), so this demonstrates the private-scoring pipeline on a synthetic domain. The model form is the CalTRACK/TOWT specification used in regulated settlement, so a utility would swap in real AMI data and its own weights while keeping the structure.
There are three versions of the same scorer, so you can see exactly where any error comes from:
- Cleartext reference: the exact model (true sigmoid), computed in the open. The ground-truth answer.
- Faithful twin: the same model, but using the polynomial approximation the encrypted circuit uses. It predicts, in the clear, exactly what the encrypted run will output.
- Encrypted run: what
run_test.shproduces under encryption.
Run the reference and twin (pure Python, a few seconds, no encryption):
./run-in-container.sh "python3 model/make_model_and_data.py && python3 model/twin.py"This prints the fitted model's quality against the true labels (accuracy,
ROC-AUC, precision/recall/F1, base rate) and the approximation cost (how far the
encryption-friendly twin drifts from the exact model; here, 100% decision
agreement, max probability error ~5e-4), and writes data/reference_outputs.csv
(cleartext) and data/twin_outputs.csv (twin). run_test.sh then checks the
encrypted output against the twin, so:
total error = approximation cost (twin vs reference) + encryption cost (encrypted vs twin, typically ~1e-7).
If the twin already disagreed with the reference, that would be a modeling limitation rather than an encryption one, and this is where you'd catch it.
The heavy math runs on ciphertext using the CKKS homomorphic-encryption scheme
(via OpenFHE); the six derived quantities are computed inside the encrypted
computation, and the whole circuit is shallow (no bootstrapping). Design
rationale, measured results, and the security/threat model are in
docs/.
Two extensions to this app.
Check the reference against OpenDSM. The plaintext reference and the faithful
twin (model/twin.py) are both written in this repository, so diffing them
verifies that they were transcribed consistently.
OpenDSM (Apache 2.0, an LF Energy project)
implements the CalTRACK methods directly. Running it over the same inputs and
diffing its output against the reference would establish that the derivation
matches the published method, and would add an external agreement figure to
docs/results-report.md beside the twin-vs-reference row. Vendoring OpenDSM
carries Apache 2.0 attribution and NOTICE obligations, so check those against this
repository's LICENSE first.
Swap in a real AMI dataset. The households and weather here come from a seeded generator. The derivation needs three things: hourly consumption, matched hourly temperature, and a calendar. Any dataset carrying all three drives it unchanged.
- EnergyBench: 78,037 real buildings at hourly resolution, CC-BY-SA-4.0. Weather accompanies roughly 13 of its 67 constituent datasets, so select from those.
- OpenSTEF liander2024: Dutch DSO load with weather covering all of 2024.
Real meter data also brings real event history, which would let the eligibility label come from observed reductions measured against the Customer Baseline Load rather than from a simulated event.
When you are done, remove the build tree and every per-run artifact:
make cleanThis deletes the compiled build/ tree, every per-mode run home (run_cpu,
run_sim, run_sim-full, run_fog, run_demo), the client_home/server_home
provisioning dirs, the generated FHETCH trace and profile directories, and any
__pycache__. The run-home glob matches directories only (run_*/), so it
cannot delete run_test.sh. The committed inputs under data/ are left untouched,
so a later run does not need to regenerate them.