Add an Event Hubs fraud detection sample by DrisDary · Pull Request #104 · localstack/localstack-azure-samples

DrisDary · 2026-07-27T18:26:01Z

Motivation

The repo had no Event Hubs sample, and none of the existing samples exercise streaming: multi-protocol ingestion, consumer groups, checkpointing, Capture or Schema Registry. This adds one, built around a scenario enterprises recognise — real-time payment fraud detection — so it demonstrates Event Hubs working alongside Storage, Key Vault, Functions, App Service and Monitor the way it would in the cloud, rather than in isolation.

Building it also served as a real-world test of the Event Hubs emulation. It uncovered four emulator gaps, fixed separately in https://github.com/localstack/localstack-pro/pull/8100 (see Dependency below).

Changes

Adds samples/eventhubs/python:

Producers (src/producers/) — the same payments published three ways: the official azure-eventhub SDK over AMQP with partition keys and batching, a legacy confluent-kafka client that needs no Azure SDK at all, and an HTTPS gateway with a hand-built SAS token. All three land in one log, which is the property under test. schema_register.py publishes the Avro contract to the Schema Registry.
Processor (src/functions/) — an Event Hubs-triggered Function App applying an amount rule and a per-account velocity rule, emitting alerts to a second hub through an output binding. Partition keys make the velocity rule correct, since an account's payments always land on one partition.
Dashboard (src/dashboard/) — a Flask app showing live partition metrics, checkpoint lag, the alert feed, Capture archives decoded in the browser, and registered schemas.
Scripts (scripts/) — deploy.sh (full topology), validate.sh (33 numbered checks over the control plane, data plane, plus geo-disaster-recovery pairing and an application group with a throttling policy), run-pipeline.sh (the end-to-end demo), cleanup.sh.
Bicep (bicep/) and Terraform (terraform/) — the same topology through both IaC paths.
Registers all three paths in run-samples.sh and adds the row to the root README.md.

Two deliberate design decisions, both documented in the sample:

Producers run on the host. In a real deployment POS terminals and IoT devices are outside Azure; everything cloud-side (processor, dashboard) is emulator-hosted, following the existing samples.
The Bicep path sets connection strings from deploy.sh, not from the template. Anything a template emits as an output is retained in deployment history, so the keys are read with az ... keys list after the resources exist and written straight into Key Vault and the app settings.

Tests

Verified against a locally built emulator image containing the fixes from the linked PR:

Azure CLI path: deploy.sh 12/12 steps → validate.sh 33 passed, 0 failed → run-pipeline.sh end to end → cleanup.sh.
Bicep path: full deploy → validate.sh 33 passed, 0 failed.
Terraform path: apply (23 resources) → re-apply converges → both apps deployed → verification → destroy (23/23). terraform fmt clean and validate passing; Bicep compiles without warnings; all shell scripts pass bash -n.
The demo run itself is the substantive check: 52 AMQP + 15 Kafka + 8 HTTPS events into one log, fraud alerts delivered to the alerts hub, alert count growing across a processor restart (checkpoint recovery — nothing lost, nothing reprocessed), and 69 Capture archives with the newest decoded from Blob Storage.

Dependency — please read before merging. The Function App part of this sample cannot work until that PR merges and a new localstack/localstack-azure image ships: without it, an Event Hubs-triggered function never starts, az webapp deploy stalls, and the Bicep template stops part-way. Everything else (namespace, hubs, consumer groups, Capture, Schema Registry, the three producers) works on the current image.

Since run-samples.sh registration means CI will exercise all three paths, it may be worth holding this PR until the image with those fixes is published, or landing it with the three run-samples.sh entries commented out and enabling them in a follow-up.

Also noted while testing, and not blocking: a second terraform apply reports in-place updates rather than "no changes", because the emulator does not echo back several properties the azurerm provider sets (App Service always_on / ftps_state / app_command_line, Key Vault tags). terraform/deploy.sh reports this rather than failing; the existing samples apply only once, so they never surface it.

A payment stream processed end to end on the emulator: producers publish over AMQP, Kafka and HTTPS into one event hub, an Event Hubs-triggered Function App scores each payment and emits alerts to a second hub, Capture archives the stream to Blob Storage as Avro, and a Web App dashboard shows partitions, checkpoint lag, alerts, decoded archives and registered schemas. Key Vault holds the connection strings, Log Analytics and Application Insights collect telemetry. Ships the three deployment paths the repo uses - Azure CLI scripts, Bicep and Terraform - plus validate and end-to-end demo scripts, and registers all three with run-samples.sh.

DrisDary · 2026-07-28T12:12:23Z

The function-app-front-door failures is not related to this PR or its changes and will be looked at separately.

Matches the version the other samples were moved to. Verified against the emulator rather than only pinned: init, fmt and validate are clean, deploy.sh applies and converges on a second apply, scripts/validate.sh passes 33 of 33, and destroy completes. No schema or deprecation changes were needed.

paolosalvatori

Review — Event Hubs fraud detection sample

This is an unusually well-built and well-documented sample — the deliberate design decisions (secrets kept out of deployment history, assertion-driven demo, identical resource names across the three deploy paths so validate.sh serves all of them) are exactly right. The findings below are posted inline with one-click suggestions where possible; two are HIGH and worth addressing before merge. Per the requester's instruction this review is submitted as an approval with comments.

1. Rules applied

localstack-pro-azure/CLAUDE.md
.claude/rules/azure/common/: coding-style.md, hooks.md, patterns.md, security.md, testing.md
.claude/rules/azure/python/: coding-style.md, hooks.md, patterns.md, security.md, testing.md
.claude/rules/azure/bicep/: coding-style.md, hooks.md, patterns.md, security.md, testing.md
.claude/rules/azure/terraform/: coding-style.md, hooks.md, patterns.md, security.md, testing.md
.claude/rules/azure/scripts/: coding-style.md, hooks.md, patterns.md, security.md, testing.md

Scoping note: these rules are path-scoped to localstack-pro-azure/** in the emulator repo; for this samples repo they were applied as conventions, not hard law. Every Azure parity claim in the inline comments was verified against Microsoft Learn or the Azure REST specs; the two claims that could not be fully confirmed are marked unverified.

2. Existing comments

No prior reviews or inline comments; the author's conversation comment about function-app-front-door CI failures being unrelated is acknowledged.

3. Out-of-diff finding (binary file — cannot carry an inline comment)

`samples/eventhubs/python/scripts/fraud_function.zip`

MEDIUM | (binary) | Committed build artifact that the scripts themselves regenerate and delete
Description: This zip is exactly what scripts/deploy.sh L430 builds and L439 deletes (its contents are currently byte-identical to src/functions/), so it reads as an accidentally committed artifact: any run of deploy.sh, run-pipeline.sh (L206/L215) or cleanup.sh (L26, rm -f "$CURRENT_DIR"/*.zip) deletes a tracked file and dirties the working tree, and a stale copy can silently drift from the sources.
Rules applied: General best practice (no build artifacts in VCS); by analogy .claude/rules/azure/terraform/security.md (generated files belong in .gitignore).
Fix:

git rm --cached samples/eventhubs/python/scripts/fraud_function.zip

and extend the ## Sample deployment artifacts block of .gitignore:

fraud_function.zip
dashboard.zip

4. Clean files

.gitignore · README.md · requirements-runtime.txt · run-samples.sh · samples/eventhubs/python/README.md · bicep/README.md · bicep/main.bicep · bicep/main.bicepparam · bicep/modules/event-hubs.bicep · bicep/modules/monitoring.bicep · bicep/modules/storage.bicep · bicep/modules/workloads.bicep · scripts/cleanup.sh · scripts/read_capture.py · scripts/roundtrip_check.py · scripts/run-pipeline.sh · src/dashboard/gunicorn.conf.py · src/dashboard/requirements.txt · src/dashboard/static/style.css · src/dashboard/static/favicon.ico · src/dashboard/static/bootstrap/** (pristine vendored Bootstrap 5.1.3 dist — see the index.html comment for the version/class mismatch) · src/functions/function_app.py · src/functions/host.json · src/functions/requirements.txt · src/producers/producer_amqp.py · terraform/README.md · terraform/outputs.tf · terraform/providers.tf · terraform/variables.tf

(scripts/README.md, src/producers/schema_register.py, and terraform/deploy.sh are intentionally not listed: they are touched by the requirements, Schema-Registry-route, and .deployment-env findings, each described in the corresponding inline comment.)

5. Verdict

Approve — submitted as an approval with comments per the requester's explicit instruction. Under the standard severity rule the two HIGH findings (/api/overview deadlocks by construction; the documented quick start crashes at demo step 5 on missing dependencies) would have driven Request changes, so please review the inline suggestions — the rest of the sample is in excellent shape.

Severity counts: HIGH 2 · MEDIUM 6 (5 inline + the binary-file finding above) · LOW 9.

Service parity gaps (emulator observations surfaced by this sample)

Notes for the emulator team rather than defects in this PR:

The emulator accepts application groups on a Standard namespace; real Azure restricts them to the Premium/Dedicated tiers (resource governance overview).
The emulator accepts a geo-DR pairing between two same-region namespaces; the documented Azure setup requires the secondary in a different region (configure geo-DR).
The emulator serves GET /$schemaGroups/{group}/schemas, which does not exist in the Azure Schema Registry data-plane spec (2023-07-01) — samples relying on it behave differently in the cloud.
The emulator honors enableSoftDelete: false on Key Vault creation; real Azure no longer allows disabling soft delete (soft-delete overview).
Already documented by the author in this PR: Kafka served in plaintext (Azure: SASL_SSL on 9093), az functionapp restart not implemented, and several App Service/Key Vault properties not echoed back on read (Terraform re-apply drift).
Worth confirming alongside the above: the REST Send Event success status (Azure documents 201 only; the sample tolerates 200 | 201).

Fixes every comment on PR #104. The dashboard's alerts endpoint could never return: receive_batch() ends only when the client is closed, and the only close was the context-manager exit that came after it. It now receives on a worker thread and closes from the request thread, waiting for every partition to drain before returning. src/producers/requirements.txt was missing azure-storage-blob and fastavro, so the documented quick start could not run its own demo. Cloud-parity corrections: application groups are documented as an emulator-only check (Azure offers them on Premium and Dedicated only), geo-DR now pairs across regions, the Key Vault template asks for soft delete instead of a value Azure refuses, and the schema panel uses the documented versions route rather than a list route that is not in the data plane. The dashboard now holds a namespace-wide Listen-only rule instead of RootManageSharedAccessKey, in all three deployment paths, and every connection string is validated before use. Both IaC paths now write scripts/.deployment-env, so the end-to-end demo runs after a Terraform or Bicep deployment too. Smaller fixes: a wrong recovery hint, a tautological port fallback, TLS verification scoped to the emulator host, the Kafka SASL username, a caveat on the undocumented BrokerProperties header, Bootstrap classes matched to the vendored 5.1.3, and the Terraform Key Vault secret reusing the explicit-endpoint connection string. Also drops a function package that was committed by accident.

DrisDary self-assigned this Jul 27, 2026