Introduce BlobPump and MessagePump

.azure/README.md

@@ -6,7 +6,7 @@ This guide helps you deploy the SimpleL7Proxy service to Azure using the Azure D
 
 1. [Azure Developer CLI (AZD)](https://learn.microsoft.com/en-us/azure/developer/azure-developer-cli/install-azd)
 2. [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli)
-3. [Docker](https://www.docker.com/products/docker-desktop/)
+3. [Docker](https://www.docker.com/products/docker-desktop/)(optional; only needed for local container builds)
 
 ## Deployment Steps
 
@@ -58,6 +58,13 @@ This command will deploy the Azure infrastructure defined in the Bicep templates
 
 ### 4. Build and Deploy the Application
 
+Important
+Current deployment scripts are Docker-based.
+
+deploy.sh and deploy.ps1 build and push images using local Docker.
+deploy.sh expects images already built by Docker-based build scripts.
+If Docker is unavailable, use remote ACR build commands from CONTAINER_DEPLOYMENT.md and then update/deploy using the resulting image tags.
+
 #### For Windows:
 
 ```powershell

.azure/local-setup.sh

@@ -111,6 +111,13 @@ echo "# Primary backend host and probe" >> "$local_config_file"
 # Ensure Host1 is exported here so child processes see it and it won't be overridden by seeded defaults
 echo "export Host1=$Host1" >> "$local_config_file"
 echo "export Probe_path1=$Probe_path1" >> "$local_config_file"
+# Extract backend port from Host1
+backend1_port=$(printf "%s" "$Host1" | sed -n 's|.*:\([0-9]*\)$|\1|p')
+if [ -z "$backend1_port" ]; then
+  backend1_port="3000"
+fi
+echo "export BACKEND1_PORT=$backend1_port" >> "$local_config_file"
+echo "# Proxy listening port" >> "$local_config_file"
 # Export port (queried by the script) so the dynamic section contains all interactive values
 echo "export Port=$proxy_port" >> "$local_config_file"
 
@@ -184,8 +191,8 @@ EOF
     dotnet_dir="$script_dir/../test/nullserver/dotnet"
   sed -e "s|{{PY_DIR}}|$py_dir|g" \
     -e "s|{{DOTNET_DIR}}|$dotnet_dir|g" \
-    -e "s|{{START_CMD_PY}}|source $local_config_file && python3 stream_server.py --port \$BACKEND1_PORT|g" \
-    -e "s|{{START_CMD_DOTNET}}|source $local_config_file && dotnet run --urls http://localhost:\$BACKEND1_PORT|g" \
+    -e "s|{{START_CMD_PY}}|source $local_config_file \&\& python3 stream_server.py --port \$BACKEND1_PORT|g" \
+    -e "s|{{START_CMD_DOTNET}}|source $local_config_file \&\& dotnet run --urls http://localhost:\$BACKEND1_PORT|g" \
     "$script_dir/scenarios/null_server.txt" | sed 's/^/  /'
   # Print proxy run instructions using template
   sed -e "s|{{LOCAL_CONFIG_FILE}}|$local_config_file|g" "$script_dir/scenarios/proxy_run.txt" | sed 's/^/  /'
@@ -197,7 +204,7 @@ cat <<EOF
   After both have started, you can run some quick commands to test.
 
   Quick test examples (after starting server/proxy):
-    curl -v http://localhost:\$BACKEND1_PORT\$Probe_path1
+    curl -v http://localhost:\$Port\$Probe_path1
     curl -v \$Host1\$Probe_path1
 EOF
 

.github/copilot-instructions.md

@@ -58,6 +58,20 @@ If the user wants to author a new one, help them.
 Ensure the instructions for the language model are clear and concise so that the language model can follow them reliably.
 The prompts are located in the `src/prompts.ts` file.
 
+## Documentation Standards
+
+When updating or creating documentation in the `docs/` folder, apply all five steps below.
+
+1. **Lead with the answer.** Open with a one-line purpose sentence and a 3-item TL;DR. Put the single most important rule first (e.g., "Earliest expiration wins"). This mirrors readable docs that answer the question before explaining it.
+
+2. **Make units and defaults consistent and visible.** Consolidate all defaults, units, config keys, and reload types into a single reference table near the top. Add a one-line "Units used in this doc" note where units differ across settings. Remove duplicate tables scattered through the doc.
+
+3. **Reorganize by reader task, not by config key.** Create short sections named after what the reader is trying to do (e.g., *Selecting a Backend*, *Retrying Across Backends*, *Per-Request Overrides*). Each section must contain: a bolded one-sentence rule, a 3-line code/config example, and a short troubleshooting callout.
+
+4. **Use one annotated diagram and one worked example.** Replace multiple separate diagrams with a single annotated flow covering the full pipeline. Follow it with a step-by-step worked example table using concrete numbers that shows how the settings interact to produce the effective outcome.
+
+5. **Shorten prose and use callouts.** Convert long paragraphs into 2–3 sentence blocks. Use GitHub Markdown callouts (`[!NOTE]`, `[!TIP]`, `[!WARNING]`) for defaults, override behavior, and errors. Bold the single most important sentence in each subsection.
+
 ## Learning Journal
 
 IMPORTANT: At the beginning of EVERY session, you MUST read the file `COPILOT_LEARNINGS.md` in the root of the repository. This file contains lessons learned from previous sessions and best practices to follow. This is critical for maintaining continuity between sessions and avoiding repeated mistakes.
@@ -66,3 +80,10 @@ If the file exists, read it completely before starting any work. Apply the lesso
 
 If the file doesn't exist, create it and document any important lessons learned during the session.
 
+## Change Control
+
+- **Do NOT create new classes, methods, or files without explicit user permission.** Always describe the proposed approach and wait for approval before proceeding.
+- **Do NOT rename, remove, or change existing variables, fields, or properties** without explicit user approval.
+- When a change requires modifications outside the immediate scope of what was requested, ask first.
+- When the user says "undo", revert ALL changes from the last action, not just some.
+

README.md

@@ -36,12 +36,20 @@
 ## Prerequisites
 
 - [.NET 10 SDK](https://dotnet.microsoft.com/download)
-- [Docker](https://docs.docker.com/get-docker/) (container builds)
+- [Docker](https://docs.docker.com/get-docker/) (optional; only needed for local container builds)
 - [Azure Developer CLI (azd)](https://learn.microsoft.com/en-us/azure/developer/azure-developer-cli/install-azd) (cloud deployment)
+- [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli) (required for remote ACR builds)
 - Azure subscription with Container Apps; optionally AI Foundry / APIM
 
 ## Quick Start
 
+Important
+Current deployment scripts are Docker-based.
+
+deploy.sh and deploy.ps1 build and push images using local Docker.
+deploy.sh expects images already built by Docker-based build scripts.
+If Docker is unavailable, use remote ACR build commands from CONTAINER_DEPLOYMENT.md and then update/deploy using the resulting image tags.
+
 **Local (2 commands):**
 ```bash
 git clone https://github.com/your-org/SimpleL7Proxy.git
@@ -61,8 +69,29 @@ chmod +x .azure/setup.sh .azure/deploy.sh
 ./.azure/setup.sh && azd provision && ./.azure/deploy.sh
 ```
 
-> See [Development & Testing](docs/DEVELOPMENT.md) for local mock backends.  
-> See [Container Deployment](docs/CONTAINER_DEPLOYMENT.md) for VNET and high-performance variants.
+> No local Docker available? Use the remote ACR build workflow in [docs/CONTAINER_DEPLOYMENT.md](docs/CONTAINER_DEPLOYMENT.md).
+> See [Getting Started — Local Development](docs/BEGINNER_DEVELOPMENT.md) for the fastest setup paths.  
+> 
+---
+
+## Local Development Paths
+
+**Fastest: Port + Backend Only**
+```bash
+export Port=8080
+export Host1=http://localhost:3000
+dotnet run
+```
+
+**Second-fastest: Azure App Configuration**
+```bash
+export AZURE_APPCONFIG_ENDPOINT=https://your-appconfig.azconfig.io
+export AZURE_APPCONFIG_LABEL=dev
+dotnet run
+```
+
+→ **Need mock backends?** See [DUMMY_BACKEND.md](docs/DUMMY_BACKEND.md) for null server and Python HTTP server setups.  
+→ **Need help diagnosing?** See [TroubleshootTOC.md](docs/TroubleshootTOC.md) for issue-driven guidance.
 
 ---
 
@@ -89,8 +118,11 @@ chmod +x .azure/setup.sh .azure/deploy.sh
 | AI Foundry Integration | [docs/AI_FOUNDRY_INTEGRATION.md](docs/AI_FOUNDRY_INTEGRATION.md) |
 | APIM Policy | [APIM-Policy/readme.md](APIM-Policy/readme.md) |
 | Container Deployment | [docs/CONTAINER_DEPLOYMENT.md](docs/CONTAINER_DEPLOYMENT.md) |
-| Development & Testing | [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) |
+| Getting Started — Local Development | [docs/BEGINNER_DEVELOPMENT.md](docs/BEGINNER_DEVELOPMENT.md) |
+| Advanced Development & Tuning | [docs/ADVANCED_DEVELOPMENT.md](docs/ADVANCED_DEVELOPMENT.md) |
+| Mock Backends for Testing | [docs/DUMMY_BACKEND.md](docs/DUMMY_BACKEND.md) |
 | Response Codes | [docs/RESPONSE_CODES.md](docs/RESPONSE_CODES.md) |
+| Troubleshooting (Quick Diagnosis TOC) | [docs/TroubleshootTOC.md](docs/TroubleshootTOC.md) |
 
 ---
 

ReleaseNotes/version2.2.md

@@ -1,5 +1,24 @@
 # Release Notes #
+2.2.11.0
+
+Proyx:
+* Code restructure for Async work
+* Streamline event draining process for SB and file logger
+* Refactor blob writer methods
+* Load async templates at startup
+* Bug fix: don't activate ciruit breaker on async shutdown
+* Cleanup async shutdown
+* Bug fix: possible loss of async Queue message
+* Add configurations for AsyncBlobMaxQueue and AsyncStreamingBufferSizeBytes
+* Don't trigger Async if AsyncBlobMaxQueue is exceeded
+* Bug fix:  don't cache the streaming response when writing to a blob
+* Optimize the blob writer for throughput for streaming and file writing
+* If the templates folder is not readable, turn off async mode
+* Bug fix: storage account by connection was not working
 
+Deployment:
+* Create a deployment script to configure storage account
+
 2.2.10.7
 
 Proxy:

deployment/AppConfiguration/deploy.sh

@@ -228,7 +228,7 @@ mapfile -t CONFIG_ENTRIES < <(
                 if ($0 ~ /^[[:space:]]*\[/) continue;
 
                 if ($0 ~ /^[[:space:]]*public[[:space:]]+/) {
-                    if (match($0, /^[[:space:]]*public[[:space:]]+[^ ]+[[:space:]]+([A-Za-z_][A-Za-z0-9_]*)[[:space:]]*\{/, p)) {
+                    if (match($0, /^[[:space:]]*public[[:space:]].*[[:space:]]([A-Za-z_][A-Za-z0-9_]*)[[:space:]]*\{/, p)) {
                         prop = p[1];
                         # Extract default value from "} = VALUE;" pattern
                         defVal = "";
@@ -278,6 +278,9 @@ DEFAULT_COUNT=0
 WARM_COUNT=0
 COLD_COUNT=0
 
+# Accumulate env var → default value mappings for JSON output
+declare -a ENV_DEFAULT_ENTRIES
+
 # Build a single JSON file for batch import (all keys, single label).
 IMPORT_JSON_FILE="$(mktemp)"
 trap 'rm -f "${IMPORT_JSON_FILE}"' EXIT
@@ -355,6 +358,9 @@ for entry in "${CONFIG_ENTRIES[@]}"; do
     if [ "${SOURCE}" = "cs-default" ] || [ "${SOURCE}" = "placeholder" ]; then
         DEFAULT_COUNT=$((DEFAULT_COUNT + 1))
     fi
+
+    # Record env var name and its default value for final JSON output
+    ENV_DEFAULT_ENTRIES+=("${ENV_NAME}|${CS_DEFAULT}")
 done
 
 # Add Sentinel and RefreshSeconds to the import batch (always Warm)
@@ -401,3 +407,31 @@ echo -e "${GREEN}Label: ${APPCONFIG_LABEL:-(none)}${NC}"
 echo -e "${GREEN}Config keys published: ${SET_COUNT} (Warm: ${WARM_COUNT}, Cold: ${COLD_COUNT})${NC}"
 echo -e "${GREEN}  of which ${DEFAULT_COUNT} used C# default or '${DEFAULT_PLACEHOLDER}' placeholder${NC}"
 echo -e "${GREEN}======================================${NC}"
+
+# ----------------------------------------------------------------------------
+# Output JSON mapping: ENVIRONMENT_VARIABLE → default value
+# ----------------------------------------------------------------------------
+echo ""
+echo -e "${YELLOW}Environment Variable Defaults (JSON):${NC}"
+ENV_JSON="{"
+ENV_JSON_FIRST=true
+for edentry in "${ENV_DEFAULT_ENTRIES[@]}"; do
+    ED_NAME="$(echo "${edentry}" | cut -d'|' -f1)"
+    ED_DEFAULT="$(echo "${edentry}" | cut -d'|' -f2-)"
+    # Use null for empty defaults
+    if [ -z "${ED_DEFAULT}" ]; then
+        ED_JSON_VAL="null"
+    else
+        # Escape for JSON
+        ED_ESCAPED="$(printf '%s' "${ED_DEFAULT}" | sed 's/\\/\\\\/g; s/"/\\"/g')"
+        ED_JSON_VAL="\"${ED_ESCAPED}\""
+    fi
+    if [ "${ENV_JSON_FIRST}" = true ]; then
+        ENV_JSON_FIRST=false
+    else
+        ENV_JSON+=","
+    fi
+    ENV_JSON+="$(printf '\n  "%s": %s' "${ED_NAME}" "${ED_JSON_VAL}")"
+done
+ENV_JSON+=$'\n}'
+echo "${ENV_JSON}"

deployment/BlobStorage/README.md

@@ -0,0 +1,157 @@
+# Azure Blob Storage Deployment
+
+Provisions the Azure Storage account that SimpleL7Proxy uses for **async-mode**
+artifacts (response/header blobs) and for the **templates** container that
+holds the async response templates (`welcome.json`, `notready.json`,
+`notauthorized.json`).
+
+The script focuses on **storage account setup and the Container App's
+connection to it** — it does not touch App Configuration, Service Bus, or
+the Container App's environment variables.
+
+## What the Script Does
+
+1. **Reads the live Container App** to obtain (or create) its
+   system-assigned managed identity.
+2. **Creates the resource group and storage account** if they don't
+   already exist:
+   - `kind=StorageV2`
+   - `--allow-blob-public-access true`
+   - `--public-network-access Enabled` (required for the Container App to
+     reach the blob endpoint)
+   - `--min-tls-version TLS1_2`
+3. **Assigns RBAC** to the Container App's managed identity on the storage
+   account scope. Default role: **`Storage Blob Data Contributor`** — the
+   proxy both reads templates and writes async result blobs, so Reader is
+   not sufficient.
+4. **Optionally creates blob containers** (`templates`, `simplel7proxy`)
+   when `CREATE_CONTAINERS=true`. Container creation uses
+   `--auth-mode login`, so the signed-in user is JIT-granted
+   `Storage Blob Data Contributor` on the account first (with a 30 s wait
+   for RBAC propagation). This works whether or not shared-key auth is
+   enabled on the account.
+
+The script is **idempotent** — re-running it skips work that's already done.
+
+## Prerequisites
+
+| Requirement | Details |
+|---|---|
+| **Azure CLI** | `az` ≥ 2.50 with the `containerapp` extension |
+| **jq** | Used to parse the Container App JSON |
+| **Azure login** | `az login` (the script will prompt if needed) |
+| **A running Container App** | The script reads its identity and enables it if absent |
+| **Bash 4+** | Uses `${VAR,,}` lowercase expansion |
+
+## Quick Start
+
+```bash
+cd deployment/BlobStorage
+
+# 1. Create your parameters file
+cp deploy.parameters.example.sh deploy.parameters.sh
+
+# 2. Edit deploy.parameters.sh with your values
+#    (see Parameters section below)
+
+# 3. Run
+./deploy.sh
+```
+
+## Parameters
+
+All parameters are set in `deploy.parameters.sh`.
+
+### Required
+
+| Parameter | Description |
+|---|---|
+| `CONTAINER_APP_NAME` | Container App that will read/write blobs using its managed identity |
+| `CONTAINER_APP_RESOURCE_GROUP` | Resource group where the Container App lives |
+| `RESOURCE_GROUP` | Resource group for the storage account (created if missing) |
+| `LOCATION` | Azure region for the storage account |
+| `STORAGE_ACCOUNT_NAME` | Globally unique storage account name (3–24 lowercase alphanumeric) |
+
+### Optional
+
+| Parameter | Default | Description |
+|---|---|---|
+| `STORAGE_SKU` | `Standard_LRS` | Storage replication SKU. Short forms (`lrs`, `grs`, `zrs`, `ragrs`) are normalized. |
+| `CREATE_CONTAINERS` | `false` | When `true`, creates the containers listed in `BLOB_CONTAINERS`. |
+| `BLOB_CONTAINERS` | `templates simplel7proxy` | Space-separated list of containers to create when `CREATE_CONTAINERS=true`. |
+| `CA_BLOB_ROLE` | `Storage Blob Data Contributor` | Role assigned to the Container App's managed identity on the storage account. The proxy writes blobs, so Reader is not enough. |
+
+> **Do not commit `deploy.parameters.sh`** — it contains environment-specific values.
+> Only `deploy.parameters.example.sh` is checked in.
+
+## Containers Used by the Proxy
+
+| Container | Purpose |
+|---|---|
+| `templates` | Holds async-mode response JSON templates (`welcome.json`, `notready.json`, `notauthorized.json`). Loaded once at startup by `TemplateLoader`. |
+| `simplel7proxy` | Default `data` container for async result/header blobs (override per user via the `async-config` user-profile field). |
+
+After the script creates the `templates` container, you must upload the
+template files yourself:
+
+```bash
+az storage blob upload-batch \
+    --account-name "${STORAGE_ACCOUNT_NAME}" \
+    --destination templates \
+    --source ../../src/SimpleL7Proxy/templates \
+    --auth-mode login \
+    --overwrite
+```
+
+## How the Proxy Connects
+
+The proxy reads its blob endpoint from the `AsyncBlobStorageConfig`
+setting (in App Configuration or as an env var). Two formats are
+accepted:
+
+1. **Comma-separated** (managed identity):
+   ```
+   blobserviceuri=https://<account>.blob.core.windows.net, useMI=true
+   ```
+2. **Raw portal connection string** (key-based, treated as
+   `useMI=false`):
+   ```
+   DefaultEndpointsProtocol=https;AccountName=<account>;AccountKey=...;EndpointSuffix=core.windows.net
+   ```
+
+This script's role assignment makes option (1) work: the Container App's
+managed identity gets `Storage Blob Data Contributor` on the storage
+account, and the proxy authenticates with `DefaultAzureCredential`.
+
+## RBAC Notes
+
+- **Container App MI** — assigned `${CA_BLOB_ROLE}` (default
+  `Storage Blob Data Contributor`) on the storage account scope.
+- **Signed-in user** — only when `CREATE_CONTAINERS=true`, the script JIT
+  assigns `Storage Blob Data Contributor` to the operator running it so
+  that `az storage container create --auth-mode login` succeeds. This
+  assignment is **not removed** by the script; remove it manually if your
+  org's policy requires it.
+- RBAC propagation can take a few minutes. The script sleeps 30 s after
+  granting the operator role; if subsequent commands still 403, wait and
+  re-run.
+
+## Re-running
+
+The script is idempotent:
+
+- Existing resource group / storage account / containers are reused.
+- Existing role assignments are detected and skipped.
+- Safe to run repeatedly to verify state.
+
+## Cleanup
+
+```bash
+az storage account delete \
+    --name "${STORAGE_ACCOUNT_NAME}" \
+    --resource-group "${RESOURCE_GROUP}" \
+    --yes
+```
+
+Role assignments scoped to the storage account are removed automatically
+when the account is deleted.