Scripting depth: runtime context, first-class arrays, dedicated scripting docs

.gitignore

@@ -5,3 +5,4 @@ greengrass-build/
 *.avro
 /data/
 /out/
+/logs/

docs/README.md

@@ -10,7 +10,8 @@ wherever you deploy it — as a Greengrass v2 component, a standalone process, o
 | Doc | Start here when you want to… |
 |-----|------------------------------|
 | **[Tutorial](tutorial.md)** | learn by doing — bring the processor up against a local broker and watch it downsample and archive telemetry, end to end |
-| **[How-to guides](how-to-guides.md)** | accomplish a specific task — filter, downsample, window-aggregate, archive to Parquet, forward alarms northbound, deploy |
+| **[How-to guides](how-to-guides.md)** | accomplish a specific task — filter, downsample, window-aggregate, handle array signals, script, archive to Parquet, forward alarms northbound, deploy |
+| **[Scripting](scripting.md)** | write Rhai `filter`/`script` logic — the scope (incl. runtime context), return semantics, a language primer, array handling, and a worked cookbook |
 | **[Reference](reference/)** | look up an exact option, topic, payload, or column type |
 | **[Explanation](explanation.md)** | understand how it works and why — the route/worker model, the processing-and-timing pipeline, targets and the file sink |
 
@@ -21,6 +22,8 @@ wherever you deploy it — as a Greengrass v2 component, a standalone process, o
 - **"What message do I subscribe to / publish, and on which topic?"** → [Reference — Messaging Interface](reference/messaging-interface.md).
 - **"How are values stored in the Parquet / AVRO files?"** → [Reference — Data Types](reference/data-types.md).
 - **"How do I downsample a fast signal or aggregate per signal?"** → [How-to guides](how-to-guides.md).
+- **"How do I write a script, and what can it see/return?"** → [Scripting](scripting.md).
+- **"My signal's value is an array — how do I handle it?"** → [Handle array-valued signals](how-to-guides.md#handle-array-valued-signals).
 - **"How do I control file rotation — by size, time, or count?"** → [How-to guides](how-to-guides.md).
 - **"Why is my data too fast / slow / laggy, and how does windowing work?"** → [Explanation — the processing-and-timing pipeline](explanation.md).
 - **"How does one route share a topic with another?"** → [Explanation — one route, one worker](explanation.md).

docs/explanation.md

@@ -80,90 +80,31 @@ caught at startup; runtime errors) drops the message rather than crashing the ro
 
 The built-in stages cover the common shapes — quality gating, value thresholds, downsampling,
 windowed reduction, whitelisting. **Scripting is the escape hatch** for logic they don't express:
-a derived field, a conditional drop, a reshape into a bespoke body, a multi-condition predicate.
-[Rhai](https://rhai.rs) is a small, sandboxed expression language embedded in the processor; you
-reach for it only when a built-in won't do, because the built-ins are faster and need no parsing.
+a derived engineering unit, a conditional drop, a reshape of a bespoke payload, a predicate that
+spans several samples or inspects an array. [Rhai](https://rhai.rs) is a small, sandboxed,
+Rust-native language embedded in the processor; a script is compiled **once at startup** and then run
+per message, with no I/O and a bounded op budget, so it can shape data but can't reach outside the
+pipeline.
 
-Scripting appears in **two places**, both backed by the same engine and the same message view:
+Scripting appears in **two roles**, both backed by the same engine and the same scope:
 
-- a **`filter` `script`** — a predicate that evaluates to a **boolean**; `true` keeps the message.
+- a **`filter` `script`** — a predicate that evaluates to a **boolean**; `true` keeps the message
+  (it fails *closed* — an error or non-boolean drops).
 - a **`script` stage** — a transform that evaluates to the **new body** (a map), or to `()` to
   **drop** the message.
 
-### Where the source lives — inline or a file
-
-A script's source is given **inline** (`{"script": "<source>"}`) or read from an external **file**
-(`{"script": {"file": "rules/derive.rhai"}}`). A relative file path resolves against
-`component.global.defaults.scriptsDir` (default: the working directory); an absolute path is used
-as-is. Both forms are **compiled once at startup** — a missing file or a syntax error fails the
-component immediately, before any telemetry flows, rather than at the first message. Inline is for a
-one-liner; once a script grows past a line or two, move it to a file (it version-controls cleanly,
-needs no JSON string-escaping, and ships as a deployment artifact — see [the how-to](how-to-guides.md#use-an-external-script-file)).
-
-### The scope a script sees
-
-Before each evaluation the stage builds a fresh **scope** from the current message and binds these
-variables (the same set for a `filter` predicate and a `script` transform):
-
-| Binding | Type | What it is |
-|---|---|---|
-| `topic` | string | the source topic the message arrived on |
-| `body` | map | the full message body (`body.signal`, `body.samples`, `body.device`, …) |
-| `tags` | map | the message-**envelope** `tags` metadata (`tags.site`, `tags.thing`, …) — **not** the signal |
-| `samples` | array | `body.samples` (or `[]` if absent) — each element a map with `value`, `quality`, … |
-| `value` | any | a convenience: the **first** sample's `value` (number, string, or bool) |
-| `quality` | string | a convenience: the **first** sample's `quality` (`""` if absent) |
-
-`body` and `tags` are the two roots. `value`/`quality`/`samples` are conveniences derived from the
-southbound shape — on a non-southbound payload `samples` is `[]` and `value`/`quality` are
-empty/`""`, but `body` always holds whatever JSON arrived, so a script is **payload-agnostic**: read
-your own paths off `body`.
-
-### What a script returns
-
-The return value is the whole contract — there is no other output channel:
-
-- **`filter` `script`** → a **boolean**. `true` keeps the message, `false` drops it. A non-boolean
-  result or a runtime error is treated as `false` (drop) and logged at WARN — a filter fails *closed*.
-- **`script` stage** → the **new body**. A map (`#{ … }`) replaces `body`; the envelope (`header`,
-  `tags`) is preserved. **`()`** (Rhai unit) **drops** the message. A result that can't convert to
-  JSON, or a runtime error, also drops the message (logged at WARN).
-
-So a transform either reshapes (`return a map`) or drops (`return ()`); a predicate either keeps
-(`true`) or drops (`false`). Nothing partially mutates the message in place.
-
-### Scripts are stateless (one message in, one decision out)
-
-Each evaluation gets a **fresh scope** and sees **only the current message** — there is no variable
-that persists from one message to the next, no counter, no rolling window inside a script. This is
-deliberate: it keeps a script a pure function (easy to reason about, impossible to leak memory
-across messages, safe to share one engine across every route). **Cross-message state lives in the
-built-in stages**, which the worker owns and drives on a timer: use `sample` for rate state and
-`aggregate` for windowed counters/min/max/avg. (A future revision may add an opt-in per-key script
-state object; it is intentionally not in this version.)
-
-### Worked examples
-
-```jsonc
-// filter: keep only when every sample is GOOD and under 100
-{ "filter": { "script": "samples.all(|s| s.quality == \"GOOD\" && s.value < 100.0)" } }
-
-// transform: rescale the first value and stamp the source topic, keeping the signal identity
-{ "script": "#{ \"signal\": body.signal, \"scaled\": value * 0.1, \"src\": topic }" }
-
-// conditional drop: keep the body as-is below the limit, drop it above
-{ "script": "if value > 1000.0 { () } else { body }" }
-
-// payload-agnostic: a non-southbound body, read your own paths and fold in envelope metadata
-{ "script": "#{ \"site\": tags.site, \"tempF\": body.temperature * 1.8 + 32.0 }" }
-
-// from a file (multi-line logic): rules/derive.rhai resolved under scriptsDir
-{ "script": { "file": "rules/derive.rhai" } }
-```
-
-`#{ … }` is a Rhai object-map literal; `||` is a closure; arrays have `.all` / `.any` / `.map`; `()`
-is unit (drop). JSON numbers arrive as integers or floats and `quality` as a string. Keep a script
-short — it runs on **every** message on the route's hot path, under the shared 1,000,000-op budget.
+A script sees the **message view** (`topic`, `body`, `tags`, `samples`, and the first-sample
+conveniences `value`/`quality`) plus the **runtime context** (`thingName`, `componentName`,
+`componentFullName`, `routeId`, `recvMs`) so a generic, reusable script can branch on which
+component/route/thing it runs in. Scripts are **stateless** — each evaluation sees only the current
+message; cross-message state belongs in `sample`/`aggregate`. Array-valued fields arrive as Rhai
+arrays, so a script can `for`/`map`/`filter`/`reduce` over them like any collection.
+
+> **Scripting has its own guide.** The dedicated **[Scripting page](scripting.md)** is the full
+> treatment: every scope binding, return and error semantics, a Rhai language primer (functions,
+> loops, ranges, `switch`, array methods), array handling, and a **cookbook of worked examples**
+> (derived units, array mean/peak/RMS, rate-of-change, reusable identity-stamping, payload
+> normalization, status mapping) — each explained goal → how → why, and each backed by a test.
 
 ## The two flows: from adapter to cloud
 

docs/how-to-guides.md

@@ -78,6 +78,38 @@ A `filter` stage takes exactly one form, checked in this order:
 
 ---
 
+<a id="handle-array-valued-signals"></a>
+## Handle array-valued signals
+
+**Goal:** process a signal whose sample `value` is an **array** (an OPC UA array node, a batched
+register read, a vector) — the OPC UA adapter and the wire format both carry these.
+
+Array values are first-class across the stages; pick the tool for the job:
+
+- **Aggregate across the elements.** The `aggregate` stage folds an array value **element-wise**, so
+  the numeric reducers span every element of every sample in the window — no special syntax:
+
+  ```jsonc
+  { "aggregate": { "window": "10s", "by": "body.signal.id", "fn": ["avg", "min", "max", "count"] } }
+  // value [10,20,30] then [40,50] in a window → avg 30, min 10, max 50, count 5
+  ```
+
+- **Filter on the elements.** A trailing `[]` flattens the array so an **any-element** predicate works:
+
+  ```jsonc
+  { "filter": { "field": "body.samples[].value[]", "op": "gt", "value": 100 } }  // keep if any element > 100
+  ```
+
+- **Compute over the array in a script.** An array `value` arrives as a Rhai array — `map`/`filter`/
+  `reduce`/`for` over it to emit mean, peak, RMS, counts, etc. See the
+  [Scripting cookbook](scripting.md#2-array-node-mean-peak-and-rms).
+
+- **Archive the array.** The file sink's default `rows` projection stores an array as JSON in the
+  `valueString` column; to spread it into one row per element, declare a
+  [`rows` projection](reference/data-types.md#rows-user-projection) with `explode`.
+
+---
+
 ## Reshape a message
 
 **Goal:** keep a whitelist of fields and/or stamp in literals.
@@ -107,12 +139,14 @@ A `filter` stage takes exactly one form, checked in this order:
 
 - A **`filter` `script`** returns a boolean — `true` keeps the message.
 - A **`script`** stage returns the **new body** map, or `()` to **drop** the message.
-- Scope exposed to both: `topic` (string), `body` and `tags` (maps — `tags` is envelope metadata, not
-  the signal), `samples` (array), and the convenience bindings `value` / `quality` (the first
-  sample's). An eval error or a non-JSON result drops the message (logged at WARN).
-
-For the full scripting model — bindings, return values, statelessness, more examples — see
-[Scripting with Rhai](explanation.md#scripting-with-rhai).
+- Scope exposed to both: the message view — `topic`, `body` and `tags` (maps; `tags` is envelope
+  metadata, not the signal), `samples`, and the conveniences `value` / `quality` (the first sample's) —
+  **plus the runtime context** `thingName` / `componentName` / `componentFullName` / `routeId` /
+  `recvMs`. An eval error or a non-JSON result drops the message (logged at WARN).
+
+For the full scripting model — every scope binding (incl. the runtime context `thingName` /
+`componentName` / `routeId` / `recvMs`), return/error semantics, a Rhai language primer, array
+handling, and a cookbook of worked examples — see the dedicated **[Scripting guide](scripting.md)**.
 
 ---
 

docs/reference/configuration.md

@@ -140,19 +140,25 @@ source is given **inline** or from an **external file**:
 
 Both forms are **compiled once at startup** (a bad path or a compile error fails fast, before any
 message flows); the shared engine is bounded (1,000,000 ops/eval) so a runaway script cannot wedge a
-worker. For the full scripting model — scope, state, return values, examples — see the
-[Scripting explanation](../explanation.md#scripting-with-rhai).
+worker. For the full scripting model — scope, state, return values, the Rhai language, and a cookbook
+of worked examples — see the dedicated **[Scripting guide](../scripting.md)**.
 
-<a id="rhai-scope"></a>**Rhai scope** (available to both `filter` `script` and the `script` stage):
+<a id="rhai-scope"></a>**Rhai scope** (available to both `filter` `script` and the `script` stage) —
+the per-message **message view** plus the constant **runtime context**:
 
 | Binding | Type | Value |
 |---------|------|-------|
 | `topic` | string | the source topic |
 | `body` | map | the message body |
 | `tags` | map | the envelope `tags{}` (message metadata — *not* the signal) |
 | `samples` | array | `body.samples` (or `[]`) |
-| `value` | any | the first sample's `value` |
+| `value` | any | the first sample's `value` (scalar **or array**) |
 | `quality` | string | the first sample's `quality` |
+| `thingName` | string | the IoT Thing name (`{ThingName}`) |
+| `componentName` | string | the short component name (`{ComponentName}`) |
+| `componentFullName` | string | the fully-qualified component name (`{ComponentFullName}`) |
+| `routeId` | string | the id of the route running the script |
+| `recvMs` | integer | this message's broker receive time (Unix ms) |
 
 <a id="key-paths"></a>
 > **Key paths** are dotted paths over the message: roots `body.` (the default when no known root

docs/reference/data-types.md

@@ -127,6 +127,10 @@ A sample `value` is narrowed to one typed column:
 > may lose precision for `|value| > 2^53`. An unsigned value above `2^63` is cast into the signed
 > int64 column.
 
+> **Array values** land in `valueString` as compact JSON (`valueType = "string"`) under the default
+> projection. To spread an array into one row per element, or to fold it in `aggregate`, filter, or a
+> script instead, see [Handle array-valued signals](../how-to-guides.md#handle-array-valued-signals).
+
 ## Quality & timestamps
 
 - **`quality`** is the normalized, protocol-independent verdict — `GOOD` \| `BAD` \| `UNCERTAIN`