0.38.0

0.38.0 — Line-slicing and field-projection filters

Two small pipe filters that remove two workarounds: tailing a log used to need a shell binary (tail), and deduping a list of records used to need a hand-rolled loop. Now they're one filter each.

Upgrade impact: none (additive). Four new pipe filters; nothing changes for existing skills.

tail / head / lines — slice a string by line

Chain them on any string value. The headline case is tailing a log without a shell binary:

run:
    file_read(path="/var/log/app.log") -> LOG
    emit(text="Last 20 lines:\n${LOG|tail:"20"}")

file_read is filesystem-read-only — so file_read | tail reads and slices a log with no tail/grep on the shell allowlist and no unsafe=true. That's the whole point: you get log-tailing without granting a shell binary.

• head:"N" — the first N lines
• tail:"N" — the last N lines
• lines:"M-N" — the 1-indexed inclusive range (lines:"10-20" → lines 10 through 20)

They're CRLF-tolerant and drop the trailing empty line a file's terminal newline would otherwise leave (so tail:"1" of a file ending in "…last line\n" is last line, not "").

They never throw. A count that's too big, zero, negative, or malformed just clamps to what's there or yields an empty string — a bad ${LOG|tail:"${N}"} can't abort your run.

pluck — project one field from a list of records

pluck:"field" turns a list of objects into the list of that field's values. The driver is structural dedup — "have I seen this id before?" — which used to be a hand-rolled string-match loop:

run:
    $ data.recent -> ITEMS                       # [{id, title, ...}, ...]
    $set SEEN = "${ITEMS|pluck:"id"}"            # ["a1","b2","c3"]
    if ${CANDIDATE_ID} not in ${SEEN}:
        emit(text="${CANDIDATE_ID} is new — processing")

pluck omits any element whose field is missing or null, and any element that isn't an object — so you get a clean list of the values that are actually there. It's single-level (pluck:"id", not pluck:"meta.id").

Gotchas

• pluck output composes with in / not in / |length / |contains — but bind it to a variable first. pluck produces a JSON array string, which those all understand. What you can't yet do is inline it on the right of in: if ${ITEM.id} in ${ITEMS|pluck:"id"}: doesn't parse. Bind it ($set SEEN = "${ITEMS|pluck:"id"}") then test against ${SEEN} — as above. (The inline form is a planned follow-up.)
• Filter arguments are quoted: tail:"20", lines:"2-10", pluck:"id" — same as contains:"x".
• The line filters work on a string. Handed structured data, they slice its JSON serialization, which is rarely what you want — reach for them on text (a file, a log, an $ llm reply), and use pluck for lists of records.

Feeding pluck a demo list

If you're trying pluck out, note that hand-writing a list of objects inline ($set X = [{...}]) doesn't build real objects — it splits on the commas. The honest way to get a record list into a skill is a connector ($ data.recent -> ITEMS) or a JSON file:

run:
    file_read(path="/data/items.json") -> ITEMS     # a JSON array of objects
    emit(text="ids: ${ITEMS|pluck:"id"}")           # pluck parses the JSON directly