Pipeline Formats

Pipeline formats

Conversion descriptors belong to the pipe boundary, not either command's arguments.

Boundary	Operation
value | stage	Pass a value, or connect external byte streams.
value |:json stage	Encode the outgoing value as JSON.
command json:| stage	Decode incoming JSON.
command json:|:Record stage	Decode and convert into Record.
producer csv:|:tsv consumer	Decode CSV records, then encode each as TSV.

When both sides specify conversions, decoding happens first. A name after |: can denote a registered format or a type; conflicting interpretations are an error. An explicit format must be supported. Conversion failure does not fall back to printing a display string.

JSON into an application record

struct Forecast { pub city: String, pub degrees: Float }
let $forecast: Forecast
let $status = printf '%s' '{"city":"Oslo","degrees":8}' json:|:Forecast $forecast
echo $forecast.city $forecast.degrees $status

Oslo 8.0 ExecResult(0)

Tabular records

CSV, TSV, and SSV decode one [String] row at a time. They use comma, tab, and space separators respectively. Quoted fields support separators and newlines; a doubled quote represents a quote inside a quoted field. There is no automatic header in an untyped row stream.

fn describe($row: [String]): String
{
    let ($place, $seats) = $row
    "${place} has ${seats} seats"
}
let $descriptions: [String]
printf '"Garden, east",12\nHarbor,8\n' csv:| describe | $descriptions
echo $descriptions[0]
echo $descriptions[1]

Garden, east has 12 seats
Harbor has 8 seats

When decoding into records, the first row supplies field-name headers, and fields are converted to their declared types. A final array destination gathers the records. A header-removal stage can handle a header when plain rows are desired.

Shared conversion methods

Formats use the normal conversion machinery: String::from_<format> encodes, and <format>::from_string decodes. Tabular row parsing belongs to those converters, rather than a second parser hidden in pipeline execution. Custom formats can expose the same methods. Explicit destination types use conversion methods and supported structural mapping.

An outgoing annotation encodes a source collection as a whole, so a JSON array can be sent as one document rather than as unrelated JSON scalar documents. See JSON for exact documents and richer mapping controls, and Item streams for explicit lazy processing outside pipe syntax.

---

Related: Pipelines · Conversions · Structs · JSON