Repository navigation
stury 0.0.2
Adds the CSV and TSV codec (stury:csv), the shared stury:core package, and the CLI's --from and --to.
-
stury:coreholds what the formats share. JSON's walk substrate moved from it to
stury:json/scan/walk: the rows and their stores (Node,Capacity,Number_Spelling), the caps (Limits,
MAX_DEPTH,MAX_SCALAR_DIGITS), the scan cursor, the query context and its matchers, and the typed walk's
state (Shape_Run,Shape_Frame).json's names for them are unchanged. -
stury:core/spellspells integers and floats for every format's writer (spell_i64,spell_u64,spell_f64,
shortest_digits,is_non_finite), moved fromstury:json/encode;stury:core/bitsholds the shared bit
and UTF-8 kernels. -
coredeclaresUnknown_Fields,Unmarshal_Hook,Type_Hook(W)andHooked_Plan(W)(with
hooked_plan_initandhooked_plan_destroy);jsonandcsvalias them,
so theirType_HookandType_Planarecore.Type_Hook(Marshal_Hook)andcore.Hooked_Plan(Marshal_Hook)
with the same fields.core.Value_Classnames the classes a union's arm is picked by. -
CLI:
--from FORMATand--to FORMAT(json,jsonc,ndjson,csv,tsv) choose how the input is read
and the output written.--ndjson,--jsonc,--csvand--tsvset both halves, and an explicit--from
or--toreplaces its half, in either order. Behaviour change:--ndjsonnow prints compact, one value per
line (it left the layout alone), so--ndjson --indent Nand--ndjson --tab, which printed indented JSON, are
now a usage error (exit 2);--to ndjsonis-c. -
CLI:
--to csvand--to tsvwrite each output as one record. An object is a row under a header of the first
object's keys (sorted under-S); a missing key is an empty cell and an extra key ends the run with exit 5.
--grow-headerholds the output and writes a header of every key in the order first seen, or in byte order
under-S(memory holds all the output). An array or scalar is a row with no header, and the two kinds do not mix. A string is its text, a
number is spelled as jq spells it,nullis an empty cell, and an array or object in a cell ends the run.
-r,-j,-a,-C,-c,--indent,--tab,--seqand--raw-output0are a usage error with it. A
filter that is only columns is printed in pieces, with the same output as the interpreter. -
CLI:
@tsv,@csvand@shwrite a NUL as\0, and@csv,@tsvand--to csvwrite NaN as an empty
cell, as jq does. They wrote the raw byte andnull. -
CLI: reading CSV, an empty cell with no quotes is
nulland a quoted""is""; every empty TSV cell is
null.stury --csv .writes back what it read, up to normalisation. TSV cannot tellnull
from"". -
CLI:
stury checktakes--fromand the aliases; CSV and TSV go throughcsv.validate, and a fault is reported
as in a filter run. -
CLI:
--csvand--tsvread CSV (RFC 4180) and TSV (jq's@tsvescapes). The first record of each file names
the fields, and each later record is one input, an object of strings (stury --from csv -r '[.a, .b] | @csv'
writes back what it read). A repeated header name, a record of another width and a fault in the text end the
file, reported asFILE:LINE:COLUMN: faultwith exit 5. A file up to 64 MiB is read whole. A larger file, a pipe
and a file under-nare read in windows, so memory holds a window and the longest record. A filter that is only
columns (.name,.a, .b,[.a, .b]under--from csv -c, an object of columns,., andselect(F)of columns)
is printed from the cells, in pieces on every core for a whole file. Not with-R,--seq,--ndjson,--jsonc,--streamor each other. -
stury:csv, a CSV codec.validateand therecordsiterator check every byte of RFC 4180 text (quote
placement, CR before LF, UTF-8, one field count throughout).field_textreads a field's text.unmarshal
reads records into a list of structs on the shared type plan, with columns named by a header or taken in
order, and judges each cell against its field's type.marshalwrites such a list as text thatunmarshal
reads back into equal values. -
CSV dialects.
validate,records,field_text,unmarshalandmarshaltake aDialect(delimiter,
Quoting.Nonewhere a quote is text,Escapes.Backslash, a comment byte, ragged records).TSVis
tab-separated values with backslash escapes (\t,\n,\r,\0,\\). They also take aStrictness
(.Structuralskips the UTF-8 check;unmarshalunder.Lazyjudges only the cells it reads) and
Limits.max_fields.marshalrefuses with.Unwritablea cell its dialect cannot spell so that it reads
back, and a record that would be a blank line. Thedelimiteroptions are nowdialect.delimiter. -
CSV documents and edits:
buildkeeps the field spans of a text as aDocument(row_count,column,
get_string,get_int,get_float,get_bool,get_name),findandfind_columnread one cell or one
column by path (.[3].name,.[3][1],.[].name) off the front of the text, andset,insert,remove
andsplicereturn the text with a cell, a column, a record or a span changed and every other byte kept. A
refusal is aPath_Error; the edits spell values asmarshaldoes and refuse what would not read back as the
same table. -
CSV
Plan_Opts.hooks: aType_Hookgives a type aMarshal_Hook(appends the cell's text, whichmarshal
quotes) and anUnmarshal_Hook(gets the decoded cell text, never an absent cell, and may refuse it); a field
of a hooked type is a cell whatever its type. -
CSV
Unmarshal_Opts.duplicates(Duplicates:Refuse,First,Last,All) for a header that names a column
twice, andunmarshal_check, which judges a text exactly asunmarshalwould and keeps nothing. -
CSV
marshal_append(appends to a buffer, which a refusal leaves as it was) andmarshal_to_writer(streams
in bounded chunks; addsMarshal_Error.Write_Failed).marshalis a wrapper over the same writer. -
CSV
write_recordappends one record of text cells ([]Maybe(string), nil an empty cell that reads back absent)
to a buffer in a dialect, spelling each cell asmarshaldoes. A blank line, which reads back as no record, is
.Unwritable. -
error_position,render_errorandrender_error_to_writerfor CSV, sharing JSON's implementation through
core; JSON's names and outputs are unchanged. -
Path_ErrorandPath_Error_Kind: one refusal type for the path procedures of every format, with each
format's docs listing the kinds it reports.json.Edit_Errorandjson.Edit_Error_Kindare now
json.Path_Errorandjson.Path_Error_Kind, whose kinds are renumbered and gain CSV'sUnsupported_Path,
Bad_OptionandBad_Row. -
Marshal_Error: one marshal refusal for every format;json.Encode_Errorandcsv.Marshal_Errorare its
aliases.json.Encode_Errorgains CSV'sBad_OptionandUnwritable, which JSON never reports. A path
refusal'sencodefield is aMarshal_Error.render_errorandrender_error_to_writer
write it for CSV as for JSON. -
Duplicates: one policy for repeated keys in every format, in core. Its new zero value.Defaultis the
format's default (JSON.Last, CSV.Refuse), so a zero-valued option reads as before.json.Duplicates
andcsv.Duplicatesare its aliases. JSON's is now au8. -
Error_Kind.Field_Count, the CSV fault of a record whose field count is not the first record's, and
Error_Kind.Limit_Fields, of a record with more fields thanLimits.max_fields. JSON never reports them. -
CSV streaming and threads:
csv.records_fromreads the records of anio.Readera chunk at a time, with the
same records, offsets and fault asrecordson the whole text.csv.each_recordruns a task on pieces of a
text on aBatch_Runner's threads.csv.validatetakes a runner too and gives the same answer as without one.
Limits.max_record_bytescaps a record (Error_Kind.Limit_Record) and so bounds the windowrecords_fromholds. A
Recordcarries thesrcits fields index.Batch_Runneris defined instury:core;json.Batch_Runner
names it. -
RequestandPreparedname their type parameter$KINDS(was$Q), as the generic procedures that take
them do. Callers that writeRequest({.Path})are unaffected. -
marshalwrites an emptyjson.Digitsasnull, as it writes an emptyjson.Raw, instead of failing with
.Bad_Raw. -
render_errorof anUnmarshal_Errorends with its path (1:11: Wrong_Kind at .a.b), with…when the path
was truncated. -
unmarshal's.Unsupportedfor a,stringfield, or for two fields writing one key, names that field in the
error path. -
scanat.Structuralchecks the UTF-8 of values no path reads, asvalidatedoes: invalid UTF-8 anywhere
in the document now fails the scan. -
CLI:
halt_error/1with a non-number argument names its input as jq does (number (1) halt_error/1: number required). -
CLI:
select(PATH OP literal)on a large single document reads only its path, and prints the text only when it
keeps it, instead of building the whole document.