zy v1.0.43

Official binary release for zy 1.0.43.

What's changed in 1.0.43

Changed

• Filesize units now match nushell (SI/metric): 1kb is 1000 bytes (was 1024), 1mb is 10^6, and so on, while the IEC *ib forms stay binary (1kib is 1024). Display uses B/kB/MB/GB/TB/PB with the value truncated to one decimal, like nu. If you relied on kb=1024, use kib.

Added

• to msgpack / from msgpack round-trip values through the MessagePack binary format ([foo, 42, false] ↔ the packed bytes).

• encode / decode support text charsets via iconv: shift-jis, euc-jp/euc-kr, utf-16/-le/-be, big5, gb18030, latin1, windows-1252, and more, with --ignore-errors. utf-16 is decode-only.

• nu-check validates nushell syntax with zy's own parser: '1 + 1' | nu-check is true, 'if true {' | nu-check is false; --debug prints the parse error.

• Typed math aggregations: math sum/avg/min/max/median/mode preserve the unit, so [1min 2min 3min] | math sum is 6min and [5kb 10mb 200b] | math median is 5.0 kB; datetime supports min/max. Mixed numeric input is unchanged.

• str length / str substring / str index-of take --grapheme-clusters (-g) to count/index by extended grapheme cluster (flags, ZWJ sequences, combining marks) instead of bytes.

• More ansi codes: cursor_on/cursor_off, cursor_blink_*, screen/line erase, cursor movement, attr_*/reset_* aliases, and nu's single-letter aliases (r g y b m p c w k i u d h s n).

• job namespace (nu-style closure-thread jobs): job spawn { … } runs a closure on a background thread and returns its id, with job id, job list, job kill, job send/job recv (a thread-safe per-job mailbox), and job flush. Closure execution is serialized against the main thread by a per-command executor lock (released while a worker blocks on sleep/job recv), and a worker's exit-status is thread-local, and the whole thing is TSan-clean. The bash-style jobs process control is unaffected.

• describe -d / describe --detailed returns nu's detailed record instead of the plain type string: a nested {type, detailed_type, value, …} with length for lists and binary, columns for records, and a per-element table for lists, built recursively so each nested value is itself a detailed record. The detailed_type field matches nu's full type spelling (table<a: int> for a list of records, list<oneof<int, string>> for mixed lists, recursive record<a: record<b: int>>). nu's rust_type field is intentionally omitted (it exposes nushell's internal Rust types, meaningless in a C shell). --no-collect / -n is accepted but a no-op, since zy is eager and has no streams. The plain describe (no flag) is unchanged.

• into sqlite <path> writes the pipeline table into a sqlite database file (default table main, --table-name to override), matching nu.

• Timezone-aware, sub-second datetimes (ZY-062): a datetime now carries its source UTC offset and nanoseconds, so into datetime round-trips +HH:MM and fractional seconds, into record includes millisecond/microsecond/nanosecond and the source timezone, into int gives the full nanosecond count, and into string matches nu's Sat Oct 10 10:00:00 2020 form. into datetime also parses a {year..timezone} record, a list of strings, %P am/pm, %z offsets, and the European d.m.Y forms.

• transpose -d/--as-record and the -l/--keep-last / -a/--keep-all dedup companions; rename --block {closure} renames each column by running the closure on its name.

• Digit separators in number literals: 1_000, 1_234_567, 3_141.59.

Fixed

• A bare builtin statement before ; no longer prints its value (job id; 99, whoami; 99 → just 99, matching nu): nu_output now honors force-capture like into_output.

• to json of a binary emits nu's byte-int array (0x[ff aa 01] → [255, 170, 1], indent-aware), not a quoted "0x[ff]" string.

• to nuon of a duration and binary match nu: a duration serializes as its raw nanosecond count (1day → 86400000000000ns) and a binary as uppercase no-space hex (0x[ff] → 0x[FF]), so they round-trip and describe -d of a duration/binary matches nu in every field.

• Duration and filesize arithmetic keep their unit: (500day + 4hr + 5sec) is a duration (was a float), so into record breaks it into week/day/hour; dur / dur is a float ratio.

• flatten --all recurses into nested list/table columns; a nested [[ ]] table literal inside a cell now parses.

• join accepts an inline [ … ] table as its right side and supports --suffix/--prefix for colliding columns.

• insert/upsert with a dotted cell-path key (a.0.b) descend instead of making a literal column; group-by --to-table emits [{group, items}, …]; find matches list elements by value for numeric/filesize terms; seq honours an explicit 3-arg step like nu (ZY-061); inspect as the last stage forwards its value to stdout (ZY-060).

• Empty quoted strings survive in records and tables (['', 'a'], {a: '', b: 1}, {a:'' b:1}, [[a]; ['']]), and a $"…" value in a brace block (if c { $"label: (x)" }) no longer mis-parses as a record.

• to nuon quotes digit/dot/space-bearing strings like nu ({c1: 1} → {"c1": 1}).

• Cell-paths on list and record literals (ZY-053): [10 20 30].1 → 20, {a:1}.a → 1, {a:{b:2}}.a.b → 2.

• $nu.os-info is a record (ZY-056) with name/arch/family/kernel_version, so | get name and .arch work.

• String interpolation inside a record value (ZY-054): {a: $"x ($n)"} evaluates the ($n) block.

• bits shr --signed sign-extends and the width auto-sizes (ZY-044): -2 | bits shr 1 --signed → -1
  (was 127), and ints over 255 no longer wrap (256 | bits shr 1 → 128). -s/-n short flags now parse (ZY-039).

• const accepts a list or record value (ZY-038): const x = [1 2 3] binds the whole list, not just [1].

• The fff MCP server advertises usage instructions so a fresh client knows the grep/find_files/multi_grep tools.

• Whole-valued floats print with .0 (ZY-057), matching nu: 10 / 2 → 5.0, 9 | math sqrt → 3.0,
  string interpolation $"(10 / 2)" → 5.0. into string still drops it (2.0 | into string → 2).
  math abs of an int returns an int; math median of an even-length list returns a float.

• const accepts both const X=42 (POSIX) and const x = [1 2 3] (nu) forms again. The fused form had
  regressed to "not a valid identifier".

• A def function now returns an integer value (ZY-042). def double [x] { return ($x * 2) }; double 5
  gave nothing because an integer return was always read as a POSIX exit code. A def now puts the value
  on the pipeline like nu, while a POSIX name() { … } still uses the exit code.

• select / reject / update error on a missing column (ZY-045), like nu, instead of silently dropping it
  or creating it. A trailing ? marks a column optional (select a?); use upsert to add a column.

• where handles column arithmetic (ZY-055): where a mod 2 == 0, where a + b > 6 and similar now
  filter per row instead of returning nothing.

• A cell-path on a record literal works (ZY-053): {a:1}.a → 1, {a:{b:2}}.a.b → 2.

• sort compares by value type (ZY-043): a list of genuine strings sorts lexicographically like nu
  ("2 10 1" | split row " " | sort → 1, 10, 2) instead of coercing to numbers.

• split chars and str reverse are UTF-8 aware (ZY-040): 'café' | split chars → c a f é and
  'café' | str reverse → éfac, no more byte mojibake.

• Records can mix spaced and compact colon entries (ZY-059): {b: 4, a: 3, c:1} parses (and so
  … | sort -v), in any order of b: 4 / c:1 style.

• A huge seq or range no longer freezes the terminal. seq 4 600000000 and a
  bare 1..600000000 used to allocate the whole list and exhaust memory; they now
  error with a clear hint past 10M elements. This also removes the old silent
  truncation of a range literal at 100k, so ranges up to 10M materialise fully
  (1..200000 | length is 200000, not 100000). Lazy streaming (so big ranges
  don't materialise at all, like nu) is planned separately.

• where shorthand matches Nushell. where a > 1 and b > 1 now honours the
  whole and/or chain instead of just the first comparison; where p.age > 18
  follows a dotted column path; where c in ["red" "green"] tests membership; and
  where a >= 2 / <= 25 / == 2 / != 2 work (the two-char operators were being
  split). where $it mod 2 == 0 over a list evaluates with full precedence.

• More commands take closures and map over lists. sort-by {|x| …} and
  group-by {|x| …} work on a plain list of values; into bool / into string /
  into filesize / into record map element-wise over a list (and into record
  reads a list of [key value] pairs); parse --regex returns every match per line
  (not just the first) and names unnamed groups capture0….

• Records, ranges, and control flow. let r = try { … } catch { … } assigns the
  evaluated value; a bare (1 + 2) at the prompt prints 3; a quoted string that
  matches a command name (if false {"a"} else {"z"}) stays a string;
  error make {msg: ("a" + "b")} evaluates the field; a runtime error binds
  catch {|e| $e.msg}; the range 1..3..9 reads 3 as the second element (step 2)
  like nu; match arm bodies run a nested match or a ("x" | cmd) pipeline;
  [[a b]] is a list-of-lists; and def --wrapped foo […] defines foo.

• String and value fixes. mut s = "a"; $s = $s + "b" / += / ++=
  concatenate; str kebab-case / snake-case no longer double the separator on
  CONST_NAME; $env.HOME? drops the optional marker; an empty string keeps its
  type in a list/record/fold; str substring (-3)..(-1) works; and a filesize
  serializes to nuon as a round-trippable <bytes>b.

• A let x = {k: "v"} record no longer corrupts a string value (it had started
  turning the value into a phantom key), and update/upsert with a dotted-path
  closure (upsert a.b {|v| $v + 10}) no longer reads uninitialised memory.

• values on a table returns each column's values; reject takes a nested
  cell-path. [[name meaning]; [ls list] [mv move]] | values is now
  [[ls, mv], [list, move]] (it used to pass the table through), and
  {a: {b: 3, c: 5}} | reject a.b removes the nested key to give {a: {c: 5}}.

• A [[ … ]] literal with a ; is always a table, even with commas in the
  header. [[a, b]; [1, 2]] and [[a, b]; [1, 2] [3, 4]] | reject 1 parse
  correctly again (a regression from the comma-list change, where the comma was
  read as "list literal" before the ; was seen).

• String interpolation works inside closures, and escaped parens unescape.
  echo $"v ($x)" / echo $'($k) ($v)' inside each/items/do no longer
  word-splits the interpolated value (it was producing [[v, 1]] instead of
  ["v 1"]); the interpolation is now self-delimiting so a re-lexed closure body
  keeps it as one argument. Separately, $"escaped \(not interp\)" now renders
  escaped (not interp) instead of leaking the backslashes.

• flatten turns a record into a table and expands a list column.
  {a: 1, b: 2} | flatten is [[a, b]; [1, 2]]; a single list-valued field
  expands into rows ({a: x, d: [1 2 3]} | flatten → three rows), a record-valued
  field merges into the parent, and flatten <col> picks the column. Previously a
  record passed straight through unchanged.

• where COL =~ REGEX does a real regex match, not a substring.
  where n =~ 'b.r', where n =~ '^a', where f =~ '.txt$' now match like nu
  (they were doing a literal strstr, so any metacharacter returned nothing). Also
  fixes anchored ^/$ matching against a table cell (ZY-049).

• A record-returning closure body works in every higher-order filter.
  each/items/reduce/where/filter/any/all/collect/chunk-by/
  skip-while/skip-until/take-while/take-until no longer mis-parse a
  {|x| {a: $x}} body as a stray a: command (the closure-brace stripper used to
  eat the record literal's own closing brace).

• [[a, b], [c, d]] and [[N, u, s]] parse as list literals, not bash tests.
  A comma-bearing [[ … ]] was dispatched to the POSIX [[ … ]] test builtin
  ("test: extra argument"); it is now a list-of-lists, while genuine
  [[ -f x ]] / [[ 5 -gt 3 ]] tests still work.

• An unmatched glob passes through literally in the interactive shell, so
  zy -c and interactive zy agree. […] | get A? now works at the prompt (the
  interactive globber used to reject A? with "no matches found"); a real glob
  still expands, and zsh-style erroring is available via setopt nomatch.

• Record literals with multi-token values now parse correctly (ZY-047).
  Computed values {a: (1 + 1)}, subexpressions {n: ([1 2 3] | length)},
  space-separated lists {xs: [1 2 3]}, quoted scalars that stay strings
  {a: "1"}, quoted keys {"a b": 1}, and nested records carrying any of these
  {a: {b: (2 * 3)}} all work now; previously they were mangled (e.g.
  {a: (1 + 1)} became {a: "(", "1": ")"}). The record-literal reassembler now
  keeps each value as a balanced, quote-preserving span and the value evaluator
  handles parenthesized/list/record spans.

• A whole-valued float keeps its .0 in to nuon and to json, so
  1.0 | to nuon is 1.0 and a float survives a to nuon | from nuon (or json)
  round-trip as a float instead of turning into an int. into string still gives
  1 (matching Nushell), and scalar/table display is unchanged.

• to nuon quotes strings that look like a number, bool, or null, so a string
  like "1" round-trips as a string instead of reparsing as an int. Real numbers,
  bools, and ordinary words are unchanged.

• from tsv and from ssv infer cell types (int, float, bool, null) like
  from csv already did and like Nushell, so a numeric TSV cell is an int rather
  than a string. (from csv/tsv/ssv now share one inference helper.)

• rename --column {old: new} renames columns by name instead of treating the
  flag and map as positional names; columns not in the map keep their name.
  Positional rename <new1> [new2 …] is unchanged.

• zip reads the whole second list. [1 2 3] | zip [a b c] now gives
  [[1, a], [2, b], [3, c]] instead of stopping at the first element; it uses the
  same argument helper as append/prepend.

• url parse exposes a decoded params table and url join accepts a
  params field, so url parse | get params and building a URL from
  {scheme, host, params} both work (and url parse | get params | url build-query round-trips).

• drop nth handles open-ended ranges. drop nth 2.. drops from index 2 to
  the end, ..2 drops the first three, and ..<2 the first two; closed ranges
  and plain indices are unchanged.

• port <start> <end> returns the first free port in the range instead of
  erroring.

• timeit { ... } prints only the measured duration. The block's return value
  used to leak to the output (timeit { 1 + 1 } printed 2 then the time); it is
  now discarded like Nushell, while an explicit print inside the block still
  shows.

• uniq-by supports --count, -d, and -u. --count returns a
  [[value, count]; ...] table, -d keeps only keys that repeat, and -u keeps
  only keys that appear once. They were parsed as positional arguments before.

• Fixed a heap buffer overflow in parse. Every template parse (e.g.
  "a=1" | parse "{k}={v}") read past the end of an internal buffer while
  rewriting the pattern, because the buffer was terminated one step too late. It
  is now terminated before that scan. (ZY-052)

• split row handles a --leading separator, keeps empty fields, and honors
  --regex/--number. 'a-b-c' | split row '-' works, 'a,,b' | split row ','
  keeps the empty middle field, and --number N keeps the remainder as the last
  element.

• parse accepts list input so lines | parse "{k}={v}" produces one row per
  line.

• sort sorts a record by key, or by value with -v/--values (it used to
  pass records through unchanged), honoring -r and -i.

• transpose --ignore-titles/-i works: it drops the leading
  original-names column and shifts the auto column names down by one.

• date commands. format date "%+" now prints an ISO-8601 timestamp instead
  of the literal %+; date humanize rolls over to years past twelve months
  (6 years ago, not 78 months ago); date to-timezone uses a datetime string
  on the input instead of today's date; and date list-timezone returns a
  [[timezone]; ...] table so where timezone =~ ... and get timezone work.

• update cells works on a table literal, e.g. [[a b]; [1 2]] | update cells { |v| $v + 1 }; the closure now runs over each record in a list<record>, not
  just a table/record value.

• window --remainder keeps the trailing partial windows ([1 2 3 4 5] | window 3 --remainder ends with [4 5] and [5]).

• group-by with no column groups a list by value into { value: [items…] }
  instead of erroring.

• url split-query keeps duplicate keys and decodes values. It now returns a
  [[key, value]; ...] table (so a=1&a=2 yields two rows) and form-decodes both
  fields (%20, %26, and + become space, &, space), instead of a record
  that collapsed repeated keys. url build-query accepts that table as well
  as a record, so url split-query | url build-query round-trips.

• url encode and url decode match Nushell. url encode keeps /, :,
  and . by default (so a URL like https://x/foo bar only gets the space
  encoded) and supports --all to encode everything but letters and digits.
  url decode no longer turns + into a space; it is strict, decoding only
  %XX.

• path command flags and shapes. path expand now collapses ./..
  lexically (so /a/b/../c becomes /a/c) even when the target doesn't exist or
  with --no-symlink; path dirname --num-levels N --replace X keeps the
  intermediate directories; path parse --extension tar.gz splits the stem at the
  given multi-part extension; path join accepts a {parent, stem, extension}
  record so path parse | path join round-trips; path relative-to of an equal
  path is empty; and which reports the built-in type and omits unresolved
  names (so which missing | length is 0).

• str replace --regex expands $1/$2/${n} capture references. They were
  copied through literally, so 'abc' | str replace --regex '(.)(.)' '$2$1' gave
  $2$1c instead of bac. References now splice in the matched group;
  --no-expand keeps them literal.

• str index-of --end returns the last match (and --range N..M limits the
  search); str substring 6.. (open upper bound) returns to the end of the
  string instead of ""; char -u/-i short flags work.

• into binary default is now the full int width, and --compact trims (the
  two were swapped). 255 | into binary is 8 bytes; --compact is 1.

• into int and into float validate the whole string. '123abc' | into int
  now errors instead of returning 123, and '2.5 ' | into float (trailing
  spaces) is accepted.

• into bool matches true/false case-insensitively ('TRUE' works) and no
  longer accepts yes/no; '0'/'1' still convert.

• math abs keeps the type. -2hr | math abs is 2hr (was 0), and the
  absolute value of an int stays an int.

• first 0, take 0, and last 0 now return an empty list instead of one
  element (the count was clamped up to 1). last 1 now returns a single-element
  list [x] to match Nushell, while last with no count still returns the last
  value directly.

• seq <first> <step> <last> reads the middle argument as the step. seq 1 2 10
  now yields [1 3 5 7 9] instead of [1] (the step and end were swapped).

• uniq --repeated works. zy only recognized --duplicates; Nushell's flag is
  --repeated (both are accepted now).

• str contains honors --ignore-case when it trails the search string, e.g.
  'hello' | str contains 'ELL' --ignore-case; previously a trailing flag was
  mistaken for the text to search for.

• math sin/cos/tan and math arcsin/arccos/arctan now honor --degrees.
  The flag was silently ignored, so 90 | math sin --degrees computed the sine of
  90 radians instead of 90 degrees. Forward trig now converts the input from
  degrees, and the inverse functions return the result in degrees. Radians stay
  the default.

• Pasting a table literal at the prompt no longer flashes "unmatched ']'" or
  drops the prompt line. The interactive bracket-balance check treated [[ and
  ]] as a paired unit, but a table literal [[a b]; [1 2] [3 4]] opens with
  [[ yet closes its header with a single ] before the ;. That corrupted the
  depth count, so the line was misread as unbalanced (spurious error) and
  incomplete (the prompt went into continuation mode). Brackets are now counted
  one at a time, which balances table and nested-list literals correctly. (ZY-051)

• open now parses more formats by file extension. It auto-parsed only
  json/csv/xml; .toml, .yaml/.yml, .tsv, .nuon, .ssv, and .ini came
  back as plain strings. They are now parsed into structured data (and --raw
  still returns text), so save data.toml followed by open data.toml round-trips.
  (ZY-050)

• save now writes the format that matches the file extension. Saving a table
  to data.csv used to write JSON regardless of the name. save now serializes by
  extension like Nushell, so .csv, .tsv, .yaml/.yml, .toml, .nuon,
  .xml, .md, and .html each get their proper format (and round-trip back
  through open/from <fmt>). .json and unknown extensions still write JSON;
  --raw still writes the value's string form unchanged. (ZY-037)

• bytes add, bytes remove, bytes replace, and bytes index-of now accept
  the short flags. Only the long forms were recognized, so -a/--all,
  -e/--end, and -i/--index were silently ignored and the command ran with
  its default behavior. The short aliases now match their long forms. (ZY-039)

• find now takes its flags and more than one search term. It used to use
  only the first argument and treat flags as the text to search for, so
  find --regex "^a" literally looked for "--regex". find now supports
  --regex (-r), --ignore-case (-i), --invert (-v), and several terms
  (a row or item is kept if it matches any of them). (ZY-039)

• to toml and to xml now serialize nested data instead of flattening or
  dropping it. to toml is a real serializer now: nested records become
  [section] tables (dotted [a.b] for depth) and lists of records become
  [[section]] arrays of tables, all valid TOML that round-trips through
  from toml. to xml follows Nushell's {tag, attributes, content} shape (the
  same one from xml produces) instead of dumping a generic <record>, so
  from xml | to xml round-trips exactly. (ZY-046)

• str ends-with and str starts-with now honor --ignore-case (-i). The
  flag was being read as the text to match, so 'FOO' | str ends-with -i 'oo'
  always returned false. Case-insensitive matching works now, and the default
  stays case-sensitive. (str contains -i already worked.) (ZY-039)

• A closure that builds a record now works in each. A body like
  each {|r| {name: $r.first, city: $r.town}} failed with a name:: No such file
  error because each stripped its closure braces too greedily and ate the
  record's own closing brace. Field projection and renaming with records now
  work. (Record values that are parenthesized expressions, like
  {n: ($x * 2)}, are still a separate open issue.) (ZY-048)

• to nuon no longer drops the data when you convert a parsed table. Piping a
  table from from csv, from tsv, or from ssv into to nuon returned an empty
  [] and lost every row. to nuon now serializes the table, renders a list of
  records with matching columns as a NUON table the way Nushell does, and feeds its
  output through the pipe so ... | to nuon | from nuon round-trips. (ZY-041)

• each and par-each no longer crash when mapping over a table. Running a
  closure over a table of two or more rows and then serializing the result, for
  example [{a:1} {a:2}] | each {|r| $r.a } | to nuon, aborted the shell with a
  heap error. Table rows are stored inline, and the per-row value handed to the
  closure was sharing that inline storage, so it got freed twice. Each row is now
  copied into its own record before the closure runs. (ZY-035)

• stor update no longer overwrites every row when you use Nushell's
  --where-clause. zy only recognized --where, so passing --where-clause
  (the Nushell spelling) silently dropped the filter and updated the whole table.
  stor update now accepts --where-clause and --table-name like Nushell, and
  rejects an unknown flag with an error instead of quietly running an unfiltered
  update. A deliberate update with no where clause still updates all rows. (ZY-036)

Verify your download

curl -LO https://github.com/iskandarputra/zyshell/releases/download/v1.0.43/zy_1.0.43_amd64.deb
curl -LO https://github.com/iskandarputra/zyshell/releases/download/v1.0.43/SHA256SUMS
curl -LO https://github.com/iskandarputra/zyshell/releases/download/v1.0.43/SHA256SUMS.minisig
curl -LO https://github.com/iskandarputra/zyshell/releases/download/v1.0.43/zy-release.pub

minisign -Vm SHA256SUMS -p zy-release.pub      # confirms maintainer signature
sha256sum -c SHA256SUMS --ignore-missing       # confirms the .deb hash matches

Full signing policy: docs/ops/RELEASE_SIGNING.md

VirusTotal scan

This release was uploaded to VirusTotal at publish time. Click through to see the live verdict from 65+ AV engines:

https://www.virustotal.com/gui/file/eb33e6fa36b7661ef600c7316546d3e93bab20c55504d540259606f639e78db0/detection

(Scans run within ~5 minutes of upload. If the page shows "Item not found", wait a moment and refresh.)

Try without installing

docker run --rm -it ghcr.io/iskandarputra/zyshell:1.0.43

Install (Debian / Ubuntu)

Download zy_1.0.43_amd64.deb from the assets below, then:

sudo dpkg -i zy_1.0.43_amd64.deb
sudo apt-get install -f