v1.6.0
A broad feature release on top of 1.5.0. Highlights: bigBed / bigWig
and 2bit support via a vendored libBigWig and a hand-rolled 2bit
reader; full TUI exploration suite (column stats, sort, show/hide,
live filter, copy cell); generic Parquet range queries with column
auto-detection; markdown output and LociSSD validation; coordinate-
convention selector with UCSC and NCBI labels; color themes; LTO +
ccache static builds; and ARM64 release artifacts.
Changed
- Smarter inline truncation for
list/mapcells. The
truncator now walks every top-level comma and picks the largest
leading-element prefix that fits the column. Previous behavior
always dropped to[first, …]after the first comma, even when
[first, second]would have fit. Most visible on VCF INFO
expansions, BAM tags, and other multi-element list cells common
in real data. Falls back to char-truncation only when even one
element + ellipsis doesn't fit.
Added
-
TUI copy cell (
ykey) — copies the top-left visible cell to
the system clipboard via OSC52. Works through SSH and tmux 3.3+
with noxclip/pbcopyhelper required (modern terminal
emulators intercept the escape directly). Status bar flashes
copied: <preview>until the next key; auto-clears. -
UCSC 2bit (
.2bit) — sequence container used for genome
references (hg38.2bit, mm10.2bit). vv exposes the sequence index
rather than decoded bases (chromosome-scale strings are too large
to materialise):name,length,n_blocks(unknown-base runs),
mask_blocks(soft-masked-region runs). Endianness is detected
from the signature; long-offset 64-bit 2bit is rejected with a
clear message (usetwoBitToFafor that variant). -
--theme <name>— color palette selector. Built-in themes:
default,dark,light,solarized-dark,solarized-light
(solarizedaccepted as a synonym forsolarized-dark). Affects
both the non-interactive ASCII table (ANSI escapes) and the
ncurses TUI. Each theme provides a 256-color palette and a
16-color fallback for older terminals; thelighttheme also
suppresses zebra stripes so the table reads cleanly on a bright
background. Unknown theme names produce a clear error listing the
available choices. -
TUI live filter (
&key) — long-requested. Opens a
&<expression>input bar; expression grammar matches the
--filterCLI flag. Rows not matching the predicate are hidden
from the display; status bar showsfilter:<expr> N/M(rows
kept / total). Composes with sort (s): one full-file pass
rebuilds the combined view via the existingsource_row(display)
indirection. Empty input orEsc(in normal mode) clears.Internals: a new
rebuild_display_order()orchestrates both
sort and filter into the singlesort_order_permutation. When
both are active, rows that pass the filter are collected with
their sort keys in one pass and sorted at the end. -
--decode-threads N— separately size Arrow's CPU thread pool
(Parquet column decode and CSV / TSV parallel parsing) without
touching htslib's thread count. Defaults to--threads; useful
when cold reads bottleneck on column decompression. Bounded at
2 × hardware_concurrency().
Changed
- Release pipeline: now builds both x86_64 and aarch64
(ARM64) static Linux binaries on every tag. The release workflow
matrices overubuntu-latestand the new GitHub-hosted
ubuntu-22.04-armrunner — native builds on each architecture,
no QEMU emulation. Tarballs are named
vv-<ver>-linux-<arch>.tar.gz; a mergedSHA256SUMScovers both. - AlmaLinux 8 Docker build: dropped the
-march=x86-64
baseline flag from the build-app CMake invocation so the same
Dockerfile builds on aarch64. (-march=x86-64is the default
target on x86_64 gcc anyway, so the change is a no-op there.) - AlmaLinux 8 Docker build: rebuild every static dep + vv itself
with-flto=auto. gcc-ar / gcc-ranlib preserve LTO IR in archives
so the final link can drop unused code across library boundaries
(not just within each.a). Significant text-segment shrink vs
the previous non-LTO 15 MB static binary. - AlmaLinux 8 Docker build: install
ccache(via EPEL) and put
its compiler-symlink dir ahead of gcc-toolset-12 onPATH. A
BuildKit cache mount (id=vv-ccache) persists the compile cache
across docker builds. Requires Docker BuildKit + buildx.
Added
- TUI interactive-exploration batch — three new keybindings turn
the ncurses viewer into a real exploration tool:Sopens a per-column stats popup (count, nulls, min, max,
mean, distinct count) for the leftmost visible column. Full-file
scan with progress in the status line.ssorts by the leftmost visible column. Repeatedson the
same column toggles ascending / descending;son a different
column re-sorts ascending;uclears the sort. Numeric types
sort by raw value, others lexicographically; nulls last. Search
and the detail pane follow the sorted display order via a single
source_row(display)indirection that routes all cache lookups
through the sort permutation when active.copens a column show/hide picker overlay;Spacetoggles
the column under the cursor, hidden columns drop from the
layout, and the status bar shows ahidden:Nindicator.
--tail N— show the last N rows instead of the first N.
Mirrors-nbut operates on the tail. Reuses the
MemoryTableSourceadapter: every chunk is read (through any
active--filter) and the last N rows are sliced off, so the
result renders identically through every view / export mode
(table, TUI, TSV, CSV, JSON, Markdown, Parquet).--coords UCSC|NCBI— pick the coordinate convention for
-r. UCSC (default; aliases0-based,bed) means 0-based
half-open as in BED, bigBed, bigWig, BAM and LociSSD — the
convention introduced by Jim Kent's UCSC Genome Browser source
tree in 2000. NCBI (aliasesGenBank,1-based,tabix)
means 1-based inclusive as in GenBank (1982), VCF, GFF, and the
samtools / bcftools / tabix command lines.vv -r chr1:101-200 --coords NCBI filematches the same rows as the UCSC default
vv -r chr1:100-200 file. Conversion happens once in
apply_region_modifiers; downstream sources see normalized
UCSC.--regions-fileis always BED (UCSC) per the spec.--parquet -— write Parquet to stdout. Parquet's
footer-at-end requires seekable writes, so the data is spooled
to anmkstempstemp file under/tmp, then streamed to stdout
and unlinked. Bit-identical to writing to a file path. Enables
vv ... --parquet - | toolpipelines.- Generic Parquet range queries —
-r chr1:1000-2000 file.parquet
now works on any Parquet file with chrom/start/end columns, not just
LociSSD. Chromosome / Start / End column names are auto-detected
(common variants:Chromosome/Chrom/Chr,Start/POS/
chromStart,End/Stop/chromEnd). When auto-detection isn't
clear (or your columns have unusual names) use
--region-cols Chr,Start,End. Row-group pruning uses Parquet
ByteArray statistics on chrom plus Int statistics on Start/End;
per-row filtering inside surviving row groups is correct for
unsorted files too, just less efficient. Dictionary-encoded chrom
columns are handled transparently. --validate— LociSSD invariants checker. Verifies the
lociSSD_manifestfooter parses, the required columns
(Chromosome / Start / End / MaxEndSoFar) have the right types,
the manifest's per-chromosome row ranges cover every row
contiguously, rows are sorted by(Start, End)within each
chromosome,MaxEndSoFar[i]matchesmax(End[chrom_first..i]),
and every row'sChromosomelabel agrees with the manifest's
window. PrintsPASS/FAILper check, caps repeat violations
at 5 lines per category, exits non-zero on any failure.--md/--markdown— write a GitHub-flavored markdown table
to stdout. Useful for pasting query results into issues, READMEs,
and reports. Cells escape|as\|and turn embedded newlines
into<br>. Honours--select,--filter,-n,--no-header,
-r,--sample. Numbers keep their_thousands separators
so the rendered table stays readable.- bigBed / bigWig support via a vendored copy of
libBigWig (0.4.8, MIT-licensed,
compiled with-DNOCURLso only zlib is required). Source lives
undervendored/libBigWig/; CMake compiles it as a private static
library (libbigwig.a) and links it intovv. The static AlmaLinux 8
Docker build picks it up automatically — no new external dependency
for any distribution channel. - bigBed: the embedded autoSql definition is parsed into typed
Arrow columns (signalValue,pValue,qValue, …). A small
~150-LOC hand-rolled parser handles primitives,char[N]fixed-
width strings,enum{…}/set{…}(as strings), and arrays
(int[N]/int[fieldName]→ Arrowlist<int>). - bigWig: rendered as four columns (
chrom,start,end,value). - Range queries (
-r chr1:78-99) use libBigWig's overlap APIs
(bbGetOverlappingEntries/bwGetOverlappingIntervals); no
external index needed. - User manual at
docs/USAGE.md, with worked examples for every
flag, every output mode, the LociSSD / tabix / BCF range-query
flows, and a "by-workflow" cookbook section.docs/build_docs.sh
renders it to self-contained HTML and PDF via pandoc (with a
texlive → chromium-via-HTML fallback). - BCF range queries —
-r chr1:100-200 file.bcfnow works, using
bcf_itr_querysover an existing.csi/.tbiindex. Missing
index produces a clear hint pointing atbcftools index. --regions-file <BED>— read additional windows from a BED file's
first three columns. Combines with explicit-r; both are taken.--slop N— pad every window by N bp on each side (start clamped
at 0). Equivalent tobedtools slopinline.--stats— Parquet metadata dump (row groups, codecs, per-column
compressed / uncompressed sizes, null counts) without reading any
data. Falls back to--schemafor non-Parquet sources.--unique COL[,COL,...]— distinct value counts per column.
Top-50 values per column with overflow indicator. Honours--filter.--sample N— reservoir-sample N rows uniformly instead of
head-N. Honours--filter(filter first, then sample). Combines
with every view / export mode through a new in-memory
MemoryTableSourceadapter that wraps the sampled Arrow Table and
routes it through the normal rendering pipeline.--schema— print the schema + file metadata footer and exit.
Cheap, useful for inherited files.--describe— per-column statistics: count, nulls, min, max,
mean (numeric), distinct count (string, capped at 16). Respects
--selectand--filter.- Column projection by name —
--select Chromosome,Start,Score.
Unknown names produce a clear error. Works across the ASCII table,
vertical-head, TSV, CSV, JSON, and Parquet output paths. Display
modes still hide format-derived columns (e.g. LociSSD's
MaxEndSoFar); export modes preserve the user's explicit list. --filter <expr>— value-predicate filter:<col> <op> <value>
joined byAND/OR. Ops:== != < <= > >=. Values: integer,
float, single- or double-quoted string. Evaluated per row.- JSON / NDJSON output —
--jsonwrites a JSON array,--ndjson
writes one JSON object per row. Streams chunk-by-chunk. - LociSSD range queries:
-r chr1:78-99 file.lociss(also
--window). Pruning uses the embedded manifest to locate the
chromosome's row range, Parquet column statistics onStartand
MaxEndSoFarto skip row groups outside the window, and a per-row
predicate (Start < end AND End > start) inside surviving row
groups. Open-ended forms (chr1:,chr1:78-,chr1:-99) and
comma-separated multi-windows supported. Plain Parquet without a
LociSSD manifest is rejected with a hint. - TUI: help overlay (
HorF1) — centred panel listing every
keybinding. Any key dismisses it. - TUI: frozen first column (
ztoggle) — pin column 0 at the left
edge while scrolling sideways. Useful for wide tables where the key
column (CHROM,Name, ...) should stay visible. - TUI: mouse-wheel scrolling — scroll three rows per wheel notch.
- LociSSD support — Parquet files carrying a
lociSSD_manifestkey
in their footer (see FORMAT_SPEC.md) are detected
automatically. The derivedMaxEndSoFartechnical column is hidden
from the ASCII table, the vertical-head view, and the interactive TUI;
it remains intact in--tsv,--csv, and--parquetoutput so the
data round-trips losslessly. New virtualhidden_for_display()hook
onTabularSourceprovides the mechanism.