Skip to content

0.4.0

Choose a tag to compare

View all tags
@mcanouil-dev mcanouil-dev released this 23 Jul 16:03
0.4.0
d32f809
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Install

Quick install (script)

curl -fsSL https://m.canouil.dev/oboro/install.sh | bash

# or pin this exact release
curl -fsSL https://m.canouil.dev/oboro/install.sh | bash -s -- --version 0.4.0

The script picks the archive for your machine, verifies it against SHA256SUMS, and installs into /usr/local/bin when writable, otherwise ~/.local/bin.
It needs bash and curl; on a minimal distribution such as Alpine, install them first with apk add bash curl.

Add --features ner for the build that also finds untold names (on Linux it needs glibc 2.39+), then fetch its model:

curl -fsSL https://m.canouil.dev/oboro/install.sh | bash -s -- --version 0.4.0 --features ner
oboro models pull   # about 348 MB, once, verified against pinned hashes

Docker

docker volume create oboro-vault
docker run --rm \
  -v oboro-vault:/vault \
  -v "$PWD":/work -w /work \
  --user "$(id -u):$(id -g)" \
  ghcr.io/mcanouil/oboro:0.4.0 clean contract.docx

The vault volume is not optional.
Without it the mapping between placeholders and real values disappears with the container, and the document can never be restored.

The ghcr.io/mcanouil/oboro:0.4.0-ner tag carries the ner build with the recognition model already inside the image, so untold names are found with no download and no network at run time.

A prebuilt binary

Pick the archive for your machine from the table below, then:

VERSION=0.4.0
TARGET=x86_64-unknown-linux-musl   # or whichever row matches

curl -fsSLO "https://github.com/mcanouil/oboro/releases/download/${VERSION}/oboro-${VERSION}-${TARGET}.tar.gz"
curl -fsSLO "https://github.com/mcanouil/oboro/releases/download/${VERSION}/SHA256SUMS"

# Check it is what was published.
sha256sum --ignore-missing --check SHA256SUMS

tar -xzf "oboro-${VERSION}-${TARGET}.tar.gz"
install -m 0755 oboro /usr/local/bin/oboro

On macOS, shasum -a 256 --ignore-missing --check SHA256SUMS does the same job.

With Rust already installed

cargo install --git https://github.com/mcanouil/oboro --tag 0.4.0

From source, with the optional features

The default binaries read .txt, .md, .docx, .xlsx and text-based .pdf, and find structured values and anything on your denylist.
They do not find names nobody told them about, and they do not read images.

Names need a -ner archive from the table below (then oboro models pull), the :0.4.0-ner image, or a source build.
Images need the Tesseract system libraries, so reading them stays a source build:

cargo build --release --features ner   # names and organisations, then: oboro models pull
cargo build --release --features ocr   # images and scanned pages, needs Tesseract

If names are not being redacted, a default build is almost certainly why.
oboro doctor reports what any build can do.

To build the optional features without setting up the system libraries on your machine, use the devcontainer, which carries the pinned toolchain, Tesseract and the OCR libraries:

docker build -f .devcontainer/Dockerfile -t oboro-dev .devcontainer
docker run --rm -it -v "$PWD":/work -w /work -u vscode oboro-dev \
  cargo build --release --features "ner,ocr"

In Visual Studio Code or a GitHub Codespace, reopen the folder in the container instead.

Verify what you downloaded

Beyond the checksum, every archive carries build provenance, so you can confirm it came from this repository's workflow and not from somewhere else:

gh attestation verify "oboro-0.4.0-x86_64-unknown-linux-musl.tar.gz" \
  --repo mcanouil/oboro

This tool checks the model it downloads against a pinned hash before using it.
It would be inconsistent to ask you to trust its own binaries on sight.

Which archive is which

Archive For
x86_64-unknown-linux-musl Linux on Intel or AMD. Statically linked, so any distribution, glibc version or Alpine.
aarch64-unknown-linux-musl Linux on ARM, including most cloud instances. Statically linked.
aarch64-apple-darwin macOS on Apple silicon.
x86_64-unknown-linux-gnu-ner Linux on Intel or AMD, with name recognition. Needs glibc 2.39+ (Ubuntu 24.04+, Debian 13+).
aarch64-unknown-linux-gnu-ner Linux on ARM, with name recognition. Needs glibc 2.39+.
aarch64-apple-darwin-ner macOS on Apple silicon, with name recognition.

The -ner archives find untold names once the model is fetched with oboro models pull; the others are the smaller default build.

There is no Windows build.
The code that creates the vault key readable only by you is Unix-specific, and shipping a build where that quietly does nothing would misrepresent what the tool guarantees.

Documentation

https://m.canouil.dev/oboro/

Changes

Features

  • feat: Write each workbook sheet to its own TSV file (book.xlsx with a sheet Clients becomes book.Clients.clean.tsv), keeping the tabular structure openable in a spreadsheet tool instead of flattening the workbook into one markdown file; sheet names are sanitised for the filesystem, redacted like filenames when redact_filenames is on, and numbered apart when they collide.
  • feat: Read .csv and .tsv files, passed through as plain text so the cleaned output stays a valid tabular file.
  • feat: Name each output after its input's format, so data.csv becomes data.clean.csv and data.tsv becomes data.clean.tsv while documents keep .clean.md; restore needs no change since it rewrites placeholders in any text file.
  • feat: Refuse any two inputs whose sanitised outputs would land on one file, including sheet outputs, case-folded names, and aliased spellings of one path, before the refused document's values are stored in the vault.
  • feat: Match a denylist term against its exact case with case_sensitive = true, so a short name such as Bell is redacted without also redacting the ordinary word bell; terms still ignore case by default, and no regular expression is needed to make one case-sensitive.
  • feat: Match street addresses in the three word orders languages write them in, so 10 Downing Street, Hauptstraße 5 and 12 Kerkstraat are read alongside 12 rue de la Paix, with no language declared anywhere; postcodes now cover the British, Canadian, Dutch and American formats as well as five-digit ones, while a bare four-digit postcode stays unmatched on purpose since it cannot be told apart from a year.
  • feat: Replace default_region with regions, a list of region codes whose national phone number formats are read, so a document holding numbers from several countries is handled at once; a number valid in any listed region is redacted, an international + number is read whatever the list holds, and an unknown code is now refused by name instead of silently ignored. Without the key the region comes from the environment's locale, and oboro doctor reports which regions are in force and where they came from. A configuration still using default_region is refused, naming the unknown key.
  • feat: Choose the languages Tesseract reads images in from ocr_languages, or from whatever trained data is installed when it is unset, replacing the hard-coded fra+eng that made French trained data a requirement for reading any image at all; asking for a language with no trained data now says so and lists what is installed.
  • feat: Tidy text and markdown input before cleaning it, so trailing spaces, runs of blank lines and blank lines at either end of the file do not survive into the output; indentation is kept, since it carries markdown structure, and .csv and .tsv are passed through byte for byte.
Assets 9
Loading