PeakML 0.4.0

The training-data release. PeakML could clean a corpus and could not hand
one back. It now takes a folder of PDFs and Word files to a train/eval split in
the formats a fine-tuning run reads — and the Python it exports rebuilds the
same files, byte for byte.

New

• Export as training data. Newline-delimited JSON in four shapes: plain
  text, prompt and completion, chat messages, and preference pairs. Pick the
  shape, map your columns, and see the first row as the JSON it will become. A
  split column writes one file per split.
• Open JSON and JSONL. .json, .jsonl and .ndjson. Nested values stay
  as text rather than being flattened by guesswork, and when a file could be
  read two ways the app tells you which way it read it.
• Split into train and eval — without leaking. Chunk a document and split
  the chunks at random, and pieces of the same document end up on both sides,
  which quietly flatters every score you measure afterwards. PeakML assigns the
  split by source document instead, so a document lands wholly on one side, and
  the same recipe gives the same split every time.
• Chunk by tokens, counted with the GPT-4 vocabulary, which ships inside the
  app — nothing is downloaded.
• Remove near-duplicate text. The same page with a different footer, the
  same article with a different date line. Every pair it reports is checked
  exactly, and it shows you the least-similar pairs it grouped so you can judge
  them.
• Filter low-quality text with eleven named rules, counted one by one.
• Find or redact personal data — emails, phone numbers, card numbers,
  Aadhaar and PAN. It says on every run what it does not detect.
• Detect the writing system of each row, and catch Indic text that extracted
  wrong. Real government PDFs in Hindi and Telugu often come out of a PDF with
  their conjuncts broken or their syllables split, while still reading as the
  right script — so it goes into training data looking fine. PeakML now says so
  when it loads.
• A column's profile as training data: token lengths, near-duplicate share,
  script mix, and how much of it carries personal data.
• Notes on a step, to record why it is there. They travel into the exported
  recipe and the generated Python.
• A real sample on the start screen — 301 records from databricks-dolly-15k,
  under CC BY-SA 3.0.

Fixed

• The exported pipeline.py could not open a JSON, JSONL, TSV or Excel file.
• Casting a column to text in the exported pipeline.py changed its type
  without converting the values, so any text step after it behaved differently
  from the app.
• ISBNs and DOIs were redacted as if they were card numbers.
• Free text was treated as a category, with advice that only makes sense for
  one.

Notes

Windows x64, unsigned. SmartScreen will warn on first run: More info →
Run anyway. macOS is still not published — it needs an Apple certificate.

PeakML 0.3.0

The window has been rearranged around your data instead of around a ribbon.

No model in the product — the judgement it offers is deterministic profiling.

Downloads

Platform	File
Windows — x64	PeakML-0.3.0-win-x64.exe

Checksums are in SHA256SUMS.txt.

Requires Windows 10 or later, 64-bit. The file is large because PeakML ships
its own Python runtime — pandas, numpy, scipy, scikit-learn and pyarrow are
inside the app, so nothing needs installing alongside it.

macOS is not in this release. PeakML is not code-signed for macOS, and an
unsigned build is blocked on arrival rather than merely warned about. The
macOS builds return with a Developer ID certificate. 0.2.1 remains below for
anyone already running it.

New in 0.3.0

A new shell. The five-tab ribbon and the permanently-docked panels are
gone. There is now an activity bar down the left, one sidebar you switch
between Recipe, Columns, Find, Datasets and Problems, and a collapsible panel
along the bottom. The grid keeps the middle, always.

The count is the point: the old window drew a five-tab ribbon plus a left
panel plus a right panel plus the grid — four surfaces before a row of
data. It now opens with one sidebar, one tab and a collapsed panel. Fewer
things on screen, and more of them reachable.

Type a command instead of hunting for it. ⇧⌘P opens a palette over every
command in the app, including all 46 transforms. Searching for "fill" finds
Fill missing values whether or not you know which menu it lives under.

Type a column name to go to it. ⌘P jumps to any column. On a file with a
hundred and fifty columns this replaces a great deal of horizontal scrolling.

Column details, the flow view and provenance are tabs. They each used to
cover the entire window, so opening one hid whatever you were comparing it
against and closing it was the only way back. They now sit beside the grid.

Problems are in one place. Everything the app noticed — parse failures,
columns that are entirely empty, values that cannot be real — is collected in a
Problems panel, with the one-click fix attached where there is one.

The status bar acts. Every fact on it is a control: the fitting set opens
Datasets, the target column selects it in the grid, the warning count opens
Problems.

Also

Typography and density were reworked throughout: three weights instead of five,
tighter rows, and information carried by alignment rather than by badges.
Numbers sit right, text sits left, and missing values read as a quiet dash.

PeakML 0.2.3

A speed release. Working on a large file was slow in a way that had nothing to
do with the size of the file.

No network calls. No telemetry. No model in the product — the judgement it
offers is deterministic profiling.

Downloads

Platform	File
Windows — x64	PeakML-0.2.3-win-x64.exe

Checksums are in SHA256SUMS.txt.

Requires Windows 10 or later, 64-bit. The file is large because PeakML ships
its own Python runtime — pandas, numpy, scipy, scikit-learn and pyarrow are
inside the app, so nothing needs installing alongside it.

macOS is not in this release. PeakML is not code-signed for macOS, and an
unsigned build is blocked on arrival rather than merely warned about. The
macOS builds return with a Developer ID certificate. 0.2.1 remains below for
anyone already running it.

Fixed since 0.2.2

Everything you did to a large file re-ran the whole recipe.

PeakML keeps the result of each step so that changing one thing does not
recompute the others. On a large file that memory ran out — and when it did, it
threw away the wrong result. A recipe is replayed from the beginning, so the
oldest saved step is the one needed first; that was the one discarded. The
next click then discarded the one after it, and so on. The cache stayed almost
full and helped with nothing.

It now keeps a share of your machine's memory rather than a fixed amount, and
when it does run out it discards the step furthest along the recipe — the one
needed last, and the cheapest to redo.

Measured on a 246 MB, three-million-row file:

scrolling to row 2,000,000    1.059s  ->  0.002s
looking at a column           1.242s  ->  0.183s
previewing a fill             1.198s  ->  0.182s
previewing a scale            1.183s  ->  0.131s
reopening a step              1.210s  ->  0.153s

Peak memory fell from 2247 MB to 1880 MB at the same time.

Larger files gain more than smaller ones, because the wasted work grew with the
file and this does not. Nothing about what PeakML computes has changed — the
same recipe produces the same numbers, and the test that checks a generated
pipeline against the app value for value still passes unchanged.

PeakML 0.2.2

A correctness and speed release for Windows. If 0.2.1 would not open for you,
this is the fix.

No network calls. No telemetry. No model in the product — the judgement it
offers is deterministic profiling.

Downloads

Platform	File
Windows — x64	PeakML-0.2.2-win-x64.exe

Checksums are in SHA256SUMS.txt.

Requires Windows 10 or later, 64-bit. The file is large because PeakML ships
its own Python runtime — pandas, numpy, scipy, scikit-learn and pyarrow are
inside the app, so nothing needs installing alongside it.

macOS is not in this release. PeakML is not code-signed for macOS, and an
unsigned build is blocked on arrival rather than merely warned about. The
macOS builds return with a Developer ID certificate. 0.2.1 remains available
below for anyone already running it.

Fixed since 0.2.1

The app stopped at its splash screen and never opened. It asked Windows for
the translucent "mica" window material on every version of Windows. That
material only exists on Windows 11 22H2 and later; on anything earlier the
window had nothing to draw and never finished opening — and every timeout meant
to catch a slow start was waiting on that same signal, so nothing rescued it.
No window, no error, nothing to click.

The material is now only requested where it exists, and the splash hands over
after twenty seconds no matter what, so a launch cannot stall like this again.

Everything was slow. The app shipped without its Python bytecode cache, and
was configured so the cache could never be built — which meant that every time
the engine started it re-read and re-compiled the whole of pandas, numpy, scipy
and scikit-learn from source. That is 2,887 files, each one something for
Windows Defender to inspect, on every single start.

The cache is now built into the app. Engine startup takes roughly a third of
the processor time it did, and opening files and running steps is quicker
throughout.

The window's own close button sat underneath Windows'. On the column details
page — and the flow and history views — the header's controls were drawn under
the system minimise, maximise and close buttons. Both were drawn, Windows' on
top, and neither could be clicked reliably. All three views now keep clear of
that strip, measured from the system rather than assumed, so it stays correct
with the snap-layout chevron and at higher display scaling.

PeakML 0.2.1

A correctness release. Five defects, three of which produced quietly wrong
numbers rather than an error — so if you have used PeakML for anything you
intend to keep, this is worth taking.

No network calls. No telemetry. No model in the product — the judgement it
offers is deterministic profiling.

Downloads

Platform	File
macOS — Apple Silicon (M1 and later)	PeakML-0.2.1-mac-arm64.dmg
macOS — Intel	PeakML-0.2.1-mac-x64.dmg
Windows — x64	PeakML-0.2.1-win-x64.exe

Checksums are in SHA256SUMS.txt.

Requires macOS 11 Big Sur or later, or Windows 10 or later, 64-bit. The files
are large because PeakML ships its own Python runtime — pandas, numpy, scipy,
scikit-learn and pyarrow are inside the app, so nothing needs installing
alongside it.

Fixed since 0.2.0

A separate fitting set learned from the wrong data. If you used
File ▸ Set fitting set…, every fitted step — scalers, encoders, imputers —
learned from that file as it sat on disk rather than as your recipe had
transformed it. A scaler at the end of a recipe would take its mean and
standard deviation from a column an earlier step had already cleaned. In one
measured case the learned spread came out eight times too large, and a
frequency encoder built a table missing the value an earlier step had just
filled in, so 40% of the rows fell through to a fallback of zero.

Nothing errored, and the wrong numbers were internally consistent, which is
why this was invisible. If you have a saved recipe that uses a fitting set,
re-run it on this version and compare.

Formulas silently mismatched text containing ^, [ or ]. Comparing a
column against a value like "Guy (-ish) ^_^" or "[draft]" never matched,
because the expression was rewritten before it was read and the rewrite reached
inside quoted text. No error appeared — rows simply landed in the wrong bucket.

Exported Python is no longer dead on arrival. Export recipe as Python
could only generate arithmetic formulas. Any formula using if, isnull,
coalesce, lower, trim or an aggregate — which is most of the reason to
write one while cleaning data — produced a file that stopped at that line. All
31 formula functions now export.

The relationships view works on a yes/no target. Designating a target and
encoding it to 0/1 turned the view off entirely, which ruled it out for exactly
the modelling task it is most useful for. One-hot columns were excluded for the
same reason.

0/1 columns are no longer called identifiers. The readiness table treated
every flag and every one-hot output as "not a measurement" and stopped
assessing it.

New in 0.2.1

A flag for values that cannot be real. A column of ages containing
99999999999 used to raise nothing at all — the median read 31 and the mean
read 79 million, with an empty cautions panel. PeakML now says so, and offers
Treat as missing as a one-click fix.

It asks whether a value sits anywhere near the rest of the column rather than
matching a list of known codes, so a one-off typo is caught. It stays quiet on
genuinely long-tailed data — income, wealth, waiting times — where a large
maximum is real.

Column details also says when a histogram is a single solid bar, which is a
picture of one outlier rather than of a distribution.

Also

A ten-lesson preprocessing course now ships with the source, covering profiling
through to a trained model, with every figure reproduced by a runnable script.

PeakML 0.2.0

The column detail release — a full page for any column, and the Apple Silicon
build fixed.

No network calls. No telemetry. No model in the product — the judgement it
offers is deterministic profiling.

Downloads

Platform	File
macOS — Apple Silicon (M1 and later)	PeakML-0.2.0-mac-arm64.dmg
macOS — Intel	PeakML-0.2.0-mac-x64.dmg
Windows — x64	PeakML-0.2.0-win-x64.exe

Requires macOS 11 Big Sur or later, or Windows 10 or later, 64-bit. The files
are large because PeakML ships its own Python runtime — pandas, numpy, scipy,
scikit-learn and pyarrow are inside the app, so nothing needs installing
alongside it.

Fixed since 0.1.0

The Apple Silicon build launches. 0.1.0's native macOS build reported
itself as damaged and could not be opened at all, while the Intel build ran
fine on the same machines. The bundle was not unsigned — it carried the
signature Electron's own binary is compiled with, which packaging invalidates.
Apple Silicon refuses an executable whose signature does not verify; Intel
under Rosetta does not, which is exactly why one worked and the other did not.
The bundle is now signed after packaging and the signature verified as part of
the build.

If you are on Apple Silicon and 0.1.0 would not open for you, this is the fix.

New in 0.2.0

Column details — a full page for any column, from the inspector, the
column's right-click menu, or ⇧⌘D. It shows the distribution twice, once over
the typical range and once over every value, so a missing-value code sitting on
its own is visible rather than flattening the plot. Then what each of the three
outlier rules would call extreme and how much they disagree, the extreme rows
themselves, the quantile ladder, and where the gaps fall in the row order —
three percent missing scattered through a file and three percent missing in a
block at the end are the same number everywhere else, and they mean opposite
things.

Text columns get lengths, blanks and padded values. Date columns get a
distribution at all, having previously had only a first and last date.

Prepare from the page — configure a step at the foot of the page and apply
it there, with the charts above showing the column as it is and as the step
would leave it. Scaling a column while a -999 is still in it compresses every
real value into a band two hundredths wide, and now you watch that happen
instead of finding out later.

Relationships — every column ranked against the one you are on, with the
measure chosen by the pair of types and all of them on 0 to 1 so one list can be
read top down. Click a row for the joint view.

Readiness — every column against what the recipe has done to it, and what is
still outstanding. This is the answer to I scaled one column, what about the
other six.

Loading skeletons — the canvas draws the shape of the grid while a file is
read, and the status bar names what it is waiting on.

A new icon and splash, built from the PeakML mark.

Installing

These builds are not code-signed yet, so both systems warn on first launch.
The warning is about the absence of a paid certificate, not about anything
found in the app.

• macOS 15 Sequoia and later (including Tahoe) — open the .dmg, drag
  PeakML to Applications, double-click it once and click Done on the
  warning, then go to System Settings ▸ Privacy & Security, scroll to
  Security, and click Open Anyway. The button only appears after macOS
  has blocked the app once. The old right-click → Open shortcut was removed
  in Sequoia and does not work.
• macOS 14 Sonoma and earlier — right-click PeakML and choose Open.
• Any macOS, from the terminal — xattr -dr com.apple.quarantine /Applications/PeakML.app
• Windows — run the .exe; at SmartScreen click More info →
  Run anyway.

Verifying

SHA256SUMS.txt is attached. Check with shasum -a 256 -c SHA256SUMS.txt, or
on Windows Get-FileHash .\PeakML-0.2.0-win-x64.exe -Algorithm SHA256.

PeakML 0.1.0

First public release of PeakML — a local-first desktop app for preparing data.

No network calls. No telemetry. No model in the product — the judgement it
offers is deterministic profiling.

Downloads

Platform	File
macOS — Apple Silicon (M1 and later)	PeakML-0.1.0-mac-arm64.dmg
macOS — Intel	PeakML-0.1.0-mac-x64.dmg
Windows — x64	PeakML-0.1.0-win-x64.exe

Requires macOS 11 Big Sur or later, or Windows 10 or later, 64-bit. The files
are large because PeakML ships its own Python runtime — pandas, numpy, scipy,
scikit-learn and pyarrow are inside the app, so nothing needs installing
alongside it.

Installing

These builds are not code-signed yet, so both systems warn on first launch.
The warning is about the absence of a paid certificate, not about anything
found in the app.

• macOS 15 Sequoia and later (including Tahoe) — open the .dmg, drag
  PeakML to Applications, double-click it once and click Done on the
  warning, then go to System Settings ▸ Privacy & Security, scroll to
  Security, and click Open Anyway. The button only appears after macOS
  has blocked the app once. The old right-click → Open shortcut was removed
  in Sequoia and does not work.
• macOS 14 Sonoma and earlier — right-click PeakML and choose Open.
• Any macOS, from the terminal — xattr -dr com.apple.quarantine /Applications/PeakML.app
• Windows — run the .exe; at SmartScreen click More info →
  Run anyway.

Full instructions, including what to do if macOS says the app is damaged, are
in the README.

What it does

Every mutation becomes a recorded step — typed cells, pastes, fills, row and
column deletes, sorts, filters, replaces, and all 46 statistical transforms land
in the Applied Steps panel. The grid is always recomputed from the original file
through the ordered steps, so nothing changes invisibly and everything is
reversible.

• Columns are typed and homogeneous; an edit that does not fit is refused.
• Fitted transforms are leakage-safe — scalers and encoders learn from the
  fitting set and reapply, and the status bar says which data that was.
• The recipe is a document with an id, a version, a history and a diff, and can
  be pointed at data it was never written on.
• Recipes export as runnable Python.

Verifying

SHA256SUMS.txt is attached. Check with shasum -a 256 -c SHA256SUMS.txt, or
on Windows Get-FileHash .\PeakML-0.1.0-win-x64.exe -Algorithm SHA256.