PTify Model V3

PTify model v3 — more pianos, fewer invented notes

Model v2 trained further on piano music played through sampled pianos (two
uprights and three grands), each render with randomized tone. v2 learned from
one instrument, MAESTRO's Yamaha grand, and on any other piano it invented
notes. v3 invents far fewer. On MAPS, real recordings of a piano and room it
never trained on, it gains +1.50 onset F1, better on 14 of 14 tracks.
Everything v2 was good at stays within 0.11 points.

set (onset F1)	v2	v3		precision v2 → v3	tracks better / worse
MAPS (14 real paired tracks, unseen piano)	0.8437	0.8587	+1.50	0.866 → 0.890	14 / 0
upright, held out (sampled Yamaha, never trained on)	0.9074	0.9348	+2.74	0.898 → 0.952	11 / 1
upright, trained on (sampled Knight)	0.8343	0.9202	+8.59	0.803 → 0.956	12 / 0
SMD (50 real Disklavier recordings)	0.9553	0.9556	+0.03	0.991 → 0.993	27 / 22
clean MAESTRO	0.9562	0.9552	−0.11	0.993 → 0.991	3 / 9
phone	0.9392	0.9397	+0.05		6 / 6
YouTube-style (Opus 133 kbps)	0.9567	0.9560	−0.07		4 / 8
TikTok-style	0.9401	0.9400	−0.01		5 / 7
far phone	0.8788	0.8793	+0.05		6 / 6

The gain is precision: recall barely moves (MAPS 0.823 → 0.830). v3 hears
the same notes, but stops adding notes that were never played. On MAPS it
outputs 28,101 notes against 30,356 real ones, where v2 output 28,384.

This release passed the rule set before training: MAPS up at least 1.0,
and no other set down more than 0.2. (v2 did not pass its rule, and shipped on
the owner's call.)

Caveats

• The two upright rows are rendered, by PTify's own sample player
  (evaluation/sampled_piano.py), not recorded. The Knight row measures a
  piano v3 trained on. The Yamaha row is a fairer test, since that piano was
  held out of training, but it went through the same renderer. MAPS is the
  number that counts: real recordings, on a real piano v3 never heard.
• MAPS answer keys have known errors (unplayed notes, slow attacks), so part
  of the remaining MAPS gap is the answer key, not the model.
• Clean MAESTRO fell 0.11, with 9 of 12 tracks slightly worse. That is within
  the 0.2 allowance, but it is a real small cost on MAESTRO's own grand.

What is in the file

ptify-p43-step8000.pth — 172,046,209 bytes

sha256  eb417ccefd7db8fa32db47f12f228c98761d6ad357ae7d58a3ef5ee2ca1a69d0

Same format as v1 and v2: a deployable checkpoint
{'model': {'note_model': ..., 'pedal_model': ...}}, loadable directly by
piano_transcription_inference. The note weights are trained; the pedal
weights are still ByteDance's, frozen and unmodified.

Verify the digest before using it. The inference library checks a
checkpoint by size alone, so any other ~172 MB .pth loads without complaint.

Use it

python -m transcriber --fetch-ptify          # downloads and verifies v3
python -m transcriber song.wav --engine ptify
python -m transcriber --doctor               # reports present / absent / wrong

Or set PTIFY_CHECKPOINT=/path/to/file.pth, or put the file in checkpoints/.
The architecture is the same as v1, v2 and ByteDance's, so speed, sustain pedal
and velocity output are unchanged. Onset threshold unchanged (0.6). Frame
threshold raised 0.01 → 0.05. It controls only where notes end, never which
notes are found, so every onset F1 above holds at either value.

frame threshold (v3, mean note-end F1)	MAPS (12, no pedal data)	MAESTRO 3-min (10, pedalled)
0.01 (v2's)	0.470	0.214
0.05	0.451	0.409

MAPS records no sustain pedal, while MAESTRO and real uploads are pedalled.
The sheet-music benchmark decided it: at 0.05, rhythm went 73.0 → 75.0% and
note lengths 56.2 → 57.3%, with 17 of 38 pieces at the right beat level
instead of 15 (measured with v2 through the live service). Sweep:
benchmarks/threshold-calibration-p43-maps_paired.json.

Rolling back to v2: point PTIFY_SHIPPED_* in transcriber/ptify.py back
at the PTIFY_P39_* constants, and the host pin in hosting/modal/app.py at
the v2 file.

Training

Two runs back to back. v2 → p41 (Phase 41) → v3 (Phase 43):

	p41 (intermediate, not released)	v3
start	v2 (ptify-p39-step8000.pth)	p41
steps	8,000	8,000 (16,000 piano-varied steps in total)
sampled pianos	Knight upright, Kawai grand, Steinway B grand	the same three plus Salamander grand
share of segments re-rendered on a sampled piano	35%	50%
tone variation per render	none	decay 0–1.5/s, unison detune 0 or 1–8 cents, brightness −0.6…0.8
held out	Yamaha upright	Yamaha upright

Both runs used MAESTRO v3.0.0 train-split MIDI as the notes; sampled
segments are rendered on the fly in the data loader. v2's upload augmentation
(room, mic EQ, loudness compression, Opus/MP3) stayed on, effective batch 8
(fp32, batch 4 × accum 2), lr 5e-5 with warmup, velocity loss weight 0.1. Each
run took about 7–8 h on one free-tier Kaggle T4 (v3: 0.31 steps/s).

p41 alone gained +0.54 on MAPS, short of the +1.0 bar, so it was not
released. The second run added a fourth piano and per-render tone variation
in place of more instruments, because no clearly licensed recordings of
further uprights were found.

Run log: var/transcriber-run-p43/checkpoints/train_log.jsonl (not committed).

How it was scored

Both models ran in one Kaggle job (ptify-score v4, tools/kaggle_score.py)
on identical 16 kHz audio, with the same upload presets and the same onset
threshold (0.6), so the weights are the only difference. v2's numbers
reproduced the previous scoring run exactly. MAESTRO rows use 3-minute
excerpts of the 12 test tracks. Results: var/score-run-p43/.

On the live service, the hosted model and a local run agree note for note:
285 of 285 notes on the 25-second check clip, onset F1 1.000000
(tools/crosscheck_remote.py).

Onset F1 is the number this release stands behind. The table above was
scored at v2's frame threshold (0.01). At that threshold, note-end F1 moved
slightly down on MAPS (0.499 → 0.478), which is why v3 ships with the
re-calibrated value.

What v3 does not fix

• Missed quiet notes inside chords are still the largest error type.
  v3's gain is fewer extra notes, not fewer missed ones.
• Note lengths and rhythm on the sheet music. Those come from the rhythm
  model after transcription, and v3 does not change them.
• Pianos still differ. Rendered uprights now score 0.92–0.93 against 0.956
  on MAESTRO's own grand, and real MAPS 0.859.

Licence

CC BY-NC-SA 4.0: research and non-commercial use, same as v1 and v2.

The weights derive from ByteDance's checkpoint (Apache 2.0) and were fine-tuned
on MAESTRO v3.0.0 (CC BY-NC-SA 4.0). The share-alike term is taken to carry
over to the trained weights. That is the conservative reading, not a settled
legal one.

Training audio was also rendered from these piano sample libraries, none of
which is redistributed here:

• VCSL (Versilian Community Sample Library) uprights Knight and Yamaha, and
  grands Kawai and Steinway B, CC0.
• Salamander Grand Piano v3 by Alexander Holm, CC BY 3.0,
  https://github.com/sfzinstruments/SalamanderGrandPiano

No MAESTRO, MAPS or SMD audio is redistributed.

What's Changed

• p39 model by @ImSe4n in #29
• prod beta by @ImSe4n in #30
• resolve vercel prod issues by @ImSe4n in #31
• Phase 42 production beta by @ImSe4n in #32
• p42 by @ImSe4n in #33
• threshold settings by @ImSe4n in #34

Full Changelog: model-v2...model-v3

PTify Model V2

PTify model v2 — upload-robust piano transcription

Model v1 trained further on audio that sounds like real uploads: phone-mic
tone, loudness compression, and Opus/MP3 compression. It is better than v1 on
every simulated upload condition, by up to +1.85 onset F1 on a far-away
phone, and it no longer invents as many notes there.

condition (onset F1)	v1	v2		tracks better / worse
far phone	0.8603	0.8788	+1.85	12 / 0
TikTok-style	0.9301	0.9401	+1.00	12 / 0
phone	0.9314	0.9392	+0.78	11 / 1
YouTube-style (Opus 133 kbps)	0.9562	0.9567	+0.05	6 / 4
clean MAESTRO	0.9552	0.9562	+0.11	9 / 3
SMD (50 real Disklavier recordings)	0.9538	0.9553	+0.15	33 / 16
MAPS (14 paired tracks)	0.8503	0.8437	−0.66	3 / 11

Most of the gain is precision: on far-phone audio it rose 0.947 → 0.974, so
far fewer notes are made up.

The upload conditions are simulated. They are real MAESTRO test recordings
run through a room, mic tone, loudness compression and a lossy codec at a
measured bitrate (evaluation/upload_chain.py). No public dataset pairs real
phone or YouTube audio with exact notes, so this is the closest available test.

Known regression: MAPS, and why it is mostly timing

MAPS fell 0.66 points, more than the 0.2-point allowance set before training.
By that rule v2 should not have shipped; the owner chose to ship it because
it wins every upload condition, which is what users actually send.

Most of the MAPS drop is timing, not wrong notes. MAPS's far-mic recordings
come out about 24 ms late for both models, because the sound takes time to
cross the room. v2 places onsets about 1 ms later than v1, which pushes a few
more notes past the 50 ms scoring window. Note counts barely change. With each
model's own delay removed, MAPS is 0.8924 (v1) vs 0.8906 (v2): −0.18.
That correction uses the answer key, so it is a diagnosis, not a score. A
constant delay does not change sheet music.

What is in the file

ptify-p39-step8000.pth — 172,046,209 bytes

sha256  6687be39bdb05a55ee1c4bee907035a8f3252806032a7209adaf9903f3983f77

Same format as v1: a deployable checkpoint
{'model': {'note_model': ..., 'pedal_model': ...}}, loadable directly by
piano_transcription_inference. The note weights are trained; the pedal
weights were frozen and are ByteDance's, unmodified.

Verify the digest before using it. The inference library validates a
checkpoint by size alone, so a different ~172 MB .pth loads without complaint.

Use it

python -m transcriber --fetch-ptify          # downloads and verifies v2
python -m transcriber song.wav --engine ptify
python -m transcriber --doctor               # reports present / absent / wrong

Or set PTIFY_CHECKPOINT=/path/to/file.pth, or drop it in checkpoints/.
Same architecture as v1 and ByteDance, so the same speed, sustain pedal and
velocity output. Decoding thresholds are unchanged (onset 0.6, frame 0.01).
A lower onset threshold helped SMD but hurt MAPS, so it was not adopted.

Rolling back to v1: point PTIFY_SHIPPED_* in transcriber/ptify.py back
at the PTIFY_16B_* constants, and the host pin in hosting/modal/app.py at
the v1 file.

Training

start	v1 (ptify-16b-step6555.pth)
data	MAESTRO v3.0.0 train split
steps	8,000 at effective batch 8 (fp32, batch 4 × accum 2), ~8 h
lr	5e-5, 200 warmup steps
loss	velocity weighted 0.1 (other heads 1.0)
augmentation	v1's room/detune (20% clean passthrough) plus mic EQ (70% of segments), loudness compression (70%, threshold −34…−16 dB, ratio 2…8), Opus/MP3 round-trip (50%, compression level 0.3…0.85)
hardware	one free-tier Kaggle T4, 0.28 steps/s

This changed one thing against Phase 23's recipe, which had made v1 slightly
worse: the acoustic chain. Mic tone plus compression is the part that bit
hardest (−1.7 points on v1 per segment where both fired); the codec alone
barely mattered.

Run log: var/transcriber-run/checkpoints/train_log.jsonl (not committed).

How it was scored

Both models ran in one Kaggle job on identical 16 kHz audio, with the upload
presets applied on the same machine and the same onset threshold (0.6), so the
only difference is the weights (tools/kaggle_score.py). Results are in
benchmarks/wild/p39/. The MAESTRO rows use 3-minute excerpts of the 12 test
tracks (tools/cut_excerpts.py), not whole tracks, so they are not directly
comparable to v1's published full-track 0.9633.

Onset F1 is the number this release stands behind. Offsets and velocity were
not re-evaluated.

What v2 does not fix

• Missed quiet notes inside chords are still the largest error type, about
  45% of errors even on clean audio (tools/error_taxonomy.py).
• Instruments other than MAESTRO's grand. The same music played on sampled
  uprights scores 0.83–0.91 against 0.956 on the real recordings, mostly from
  invented notes. Training on sampled pianos is in progress (Phase 41).

Licence

CC BY-NC-SA 4.0 — research and non-commercial use, same as v1.

The weights derive from ByteDance's checkpoint (Apache 2.0) and were fine-tuned
on MAESTRO v3.0.0 (CC BY-NC-SA 4.0). The share-alike term is taken to propagate
to the trained weights, which is the conservative reading, not a settled legal
one. No MAESTRO, MAPS or SMD audio is redistributed.

What's Changed

• Phase 18 measurement honesty by @ImSe4n in #14
• Phase 19 decode calibration by @ImSe4n in #15
• Phase 21 notation benchmark by @ImSe4n in #16
• SqliteJobStore behind the JobStore interface + tests by @ImSe4n in #17
• local HS256 JWT issuer/verifier, users, get_principal seam by @ImSe4n in #18
• ARQ end-to-end with shared store, storage durability by @ImSe4n in #19
• Phase 6 frontend by @ImSe4n in #20
• Phase 7 playback and motion by @ImSe4n in #22
• Phase 9 gpu host by @ImSe4n in #23
• Phase 22 precision by @ImSe4n in #24
• Phase 24 voice separation by @ImSe4n in #25
• Phase 26 readable score by @ImSe4n in #26
• Phase 27 recall diagnosis by @ImSe4n in #28

Full Changelog: model-v1...model-v2

PTify Model V1

PTify model v1 — room-robust piano transcription

ByteDance's high-resolution piano CRNN, fine-tuned with continuous room and
detune augmentation. It beats the stock model by +5.3 onset F1 on MAPS, a
corpus neither model trained on.

	ByteDance	PTify v1
MAPS (14 paired tracks)	0.7866	0.8395	+5.3 — improved 14 of 14
MAESTRO (12 tracks)	0.9693	0.9633	−0.6 (the bounded price)

MAPS is the honest target. MAESTRO is ByteDance's own training distribution, so
its 0.969 there is a home-field score — beating it would not mean what it
appears to mean.

Why this is room robustness, not a general uplift

The MAPS paired subset is the same performances captured at two microphone
distances, so everything except the room is held constant:

mic distance	ByteDance	PTify v1	delta
ENSTDkCl close, ~50cm	0.851	0.878	+2.7
ENSTDkAm ambient, 3–4m	0.722	0.801	+7.9
room penalty	0.129	0.077	−0.052

The hard, reverberant condition gained 2.9x what the easy one did. A model
that had merely got generally better would have lifted both equally. The
generalisation gap — 18.3 points between MAESTRO and MAPS — is now 12.4, so
roughly a third of it is closed.

What is in the file

ptify-16b-step6555.pth — 172,037,521 bytes

sha256  17286ad93c5806e02a59caf0333769d9bea9f4f3e53abd7360be8cabe9d4accd

A deployable checkpoint: {'model': {'note_model': ..., 'pedal_model': ...}},
loadable directly by piano_transcription_inference. The note weights are
fine-tuned; the pedal weights are ByteDance's, unmodified, because MAPS
carries no pedal ground truth and a change there could not be scored either way.

Verify the digest before using it. The inference library validates a
checkpoint by size alone, so a different ~172MB .pth loads without complaint
and scores a model you cannot identify.

Use it

python -m transcriber --fetch-ptify          # downloads and verifies
python -m transcriber song.wav --engine ptify
python -m transcriber --doctor               # reports present / absent / wrong

Or point at the file directly with PTIFY_CHECKPOINT=/path/to/file.pth, or
drop it in checkpoints/. The engine raises when the weights are missing
rather than falling back to the pretrained model — a silent fallback would
report the baseline's score under this model's name.

Same architecture as ByteDance, so identical speed (~1.1x real time on CPU),
sustain pedal, and real velocity.

Training

base	ByteDance CRNN_note_F1=0.9677_pedal_F1=0.9186
data	MAESTRO v3.0.0 train split, 962 tracks
steps	6,555 at effective batch 8 (fp32, batch 2 × accum 4)
lr	5e-5
augmentation	rt60 log-uniform 0.2–1.6, detune triangular ±50 cents, 20% clean passthrough
hardware	one free-tier Kaggle T4, a single ~10-hour session

That is 15% of one epoch. The augmentation ranges came from a measured
degradation curve: a quarter-semitone detune cost 14.1 F1 points and a concert
hall 9.3, while 15dB-SNR noise cost 1.5 — so the augmentation targets pitch and
reverb, and does not bother with noise injection.

Run log: benchmarks/training/16b-step6555.jsonl.

Known issue — do not quote the offset metric

MAESTRO +offset rose 0.381 → 0.520 while MAPS +offset fell 0.607 →
0.431. Nothing in the training targeted offsets, and two corpora disagreeing in
direction on the same metric means something systematic is unaccounted for.
Onset F1 is the number this release stands behind. The offset movement is
under investigation and should not be cited as a result.

Reproducing the numbers

python -m evaluation.maps --out recordings/maps_disklavier
python -m evaluation --audio-dir recordings/maps_paired \
    --engine ptify --preset clean \
    --json my-run.json

Then diff against the committed baseline. Scores are not bit-identical across
machines — thread count and device change floating-point reduction order — so
record inference_threads and device alongside any number you publish.

Licence

CC BY-NC-SA 4.0 — research and non-commercial use.

The weights derive from ByteDance's checkpoint (Apache 2.0) and were fine-tuned
on MAESTRO v3.0.0, which is CC BY-NC-SA 4.0. The share-alike term is taken to
propagate to a model trained on that data, so this is the conservative reading
rather than a settled legal one. No MAESTRO or MAPS audio is redistributed here
or in the repository.