feat: waifu2x upscaling, as a resample method in Image Size - #72
Merged
Conversation
Super Zoom was only ever half an upscale -- a filter cannot resize its own buffer, so the network could put high frequencies back but could not make the image bigger. This adds the other half in the place that can: Image Size grows a Resample dropdown with waifu2x x2 (Photo) and waifu2x x2 (Art) beside bicubic, bilinear and nearest. The models are nagadomi's upconv_7, MIT and shipped in the binary at 2.2 MB each. tools/train/waifu2x.py converts them rather than training anything: it fetches a pinned revision of the Torch JSON weight dumps, checks them against a known hash, rebuilds the seven-layer graph and exports ONNX, which tract then runs unmodified. The one deliberate change to the graph is padding. upconv_7 is all valid convolutions and returns 2N-28 pixels for N in, so seven pixels of replicate padding go in front of the first layer and a tile comes back exactly doubled -- which is the contract run_scaled tiles against. Replicate for the same reason the tile driver mirrors at image edges: a network fed a black border draws one. Against held-out kodim23 it is +4.8 dB on bilinear. Input::Tiles gains a scale, and run_scaled sits beside run_tiled: the same overlapping-tile walk with mirrored image edges, every write-back coordinate multiplied by the scale, starting from a nearest-neighbour enlargement so a tile that fails degrades to soft pixels rather than a black square. Layers are resampled premultiplied, as everything here is, with alpha doubled bilinearly alongside -- a coverage ramp has no detail for a network to invent. It costs about nine seconds per input megapixel on sixteen cores, so the resize is split: deciding needs the document and is cheap, running carries the pixels instead and goes to the background executor. That wanted a "working" modal, and the one Photoshop plug-ins already use was right except for its wording, so FilterRunning is now Busy with the title and text passed in. Non-power-of-two targets double until they reach the target and let bicubic cover the remainder; downscales never involve the network; and if a model will not load, bicubic stands in and the status bar says that it did, rather than quietly substituting the thing the user did not pick. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The new resize method went in immediately above it, which put it between that function's doc comment and the function itself -- so the #[allow(clippy::too_many_arguments)] guarding its eight arguments silently moved onto the new five-argument one, and clippy failed on the function it had been guarding all along. Moved the new method above the whole doc block. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Super Zoom was only ever half an upscale — a filter cannot resize its own buffer, so the network could put the high frequencies back but could not make the image bigger. This adds the other half in the place that can: Image Size grows a Resample dropdown with waifu2x ×2 (Photo) and waifu2x ×2 (Art) beside bicubic, bilinear and nearest.
The models
nagadomi's
upconv_7, MIT, shipped in the binary at 2.2 MB each.tools/train/waifu2x.pyconverts rather than trains: it fetches a pinned revision of the Torch JSON weight dumps, checks them against a known hash, rebuilds the seven-layer graph and exports ONNX, which tract runs unmodified.The one deliberate change to the graph is padding.
upconv_7is all valid convolutions and returns 2N−28 pixels for N in, so seven pixels of replicate padding go in front of the first layer and a tile comes back exactly doubled — the contractrun_scaledtiles against. Replicate for the same reason the tile driver mirrors at image edges: a network fed a black border draws one.Against held-out
kodim23: bilinear 23.49 dB → waifu2x 28.31 dB (+4.82).Inference
Input::Tilesgains ascale, andrun_scaledsits besiderun_tiled— the same overlapping-tile walk with mirrored image edges, every write-back coordinate multiplied by the scale, starting from a nearest-neighbour enlargement so a tile that fails degrades to soft pixels rather than a black square. Layers resample premultiplied, as everything here does, with alpha doubled bilinearly alongside: a coverage ramp has no detail for a network to invent.Why the resize is split in two
It costs ~9 s per input megapixel on 16 cores, so a 12 MP photo would have frozen the window for two minutes. Deciding needs the document and is cheap; running carries the pixels instead and goes to the background executor. That wanted a "working" modal, and the one Photoshop plug-ins already use was right except for its wording — so
FilterRunningis nowBusy, with the title and text passed in.Non-power-of-two targets double until they reach the target and let bicubic cover the remainder; downscales never involve the network; and if a model will not load, bicubic stands in and the status bar says it did, rather than quietly substituting the thing the user did not pick.
Checks
807 workspace tests pass. New coverage: the upscaler beats interpolation on a held-out photograph, odd and tiny sizes survive, tile coverage holds at scale, and the document-level resize doubles, reaches a non-power-of-two target, downscales classically and undoes as one edit.
Driven in the real app headlessly — dropdown, busy modal mid-run on a 1100×800 → 2200×1600, and one undoable "Image Size" entry.
🤖 Generated with Claude Code