Repository navigation
Releases: blast1see/kiyas
Release list
kiyas v0.1.15
Two columns could share one directory, and the pictures overwrote each other
safe_directory_name sanitises a source name into a folder name. Config
rejects duplicate source names, but nothing checked the folders, and four
realistic pairs of distinct names land on one folder:
"Lionsgate GBR/USA" and "Lionsgate GBR_USA" an illegal character
"Remux." and "Remux" a trailing dot
"REMUX" and "remux" case, on any Windows disk
"...Atmos-GROUPONE" and "...Atmos-GROUPTWO" truncation at 100 characters
The last one is the one that bites. Scene names routinely pass 100 characters
and differ only in the group tag at the very end, which is precisely the part
truncation removes.
When it happened the second column's NNNNNN.png overwrote the first's, the
manifest pointed both columns at the same files, and what got published was a
comparison of a release against itself. Nothing errored, and the result looks
exactly like a comparison — the same shape of failure as uploading the grid
transposed, and just as invisible.
The folder names are now decided over the whole set. The first column keeps the
name it asked for; a later collision takes -2, trimmed back into the length
limit rather than pushed past it. The audio side had solved this for column
labels and had the same hole in its directories; it uses the shared
function now.
A file ffprobe could not read came out as a traceback
run promises RunError and every caller — the CLI, the window, align — is
written to that promise. ProbeError is a sibling of RunError, not a
subclass, so pointing kiyas at something that is not a video ended in a Python
stack trace, which reads as a crash in kiyas rather than a fact about the file.
It is wrapped at the module's own boundary now, next to where EngineError
already was, so every caller is covered rather than every caller being patched.
A typo in [tools] deleted an engine instead of reporting itself
find_binary raises for a configured path that is relative or absent, and
available_engines wrapped every engine in a bare except Exception. So
ffmpeg = "C:fmpg�infmpeg.exe" — one missing letter — became "no frame
engine is available. Run 'kiyas doctor'", and doctor, which resolves from PATH,
then reported ffmpeg as ok. The user was sent looking for a problem that was
not there. A bad configured path now comes out as itself.
Dolby Vision profile 5 was tone mapped as HDR10, and the label agreed
media/probe.py has said all along why this is wrong: "a DoVi profile 5 file
carries no usable HDR10 layer at all — tonemapping it as HDR10 produces the
green/purple cast that makes those screenshots useless." The ffmpeg engine
mapped every Dolby Vision profile to the HDR10 chain regardless, and then burnt
"tonemapped hdr10" into the frame, so the picture claimed to be right. The
existing test covered profile 8, which has a real HDR10 base layer and is fine.
The ffmpeg engine now refuses profile 5 and points at the VapourSynth engine,
which reads the Dolby Vision metadata itself.
A refusal about how a frame would look has no business stopping a
measurement, so prepare gained for_measurement. Two callers set it: the mpv
engine, which borrows an ffmpeg source for frame counts and brightness and
draws the pictures itself, and align, which reads brightness and never draws
anything. Without the second, kiyas align and run --check-sync would have
started refusing profile 5 sources they handle perfectly well.
align said how far it had looked
Measured on two clips 137 frames apart: on a 1000-frame source the search
window is 48 frames either way, so 137 was never a candidate. What came back
was a wrong number marked "weak match" — and "weak" on its own reads as "this
material is hard", not "the answer was outside the window". The window is now
named whenever the match is weak, and a block headed "Paste this into the
project file" no longer says that when nothing in it is a confident
measurement.
The ±1% window itself is unchanged and deliberate: past that a difference is a
different edition, and the source-length warning says that better. On the same
clips it did, exactly: "source lengths differ by 137 frames (5.7s)".
Three ways a measurement could be believed when it had not happened
mean_luma returned 0.0 when the ffmpeg call behind it failed, which is a
real reading -- a frame that is genuinely black. So an ffmpeg that fell over
made a good frame look like a fade and skip_dark threw it away. Ranking made
it worse: extremes sorts on that number to find the darkest frame in the
film, so a failed measurement would have won. It returns None now, the same
way combed already did, and neither rule counts a frame it could not read.
The two halves of the B-frame rule disagreed about an unreadable picture type.
The strict half refused it (kind != "B"), and the fallback half -- "avoid
I-frames", used on an encode that has none -- accepted it, although an unknown
frame may well be one.
A Retry-After longer than the retry ceiling was slept on, up to five times.
An hour asked for that way is a run that looks hung for most of a day, and the
ceiling exists to stop exactly that for the local backoff. Clamping it is not
the answer either: retrying sooner than the server asked is how a rate limit
becomes a ban. This module already holds that a refusal is never retried, and a
wait that long is the same answer with a number attached, so it stops and says
what was asked for.
Publishing to pixhost
A third destination, --to pixhost, and a fourth markup format to go with it.
pixhost is an image host rather than a comparison host, and that is the point
of it being here. slow.pics and comp.pics store a grid with a viewer that flips
between sources at one frame; pixhost stores pictures and hands back a page and
a thumbnail for each. That is what a forum post wants and what neither of the
other two gives:
[b]0:21:14.083 / 30550[/b]
UHD REMUX: [url=.../show/...][img]https://t3.pixhost.to/thumbs/...[/img][/url]
--format thumbnails is the one format that does not inline the full picture,
which is what makes a post of two dozen 4K screenshots readable. It is not a
pixhost format: any backend that fills in thumbnail_urls and page_urls can
use it.
Details that shaped it:
show_urlis an HTML page, not a picture, verified live — it answers
text/html— and the direct address of the full-size file is not documented.
Soimage_urlsstays empty for this host rather than being filled with
something that would be a broken picture inside[img].- 10 MB per image, and a 4K PNG is close to it: 7.3 MB measured on a UHD
HDR10 frame. Every file is checked before the first request, because the
alternative is a half-filled gallery. Ten million bytes, not ten mebibytes:
the documentation does not say which it means, and a file in the
half-megabyte gap is exactly the one the check exists to stop. - A failed upload still hands back the gallery's management link. By then
some images are up, on a host with no expiry, and that token appears once and
cannot be recovered. optimize_for_webis never sent. Off is its default and the only setting
under which a lossless PNG survives being hosted.- No API key, so nothing to store and nothing to configure.
The window has a destination selector now, instead of a button that said
"Publish to slow.pics" and a confirmation that promised the comparison would be
unlisted whatever the destination — true of exactly one of the three.
Publishing is tested against the real hosts
The live marker had been registered since the beginning and nothing carried
it. tests/test_publish_live.py uploads a synthetic two-column set to all
three hosts, reads every returned address back, and checks that the picture at
the far end is the one that was meant to be there — brightness, not bytes,
because the hosts resize. That is the check a fake session cannot make, and the
transposed grid is the failure it exists for.
live is now deselected in addopts, which it needed to be: registering a
marker labels tests, it does not stop them running, so pytest -q — the
command the notes describe as "unit tests, no media needed" — published to all
three hosts. Three publishes to slow.pics in quick succession is how the rate
limiting in 0.1.1 was earned in the first place. pytest -m live opts in and
replaces the expression; CI's own filter is unaffected.
The pacing, backoff and timeout machinery had been written twice, byte for
byte. It lives in publish/transport.py now. The numbers that are actually
per-host stayed where they were measured.
kiyas v0.1.14
No more console windows — this time all of them
0.1.13 stopped mpv opening a console and said, on the strength of a
measurement, that ffmpeg did not need the same treatment. That was wrong, and
wrong in a way worth writing down: a console window belongs to conhost.exe,
not to the process that owns the console. Enumerating ffmpeg's own windows
found none, so the conclusion was that there were none.
Measured properly — watching every 20 ms for console windows that were not
there before — one three-frame comparison from a windowed process put up 176
console windows. With the flag: none. At the twelve frames a real
comparison uses, that is closer to seven hundred.
Every subprocess in kiyas now passes CREATE_NO_WINDOW, and a test walks the
source to keep it that way, because this failure is invisible from inside: the
run succeeds, the pictures are right, and the only symptom is on the screen.
The single exception is kiyas setup, which inherits the console on purpose so
that pip's progress is visible. It is unreachable from a windowed build, which
refuses to run setup at all.
kiyas v0.1.13
mpv no longer opens a console window per column
Running a settings comparison from kiyas-gui.exe put up a black console
window for every variant it rendered.
PATHEXT places .com ahead of .exe, so on Windows mpv resolves to
mpv.com — a console wrapper, shipped so that the GUI build has somewhere to
write. Launched from a process that has no console of its own, it makes one.
Measured from a real windowed process rather than reasoned about, because the
answer is not the same for every child: launched with all three standard
handles redirected, ffmpeg gets no window with or without CREATE_NO_WINDOW.
mpv.com owned a visible console without the flag and none with it. So the
flag went where it was measured to matter, and the note next to it says the
others were checked and do not need it.
kiyas v0.1.12
The size warning now says which crop to use
Two columns of different shapes cannot be flipped between, and 0.1.9 started
saying so. It stopped short of the useful part: it told you the active picture
was in the Dolby Vision RPU's level 5 offsets and left you to go and extract an
RPU yourself.
It now reads them. When the sources are not the same size and one of them
carries Dolby Vision, the warning carries the line to paste —
crop = [0, 0, 276, 276] — and the size that leaves.
It costs about a second. dovi_tool extract-rpu --limit stops after a handful
of frames and closes the pipe, so a 78 GB remux is sampled in 0.2 seconds per
position and never read through.
Three things it refuses to do:
- Guess. If dovi_tool is missing, or the RPU will not parse, the warning
is the one 0.1.9 printed. Enriching a message is not worth failing a run that
has already written its images. - Answer from one place in the film. A title with an IMAX sequence changes
shape as it plays, and a reading taken inside either stretch looks perfectly
constant. Five positions spread across the film are read; if they disagree
the warning says the picture changes shape and suggests no crop at all. - Make the numbers match. Measured on the pair this was built against: the
disc's own metadata says 276 rows top and bottom, leaving 1608, while the
online WEB-DL of the same film is 1606. The warning prints the crop the
metadata asks for and then says it is still two rows taller than the other
release. Moving those two rows to the bottom edge would make the sizes agree
by inventing a framing nobody chose.
Sources that already carry a crop or a resize are skipped: the offsets
describe the frame as encoded, so against a transformed source they answer a
different question.
kiyas v0.1.11
The VapourSynth engine could not run from the window at all
Opening any source with it died on 'NoneType' object has no attribute 'flush' before a single frame was read. A windowed build has no console, so
Python leaves sys.stderr unset — it is None, not a closed file — and the
indexing capture flushed it unguarded.
The irony is that the very next line already handled this case: redirecting
file descriptor 2 is wrapped in a try whose comment names pythonw. Only the
flush above it was written as though stderr were always an object. So the
console build worked, the tests worked, and the engine had never once worked
from kiyas-gui.exe.
A stderr that raises on use is now tolerated too, for the capture modes that
hand one back.
kiyas v0.1.10
The window stops offering an engine it cannot run
The engine dropdown listed all four whatever the machine had. In a packaged
build, which has no VapourSynth, picking it was allowed and the answer came
back as a RunError — after the comparison had been set up, and with nothing
in the window to suggest it would fail.
Missing engines are still listed, greyed, with the reason in the name:
vapoursynth (not installed here). Hiding them would be worse — then the
engine that does what you want is simply absent and nothing says why.
auto stays selectable whatever is missing, and a probe that throws leaves
every entry enabled rather than emptying the list.
kiyas v0.1.9
Columns that are not the same size say so
Two pictures of different shapes cannot be flipped between, and flipping
between them is the one thing a comparison is for. Nothing noticed: every image
was written, the manifest was valid, the upload would have succeeded. Measured
on a real pair — a 3840x1606 WEB-DL against a 3840x2160 remux, run without a
crop, and the run had nothing to say about it.
It does not suggest the numbers. The obvious arithmetic is wrong: splitting the
difference gives 277 rows top and bottom where that remux's own Dolby Vision
metadata says 276, with the last row coming from the WEB-DL's conformance
window. A suggestion one row out is worse than none, because it looks like an
answer. The warning points at the RPU's level 5 offsets, which is where the
real number is, and the README shows how to read them.
kiyas v0.1.8
The packaged build stops giving instructions it cannot follow
kiyas audio in a packaged build answered "Run 'pip install kiyas[audio]'",
and doctor printed the same line beside numpy, scipy and matplotlib. A frozen
build has no pip and no interpreter to point one at, so that is an instruction
it cannot follow — the rule kiyas setup has followed since it existed, in a
different table.
Both now say what is actually possible: use a checkout.
The libraries are not bundled instead, and the numbers are why: they come to
180 MB against a 58 MB package, and being the small download is the whole
argument for the packaged build. The README now says audio needs a checkout
rather than leaving it to be discovered.
kiyas v0.1.7
The window puts screenshots where you would look for them
Typing out in the output box resolved against the process's working
directory, which a window has no way of showing. Launched from a file dialog
that had last visited another drive, a comparison of two files wrote its
screenshots to E:\out — correct by the rule, and not where the person who
asked for it would look.
The audio side of the same window already resolved a relative output against
the first source. Both modes now agree about what out means. Absolute paths
are untouched, and the command line is unchanged: there the working directory
is something you typed.
kiyas v0.1.6
The ffmpeg engine says what it left out
0.1.5 taught kiyas to compose a Dolby Vision profile 7 enhancement layer, and
the whole point was to stop producing base-layer screenshots and offering them
as screenshots of the release. The ffmpeg engine kept doing exactly that — it
cannot compose the layer, and it said nothing.
That is the engine the packaged build uses, since VapourSynth is deliberately
not frozen. So the one place the old behaviour still lived was the build most
people run. Found by comparing two profile 7 remuxes with the released
kiyas-gui.exe and noticing the run had nothing to say about it.
It now reports the layer the way the VapourSynth engine does, and refuses
dovi_el = "on" outright rather than ignoring a request it cannot honour —
an explicit ask for a specific picture is the worst place to quietly produce a
different one.