x264 lossless

[vadosnaprimer]

Please add an option for this. The only lossless options seem to be utvideo (ffv1 was tested by fsvgm777 and he said it's lossy somehow), but utvideo is very annoying to get to work, both for replay and encoding. While x264 is very broadly supported, and it has a lossless option.

I couldn't find how exactly libTAS tells ffmpeg how to encode, but here's the command that produces lossless h264 avi with bizhawk ffmpeg:
-c:a pcm_s16le -c:v libx264rgb -qp 0 -preset ultrafast -pix_fmt rgb24

And of course ideally, in the future, libTAS could support sending the direct command to ffmpeg like bizhawk does.

[clementgallet]

I added two fields in the Encoding Options to set codec-specific options (77ee8c1). The format is: key1=value1:key2=value2:etc. Also, you should remove the - from the key name, so your example becomes (for the relevant part): qp=0:preset=ultrafast. You can verify on the console the codec settings that ffmpeg is printing when the encode is starting. You'll also see if an option wasn't recognized.

I didn't check how Bizhawk is using ffmpeg, I'm using the ffmpeg API (avcodec/avformat/avutil) and feeding the raw images/waveforms.

[vadosnaprimer]

BTW, Mari0 dump reports 600fps, while the game actually runs at 60, and the video just has 10 dup frames for every unique frame (Mediainfo reports 600, but media players advance frames like it's 60). I suspected that it overshoots the dumping procedure due to some odd behavior of the game engine. Could you check this?

Another thing we noticed, it uses subsampling (to 420) for ffv1, which makes it lossy. Does it look preventable? From looking at the docs, maybe Qt sends -level 3 (whch means ffv1 version 3 instead of 1) and mistakenly uses wrong colorspace? We need ffv1 version 1 here.

[clementgallet]

About mari0, I correctly get a 60 fps (ish) video as reported by mediainfo. What are your settings ?

Indeed, ffvi seems to switch to yuv440. What I'm currently doing is checking if the pixel format of the stored images (usually rgba) is compatible with the codec, and if not, I let ffmpeg choose the codec preferred pixel format. I could override this to choose rgb specifically for ffvi codec.

[vadosnaprimer]

Can you check what VirtualDub says? Here are the dumps fsvgm777 got: https://yadi.sk/d/nnShqYmX3a92eV

[fsvgm777]

Hi, the dumps I made were with software rendering disabled. I believe I didn't touch any setting at all (and well, I can't do a whole lot in the encoding configuration dialog box, since there's only a field for encode file path as well as drop-down menus for video and audio codec (+ bitrate if it's a lossy codec).

EDIT: Forgot to mention the game is mari0, a game made on the Löve framework.

EDIT 2: The movie framerate is set at 60 FPS.

[clementgallet]

From what I am reading, the muxer can override the video codec specified framerate to choose a "better one", so this is why I usually end up with a 1000 fps video, maybe this is also what is happening to you. However, I'm taking this into account during the encode so the timestamp for the video frames should be correct. I'll look again into the ffmpeg muxer if I can override this override, but really this API is a mess.

EDIT: To be more precise, I'm setting AVCodecContext.time_base to 1/60, and at some point AVStream.time_base is set to 1/1000.

[clementgallet]

I found the right parameter, which actually fills the framerate in the file header (9f9e008). Could you test again with this change ?

[fsvgm777]

Just tested, and it still reports a frame rate of 600 FPS. I also noticed a regression: It dumps a lot slower than before.

I also tested x264 RGB lossless, but alas, it doesn't take any arguments from ffmpeg, just the codec-specific options, so it defaults to yuv420p. It did correctly process the other codec options, though.

[vadosnaprimer]

For AVI file, framerate is set by writing to 2 fields of AVIFILEINFOW: dwRate (numerator) and dwScale (denominator).

[clementgallet]

Now I choose BGR0 pixel format by default, for codecs that accept it (af6f0e8). Now FFV1 dumps in RGB colorspace.

It dumps slower by just changing a field in the header ? OK. You are using Arch right ? I could set it up on a VM to see if I can reproduce it.

libx264rgb seems broken to me also. It chooses nv21 pixel format (basically yuv420) even if it is supposed to only accept bgr24, bgr0 and rgb24 formats.

EDIT: Also, I'm testing with LOVE 11.1 and the latest mari0 commit

[fsvgm777]

Nope, I'm using a Kubuntu 18.04 VM (with everything updated).

What I did notice is that in the terminal output, it states the framerate is 0.02 FPS (when in reality, the resulting dump is at 600 FPS):
Output #0, avi, to '/home/fsvgm777/libTAS/build/loveffv1.avi':
Stream #0:0: Video: ffv1, bgr0, 800x448, q=2-31, 4000 kb/s, 0.02 fps
Stream #0:1: Audio: pcm_s16le, 44100 Hz, stereo, s16, 1411 kb/s

This is the only thing related to the recent change. It dumped about as fast as the game ran before the change.

[clementgallet]

Oh, you are using AVI container. Indeed, I'm getting the same problems as you with this container.

[vadosnaprimer]

In theory, it's possible to process mp4 and mkv dumps, but avi is just our odious legacy container that works the best with avisynth (which is also an odious legacy thing, and we hope to move away from it).

[clementgallet]

I'm going to drop the current solution and feed data to a ffmpeg process instead, like Bizhawk is doing. This should solve most of the problems by translating most of the work to ffmpeg (or least make it easier to debug), and also would not require users to use a specific version of the librairies as ffmpeg API is constantly changing.

[vadosnaprimer]

Oh wow! Thank you for this decision! ffmpeg supports basically endless variety of workflows, and we already now several that work for our "odious legacy" needs. And just as easily, it supports all the modern stuff like direct vp9+webm output, And indeed this removes the unhelpful wrapper layer where we have to really please Qt first if we want something to be told over to ffmpeg. Which, as we've learned yesterday, doesn't feel like hearing us out in the first place. Best of luck with reimplementation, we have a lot of test cases for this!

[clementgallet]

Almost done with the rewriting. I just have one problem: the file is not written on disk, despite ffmpeg telling that it is written. I think this is because ffmpeg is called from a forked process, and the fork process is also getting the libTAS preload with all the function hooks, which probably is messing with ffmpeg file writing.

[vadosnaprimer]

We need @nattthebear here!

[nattthebear]

I agree with your assessment, it sounds like you want to avoid the consequences of fork in this situation, but I don't know what you'd use to do it in linux; is there something like posix_spawn that also lets you redirect stdioerr?

[clementgallet]

I managed to unset LD_PRELOAD, which surprisingly worked (881c48d). Now encoding to raw video/audio seems to work. I have trouble with vorbis codec which puts audio out of place despite pts looking ok.

[vadosnaprimer]

For the reference, here's what dolphin is doing. Dunno if it's applicable here tho:
https://github.com/dolphin-emu/dolphin/blob/master/Source/Core/VideoCommon/AVIDump.cpp#L298

[clementgallet]

Thanks! That was pretty much what I was doing before. Now I'm putting packet into a Nut container using code from Bizhawk, so there is no more ffmpeg API code. I'm quite happy with this solution actually, because I know I won't have to dig into the code every 6 months when ffmpeg people deprecate, remove functions and add new ones all the time.

EDIT: I tested the settings from your first post, and it works correctly afaik.