[Bug] jpeg to gif conversion issue

[Toyem]

Prerequisites

• Did you read FAQ section in readme.md?
• Did you test with the latest releases or commit ?
• Did you search for existing issues in Issues?

Description

I wanted to use the option to convert ugoira to gif.
It turns out that this feature works quite well except for some ugoira from users 4734113.
We notice that the gif obtained truncates a large number of images and keeps only a few images at the end.

Steps to Reproduce

To do this, change the option createGif to True in the category [Ugoira] in the config.ini file.
Then launch PixivUtils.py by selecting option 2. Download by image_id and enter one of the following two examples (sorry for the not family firendly character of the chosen examples, it's the only ones that don't work):

• 91281700
• 96990356

Small investigation

When we display the logs in debug mode of the command passed in the function convert_ugoira of PixivHelper.py by the line subprocess.Popen, we realize that a great number of automatic conversion of pixel_format are realized and that at certain moments, EOI tags seem to be missing.

[mjpeg @ 0x7ff293c6bd00] EOI missing

[auto_scale_1 @ 0x7fc817d18300] w:300 h:300 fmt:gray sar:1/1 -> w:300 h:300 fmt:bgra sar:1/1 flags:0x0
[auto_scale_1 @ 0x7fc81980c380] w:300 h:300 fmt:yuvj444p sar:1/1 -> w:300 h:300 fmt:bgra sar:1/1 flags:0x0

After inspection, the issue does not come from the images (jpeg) which have all their SOI and EOI tags (respectively "\xff\xd8" and "\xff\xd9"). I put the mini script I used in footnote if you want to check the info.

However, the problem seems to come from the automatic conversion of the images which cuts the generation of the gif and erases the information previously generated.
To realize this, you just have to delete the images generating an automatic conversion (images 21, 22, 23, 106 and 107 for the artwork id 91281700) from the file i.ffconcat to prevent their use during the conversion and the gif is generated from the image 00000.jpeg without issues.

I tried to prevent the conversion of the pix_fmt of the images by adding (for example) -pix_fmt +yuvj444p but the command crash because it does not manage to continue the operation without converting the image.

The filters 'Parsed_split_0' and 'Parsed_palettegen_1' do not have a common format and automatic conversion is disabled.
Error reinitializing filters!
Failed to inject frame into filter network: Invalid argument

I suspect that the problem is that these images are black and white while the others are colored but I can't seem to solve the problem.
If someone experienced enough in ffmpeg could give his expertise on this problem, it would be welcome.

Versions

PixivDownloader2 version 20220311-beta3

Annex

Code to show the hex of an image :

import sys

def main(args):
    with open(args[1], "rb") as fp:
        img = fp.read()
    print(img)

if __name__ == "__main__":
    main(sys.argv)

[biggestsonicfan]

Interesting. The originally downloaded ugoria files shouldn't be affected, correct? Just the conversions? If so, perhaps an option to re-encode all existing ugoria files might also be a good option to add to this fix!

[Toyem]

Not gonna lie, i can't really open ugoira files on my mac. But i can see that it's around the same size as the zip archive of the images.
I also realised that Webm and Webp are also affected by this bug but the other way around. It's the beginning of the file that is fine but it quickly stop and become kinda glitchy ...
I'm trying to find a workaround by converting every greyscale images into an rgb one.
It look like the conversion of greyscale to rgb flux in ffmpeg isn't well supported

[biggestsonicfan]

Not gonna lie, i can't really open ugoira files on my mac. But i can see that it's around the same size as the zip archive of the images.

Yeah, the ugoira files are basically the same as the zip files but with added animation data.

PixivUtil2/PixivImage.py

Line 581 in e7c380b

def create_ugoira(self, filename) -> bool: