Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement transparency channel export for able movie formats #1115

Closed
Jose-Moreno opened this Issue Oct 26, 2018 · 7 comments

Comments

Projects
None yet
2 participants
@Jose-Moreno
Copy link
Member

Jose-Moreno commented Oct 26, 2018

Issue Summary

Movie formats that we support like MP4, WebM and APNG can support alpha channels. I've had a few request for this from professionals where they want to export the video with the transparency enabled due to editing software limitations.

This would go in tandem with #1114

scribblemaniac added a commit to scribblemaniac/pencil that referenced this issue Nov 8, 2018

Add support for transparency
Only webm, apng, and gif transparency is supported.
See pencil2d#1115 for details.

scribblemaniac added a commit to scribblemaniac/pencil that referenced this issue Nov 8, 2018

Add support for transparency
Only webm, apng, and gif transparency is supported.
See pencil2d#1115 for details.

scribblemaniac added a commit to scribblemaniac/pencil that referenced this issue Nov 9, 2018

Add support for transparency
Only webm and apng transparency is currently supported.
See pencil2d#1115 for details.

scribblemaniac added a commit to scribblemaniac/pencil that referenced this issue Nov 9, 2018

Add support for transparency
Only webm and apng transparency is currently supported.
See pencil2d#1115 for details.
@scribblemaniac

This comment has been minimized.

Copy link
Member

scribblemaniac commented Nov 12, 2018

For reference, this is very closely related to #1113.

@scribblemaniac

This comment has been minimized.

Copy link
Member

scribblemaniac commented Nov 12, 2018

I have just implemented transparency support for WebM and APNG export in bfd3d74. Obviously support for viewing the transparency will depend on the player you use to play back the exported file, but I have verified that the transparency channel works in Firefox and ffmpeg. In order to get ffmpeg to recognize WebM transparency, you must use the libvpx-vp9 codec before importing like this: ffmpeg -c:v libvpx-vp9 myanimation.webm ...

Two currently supported formats still remain: MP4 and AVI. What's important to understand is that these are both flexible container formats for various codecs. Currently we are using the h.264 codec for mp4 export and the mpeg4 codec for avi, which are both standard pairings from what I've seen. Both h.264 and mpeg4 do not support transparency. Well h.264 technically does support transparency, but nobody actually uses it and as far as I can tell ffmpeg does not even support it. We can create .avi and .mp4 files with transparency if we use a different codec, for example the png codec (different from apng) would work well for this purpose and can inside both avi and mp4 contains. However, the files it produces are much larger, there is no control over the quality of the result (always lossless I think), and it is less widely supported. For instance, Firefox views an mp4 file with a png codec as corrupt and cannot play it. To me that's a deal breakers since we do not want people thinking our program is broken because their players are unable to process our exported videos.

I think the best idea is to continue to use the same codecs for mp4 and avi, and add a new option for FFV1 export. Most people who need transparency are using it for transferring their animation from Pencil2D to some other application (ex. a video editor). The FFV1 codec supports transparency, is one of the best lossless formats, and is well supported, making it ideal for this use case. I would probably put it in the .mkv format which is a common container format for FFV1 and eliminates the confusion that would occur if avi was used instead. It should be very quick to implement so I will just make a PR for it right away and we can discuss it further there.

Until we get some export format suitable for application to application transfer, I will leave this issue open.

@Jose-Moreno

This comment has been minimized.

Copy link
Member Author

Jose-Moreno commented Nov 12, 2018

@scribblemaniac Awesome, thank you so much for looking into this. Regarding MP4 and AVI, that's fine. Normally no-one even attempts to use those formats to get transparency, the most widely used format for that is Quicktime .MOV (Usually with a PNG video codec or the ancient QT "Animation" codec).

For the other formats that do not support transparency people usually export an additional black & white video to serve as a "matte" where a 100% alpha value is recognized as pure black in compositing software. Maybe we could add this option instead to translate alpha values to binary b&w values for matte masking purposes on export, but that's just an idea worth exploring later.

Now that I think about it, probably without realizing those people were really asking for MOV support since it's basically the only one that I know off that can support that, which I think it might not be possible then for us to support due to legal reasons, but I might need to investigate that to be sure.

Note: Opentoonz exports to MOV, but I don't know if they use a third party library or the licensing for the filetype is being supported by their parent company, DWANGO.

@scribblemaniac

This comment has been minimized.

Copy link
Member

scribblemaniac commented Nov 12, 2018

@Jose-Moreno Yes we could add .mov with the png codec, you're right that it does seem to be commonly used like that in the video industry. While FFV1 is technically superior, it is more commonly used for video archiving, so it may be better to use .mov instead. Of course we could add both, but there isn't much benefit to having both of them.

A matte would be easy to implement as well. ffmpeg has an extractalpha filter which does exactly what you want. Alternatively we could do the extraction ourselves, it's a pretty trivial operation. Not sure which would be better from a resource/speed standpoint. What should the UI look like for this though? Should it create a second video file at the same time when a checkbox is clicked? Should it be a separate export matte option?

IANAL but there should not be any licensing issues since all decoding/encoding is done by ffmpeg not us.

@Jose-Moreno

This comment has been minimized.

Copy link
Member Author

Jose-Moreno commented Nov 12, 2018

@Jose-Moreno That'd be great then. I was reading also on the AVI codec and some people pointed that TGA codec was used to achieve the transparency channel, again this coming from a studio perspective as AVI is mostly used with editing consoles (hardware) and TGA is ancient as well mainly used with older but industry grade video cards, so it's kind of like a industry standard, but i digress.

As for the UI on the matte thought, I'd have to think about it and make a draft but I was considering having a "preview" window in the export dialog. If the matte functionality was to be implemented checking it's toggle would enable be a split-screen preview window with the "final" video look on the left, and the b&w matte on the right.

This preview window could be a separate widget though (perhaps invoked by a "preview..." button on the export dialog) as It would be great to also have it for the movie import as well (along a timeline to extract portions of said video to avoid importing the whole thing, but again that is a separate issue altogether)

@Jose-Moreno

This comment has been minimized.

Copy link
Member Author

Jose-Moreno commented Mar 20, 2019

@scribblemaniac given our previous discussions and that the "matte" option for exporting a separate alpha mask should be filed under a different request issue, and that you already implemented the WEBM and APNG transparency option should we close this?

@scribblemaniac

This comment has been minimized.

Copy link
Member

scribblemaniac commented Mar 20, 2019

Until we get some export format suitable for application to application transfer, I will leave this issue open.

Although I said this, it does make more sense to treat this as a separate issue too, so I suppose we can close this.

Issue TLDR; We now support transparency for the .webm and .apng formats. There are no current plans to implement transparency support for .mp4 or .avi file formats due codec limitations. Other features such as .mov and matte export will be explored further in the future.

@Jose-Moreno Jose-Moreno changed the title Implement transparency channel export for movie formats Implement transparency channel export for able movie formats Mar 20, 2019

@Jose-Moreno Jose-Moreno added this to the 0.6.3 milestone Mar 20, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.