Cross platform support

[elliotwoods]

Hey!

Very happy with HAP results on a recent installation.
I've been looking into Density * BC1 and BC4 for similar performance without the particular caveat of Hap...
Quicktime outside of OSX is a pile of crap :)

So HAP is nice, the technologies inside (DXT / Snappy) are cross-platform and easy to build/find tools for. But the .mov wrapper is a huge problem in the wider ecosystem. Do you have any thoughts about HAP outside of OSX?

Currently HAP doesn't support:

• Linux
• openFrameworks on Windows (unless you count legacy versions)
• Windows x64 (as far as i know the DirectShow playback doesn't support GPU decompression stage)

Ideally all we'd need is a cross-platform free Quicktime demuxer to get to the frame data then the rest is easy, e.g. ffmpeg/libav probably could help here, have you tried this?

Elliot

[sheridanis]

There is a directshow wrapper for hap on OSX that render heads have made that works in 64bits. It's annoying that it won't play mov's though and movies you encode with it aren't cross platform though so +1 for a universal container format and encoding/decoding framework

Sent from my iPhone

On 8 Jul 2016, at 03:27, Elliot Woods notifications@github.com wrote:

Hey!

Very happy with HAP results on a recent installation.
I've been looking into Density * BC1 and BC4 for similar performance without the particular caveat of Hap...
Quicktime outside of OSX is a pile of crap :)

So HAP is nice, the technologies inside (DXT / Snappy) are cross-platform and easy to build/find tools for. But the .mov wrapper is a huge problem in the wider ecosystem. Do you have any thoughts about HAP outside of OSX?

Currently HAP doesn't support:

Linux
openFrameworks on Windows (unless you count legacy versions)
Windows x64 (as far as i know the DirectShow playback doesn't support GPU decompression stage)
Ideally all we'd need is a cross-platform free Quicktime demuxer to get to the frame data then the rest is easy, e.g. ffmpeg/libav probably could help here, have you tried this?

Elliot

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[bangnoise]

Hi Elliot!

Hap is a frame format and does not specify any particular container (or platform).

Existing tools include the QuickTime codec (OS X and Windows), the DirectShow codec (Windows), and transcoding support in ffmpeg (OS X, Windows, Linux). Various proprietary products also support Hap encoding and/or playback.

As well as the QuickTime codec which packs Hap into .mov, the DirectShow codec packs into .avi (and can be used for GPU playback if software requests DXT pixel formats from it) and ffmpeg supports Hap in .mov as well as a range of other containers.

The .mov container has the advantage of being very common and widely integrated into existing workflows, which is why implementations thus far have tended to favour it. Using .mov does not require Apple's (deprecated) QuickTime libraries.

Using libavformat from ffmpeg is a common strategy for Hap playback. Currently you must acquire encoded frames from libavformat and perform the Hap decode yourself to get the frame in compressed texture formats, though the hope is to eventually have support in libavcodec to deliver the unpacked compressed texture formats when software requests them.

1. Perhaps all this could be better documented for developers to avoid giving the impression that there is a dependency on QuickTime?
2. Are there libraries or developer tools you think are missing or would make using Hap simpler?
3. Are there user tools (encoders, transcoders, etc) you think would make using Hap simpler?

[elliotwoods]

Hi Tom

Thanks for the detailed response.
Although HAP technically is agnostic of the container, it might be good to have official cross platform support for the most popular container (i.e. MOV).

ideally there would be prebuilt library which itself links libavcodec but isolates it from the users project, and provides functions for accessing frames, e.g.:

HapFile * hapOpenFile(const char * filename);
HapStatus hapCloseFile(HapFile *);
HapStatus hapGetFileInfo(HapFile *, HapFileInfo * out); // number of frames, frame rate, any metadata
HapStatus hapGetFrame(HapFile *, size_t frameIndex, HapFrame * out); // width, height, data (DXT), size, format. Thread safe so can get multiple frames simultaneously
HapStatus hapReleaseFrame(HapFrame *);

I'm typing this on my phone so not very fine tuned (bit of a mix of C and C++ there). But that kind of API would be amazing

[elliotwoods]

Any thoughts on that type of implementation?

[ItayGal2]

I'm using hap on windows with in an AVI container works well and it works
with touch designer.
AFAIK the ffmpeg group have no intention on adding hap support. So I've
written my own c# (almost pure c#) version of hap. using the Managed
Squish, SnappyPI and c++/CLI wrapper i wrote for DirectXTex.

I hope i'll find the time to share the code on github and the next few
weeks..

if anyone is interested, you can contact me.

On Thu, Jul 28, 2016 at 12:36 PM, Elliot Woods notifications@github.com
wrote:

Any thoughts on that type of implementation?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#16 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AJifNDQrsS33KRdWPKGA69I3WwFu36sCks5qaHg1gaJpZM4JHq9D
.

[bangnoise]

Sure @elliotwoods it would be useful, however I think generally people are interested in supporting other codecs as well, so using eg libavformat fits in to existing workflows better than having an entirely distinct code path for Hap.

There probably should be sample code for optimal Hap playback with libavformat - that sample code would probably provide roughly the API you describe.

Any sample code for any platform is very welcome, and we can link to it from the README here.

[heisters]

@ItayGal2 FFMPEG supports HAP:

https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/hap.h

I think the command ends up something like: ffmpeg -i test.mp4 -vcodec hap -format hap_q out.mov

@bangnoise regarding libavcodec: I've taken a stab at integration with libavcodec on a couple occasions, and it's always turned into a nightmare. Might be I was going at it the wrong way. I think, however, that a custom player could also open the possibility for some features that would be helpful to Hap users. Opened another discussion on that: #17.

Regarding workflow: I think the Hap user base is unusual enough that a player with only Hap support could be usable. Right now I'm using Davinci Resolve to output DNxHR, and transcoding that with FFMPEG to MJPEG because I've been having difficulty getting Hap working (and I'm on a time crunch at the moment). I've never had a project where I wanted performance enough to justify Hap, but couldn't also ensure that all content got transcoded to Hap. Just saying universal codec/container support maybe isn't so important.

[elliotwoods]

To note
This works well for me
ffmpeg -framerate 60 -f image2 -i %07d.png -vcodec hap -chunks 16 hap.mov

A common use of HAP is a mono-codec world (eg the project spec requires a codec with performance X so the whole tool set will be designed with that in mind). I understand that's not the universal case but it's definitely the one for me :)! (eg also I presume every Windows software which implements GPU accelerated HAP has some kind of dedicated pipeline for HAP video, so it would be amazing to have an officially provided solution for this).

As mentioned by @heisters
Linking / including libavcodec or ffmpeg against existing projects is insurmountable for many developers (I avoid it at all costs, and when I do need it, it can take more than 1 day to setup correctly, especially when using features which aren't always in the most available builds of libavcodec, eg HAP).

The way I'd implement it, is to isolate libavcodec from the user entirely, eg libhap only includes libavcodec in cpp's so not passed through to user's h files through includes, distribute a prebuilt lib / header for 'libhap' and dynamic libraries for libavcodec, and ideally consider that the user might want the option to use libavcodec directly also (but that most of the cases they wouldn't want to even know it existed).

[heisters]

This gets into the conversation I moved over to #17, but, @elliotwoods, what about only supporting a single container format so that you don't even need libavcodec? OGG is unencumbered, and probably an easier dependency than libavcodec, or one could even write a simple container parser from scratch--say MPEG (except that has patent issues). I think it is important that it be a container that FFMPEG could write to, otherwise some dirt-simple format like an MJPEG stream or a tar file might even make sense for these purposes.

[elliotwoods]

OGG sounds good, but it's a bit late in the game to switch from MOV, so I'd suggest discussing with all the companies who've already invested time in supporting HAP and ask them what they think (eg VJ software and media servers)
The main advantage of MOV is that you can compress direct from many applications (when on OSX), right?

[heisters]

Yeah, I guess I wasn't considering OGG as a replacement for MOV, but just an alternative to support a slimmed-down HAP-specific player library. MOV is certainly preferable for its ubiquity. Another alternative might be extracting just the MOV parsing from libavcodec. I tried that a while back, and it seemed like a pretty involved job. Writing a custom MOV parser is a bit of work, but is probably doable, especially if all it needs to do is rip frames out and pass them to the HAP decoder.

[bangnoise]

This all seems a lot of effort to avoid using ffmpeg, which is not as bad as all that. I'm going to use ffmpeg as the basis of a project soon and will put out some demo code at the same time.

If any of you are insane enough to write your own demuxer and decoder for Hap, I'd recommend sticking with .mov for interaction with other toolsets - and a lot of long nights with that enjoyable tome The QuickTime File Format await you.

You would probably be well advised to start out writing a codec-agnostic demuxer (which will be the bulk of your work, and a lot of work), and then combine that library and a simple Hap decoder. Seriously though, save yourself the tears.

[elliotwoods]

An example with ffmpeg would be amazing
A library which does what I mentioned above (random access hap frames, isolate ffmpeg from user's code with an airlock) would be perfect
Examples for that library performing multi-threaded pre-cached playback to OpenGL and DirectX would be 'job done'

( presuming out of the box cross-platform)

[heisters]

Agreed, if libavformat is straightforward it's way better than my janky and crazy approach ;)

I need to start work on this component sometime in the coming weeks. Even if your work is not ready as a sample, if you're able to push it to GitHub, I'd love to fork it and work on it myself. If you're on a slower timeline, I'll take a stab at this approach and throw it up somewhere in case it's useful to you.

[heisters]

@bangnoise I took a quick stab at this, but am unsure of the best way to include FFmpeg as a dependency. Attempting to include and compile the code directly within my project is problematic because, from what I can tell, FFmpeg expects a GNU C99 compiler (I'm trying to compile with LLVM C++11), and moreover it requires a lot of platform-specific configuration. It might be the best approach to use a system-installed FFmpeg, given that between homebrew, apt, and chocolatey, there are decent compilation scripts for Mac, Linux, and Windows. But that hardly hides the dependency from the user. What did you have in mind?

[bangnoise]

Building and packaging any library for a specific platform is well beyond the scope of this issue. FFMPEG is reasonably well documented as far as building it goes. I suggest you carefully follow its documentation to build it, and then use the usual platform-appropriate methods to link and (where necessary) bundle it with your software.

[heisters]

I threw together a quick MP4/MOV Hap + MJPEG player here: https://github.com/heisters/libglvideo

Should compile on Windows, but I haven't tried it yet. Will get that working tomorrow.