Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Concat already recorded WebM files #9

Open
sebavan opened this issue May 23, 2019 · 9 comments

Comments

@sebavan
Copy link

commented May 23, 2019

Hello,

It is definitely more a question than an issue :-)

I am wondering if it would be possible to use the same kind of setup to stitch and trim webm files together ?

I was trying to look into ffmpeg alternatives.

Thanks in advance

@beaufortfrancois

This comment has been minimized.

Copy link

commented May 28, 2019

@surma Is that something we could do here?

@sebavan

This comment has been minimized.

Copy link
Author

commented Aug 26, 2019

Thx, the idea was more to do it in the browser from a wasm build of libVPX, as I currently repurposed ts-ebml to do the job :-)

@guest271314

This comment has been minimized.

Copy link

commented Aug 26, 2019

@sebavan That is the ultimate idea with mkvmerge https://discourse.wicg.io/t/port-mkvmerge-to-javascript/3841, see merge() description at https://discourse.wicg.io/t/offlinemediacontext/3814. Currently using Native Messaging at Chromium due to Native File System being implemented to write/read files to storage device as mkvmerge currently requires the files to be on disk https://gitlab.com/mbunkus/mkvtoolnix/issues/2576, https://gitlab.com/mbunkus/mkvtoolnix/issues/2582, https://www.reddit.com/r/mkvtoolnix/comments/cdi824/mkvmerge_prints_error_when_merging_webm_files/. Had the same notion re ts-ebml legokichi/ts-ebml#26.

What found:

  • Tracks must match AV:AV. If A or V do not exist in the original container they must be created to concatenate
  • Width and height must be considered, when encoding VP8 or VP9 if the same width and height are expected to be encoded into the file https://bugs.chromium.org/p/chromium/issues/detail?id=972470, though see last comment at https://bugs.chromium.org/p/webm/issues/detail?id=1642, which have not tried, yet. Without the patches applied have not been able to concatenate or re-encode videos resulting in correct width and height using ffmpeg or vpxenc
  • Native File System can in fact perform the read/write operations, generally consistently, though the first request for permissions usually is not successful, is set to "prompt"
  • It is tedious to determine track order and create an --append-to map
  • At Chromium the only way that have been able to preserve width and height for a live stream is to use OpenH264 and save the file as .webm resulting in a non-standard file https://bugs.chromium.org/p/chromium/issues/detail?id=980822 playable at Chromium/Chrome though which means the file cannot currently be played back at Mozilla Firefox or Nightly https://plnkr.co/edit/Axkb8s?p=preview

And probably some notes that are not recollecting right now.

mkvmerge has the capabilities to merge WebM and Matroska files. That what the code was created for. Not sure if libvpx was in fact created for that purpose.

The only issue with using the linked Native Messaging approach is setup time and not being able to use the code at any URL other than chrome://<extension>. A full port of mkvmerge to JavaScript would require a fair amount of effort emscripten-core/emsdk#260, though should be possible, if the read/write operations are included in the port. An alternative approach that have been cosidering is to watch a directory using bash, write the file which contains the mkvmerge command and execute the program, similar to an IIFE in the form of a shell script WICG/native-file-system#72 (comment).

Am willing to help with the procedure (libvpx) as it appears there is ultimately the same or similar requirement. Would suggest to look into mkvmerge versions 36.0.0 and greater.

@guest271314

This comment has been minimized.

Copy link

commented Aug 31, 2019

@sebavan If you are interested am relatively certain the appendTo algorithm implementation at https://github.com/guest271314/native-messaging-mkvmerge/blob/master/app/native-messaging-mkvmerge.js#L205 through https://github.com/guest271314/native-messaging-mkvmerge/blob/master/app/native-messaging-mkvmerge.js#L228 could be improved and tested to be certain the correct track order is set before executed as an option to the mkvmerge command.

What approach are you currently using to concatenate or merge WebM and Matroska files?

@guest271314

This comment has been minimized.

Copy link

commented Aug 31, 2019

@sebavan

as I currently repurposed ts-ebml to do the job :-)

Can you post a link to the code?

@sebavan

This comment has been minimized.

Copy link
Author

commented Aug 31, 2019

Unfortunately it is not OSS at the moment but basically I simply change the timestamp of each chunk to add the previous concatenated duration.

I hope those kind of simple edition tools will make it into a richer js api.

@guest271314

This comment has been minimized.

Copy link

commented Aug 31, 2019

@sebavan

Had the same concept. Though am not sure how to achieve that goal.

For audio it is possible to use Web Audio API decodeAudioData() and OfflineAudioContext() startRendering() method to merge discrete files into one file.

For encoded images within a video AFAIK there is not an existing method in JavaScript to extract all images from a file without playing the media.

See also https://discourse.wicg.io/t/webcodecs-proposal/3662/15.

@guest271314

This comment has been minimized.

Copy link

commented Sep 9, 2019

@sebavan

Re

For encoded images within a video AFAIK there is not an existing method in JavaScript to extract all images from a file without playing the media.

see https://libwebpjs.appspot.com/vp8/webm-javascript-decoder/.

FWIW Improved the implementation of mkvmerge --append-to option algorithm in JavaScript. Where given a list of media files having arbitrary track order in the container or only a single audio track or video track

https://upload.wikimedia.org/wikipedia/commons/6/6e/Micronesia_National_Anthem.ogg#t=0,2
https://upload.wikimedia.org/wikipedia/commons/a/a4/Xacti-AC8EX-Sample_video-001.ogv#t=0,4
https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm#t=10,20
https://mirrors.creativecommons.org/movingimages/webm/ASharedCulture_480p.webm#t=22,26
https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4#t=55,60
https://raw.githubusercontent.com/w3c/web-platform-tests/master/media-source/mp4/test.mp4#t=0,5
https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/ForBiggerBlazes.mp4#t=0,5
https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/ForBiggerJoyrides.mp4#t=0,5
https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/ForBiggerMeltdowns.mp4#t=0,6

creating the missing adjacent track then piping through MediaRecorder to get WebM files (VP8 codec as VP9 does not preserve original resolution of the video)

[
  {
    "audio": 1,
    "video": 0
  },
  {
    "audio": 0,
    "video": 1
  },
  {
    "audio": 1,
    "video": 0
  },
  {
    "audio": 0,
    "video": 1
  },
  {
    "audio": 0,
    "video": 1
  },
  {
    "audio": 0,
    "video": 1
  },
  {
    "audio": 0,
    "video": 1
  },
  {
    "audio": 0,
    "video": 1
  },
  {
    "audio": 0,
    "video": 1
  }
]

note the order of indexes 1 and 3 are opposite from the remainding indexes this is the corresponding map

1:0:0:1,1:1:0:0,2:1:1:0,2:0:1:1,3:0:2:1,3:1:2:0,4:0:3:0,4:1:3:1,5:0:4:0,5:1:4:1,6:0:5:0,6:1:5:1,7:0:6:0,7:1:6:1,8:0:7:0,8:1:7:1

the resulting file https://github.com/guest271314/native-messaging-mkvmerge/blob/master/native-messaging-mkvmerge-vp8.webm?raw=true, plnkr https://plnkr.co/edit/8J61Rw?p=preview.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.