Improve audio merging from the backend service

[drewcook]

Enhancement

User Story

Currently audio merging is handled in a flattening service over https, this requires the user to upload the individual file blobs both to the flattening service and to IPFS during the metadata creation.

In order to remove the need for multiple uploads, we should handle the audio file muxing in browser.

Where we are currently sending the audio blob to the backend merging service, instead we should use ffmpeg to handle merging of multiple file types. This should also enable us to use all of the filetypes supported by wavesurfer. Once the merging is done, the user will still need to upload the flattened audio file to IPFS.

References

• FFmpeg Github
• Web Audio API AudioWorklet

[noahdahlman]

ffmpeg.wasm is a pure WebAssembly / JavaScript port of FFmpeg. It enables video & audio record, convert and stream right inside browsers. We should try to use this lib to do in browser audio merging.

[drewcook]

The merging logic with ffmpeg would probably be best to be pulled out in a separate file as a utility and then imported into the component file for use, for separation of concerns and to be able to reuse it later, or create a suite of ffmpeg utilities.

[noahdahlman]

This is an example of doing concatenation. See also this , which demonstrates the flags for overlaying audio.