Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Any plans to have a version without SharedArrayBuffer? #137

Closed
PaTiToMaSteR opened this issue Dec 23, 2020 · 69 comments
Closed

Any plans to have a version without SharedArrayBuffer? #137

PaTiToMaSteR opened this issue Dec 23, 2020 · 69 comments

Comments

@PaTiToMaSteR
Copy link

Is your feature request related to a problem? Please describe.
It's a known problem that SharedArrayBuffer was removed due to Spectre and Meldown hacks.

Describe the solution you'd like
A safe version without SharedArrayBuffer

Describe alternatives you've considered
I don't have the knowledge in Emcripten nor how you transpile the code to know if it's possible.

Additional context
To be able to run this amazing project everywhere we need a version without SharedArrayBuffer!

@Quackdoc
Copy link

sharedarraybuffer support is still available in firefox and chrome. and will soon be enabled by default in both again as the security issues have been mitigated (see coop/cope)

I believe chrome desktop has enabled it already, snd chrome mobile is working on it
https://www.chromestatus.com/feature/5171863141482496

Basically, there is no need infact if you follow the guide it should work on updated firefox and chrome no?

@PaTiToMaSteR
Copy link
Author

That’s good to know, however my target is cross-platform so iOS, Android, PC... without that just simply doesn’t work use ffmpeg as a base framework. I was wondering if can be like a temporal workaround using other method instead of that🤔

@Quackdoc
Copy link

again shared buffer array is comming to mobiles (im not sure about IOS but im sure it wouldn't be too hard to find out), firefox nightly on android starts it but crashes before it starts encoding suggesting the issues something else

@rakeshb-iprogrammer
Copy link

Any other alternate way to achieve trimming video on mobile web app browser?

@jeromewu
Copy link
Collaborator

For the version without SharedArrayBuffer, I would suggest to use ffmpeg.js (https://github.com/Kagami/ffmpeg.js/), right now ffmpeg.wasm doesn't have plan to do that.

@selmalee
Copy link
Contributor

Can i build my own ffmpeg.wasm without SharedArrayBuffer(using ffmpeg.wasm-core)? Where is SharedArrayBuffer used ?Thank you @jeromewu

@jeromewu
Copy link
Collaborator

@seminelee I would suggest you use ffmpeg.js instead, it doesn't use SharedArrayBuffer.

SharedArrayBuffer in ffmpeg is used to enable multi-threading, it makes it faster when transcoding media.

@selmalee
Copy link
Contributor

selmalee commented May 20, 2021

@seminelee I would suggest you use ffmpeg.js instead, it doesn't use SharedArrayBuffer.

SharedArrayBuffer in ffmpeg is used to enable multi-threading, it makes it faster when transcoding media.

@jeromewu thank you. But I can't find SharedArrayBuffer both in ffmpegwasm/ffmpeg.wasm and ffmpegwasm/ffmpeg.wasm-core. Searching SharedArrayBuffer in code:

image

image

@jeromewu
Copy link
Collaborator

You can find it here: https://unpkg.com/@ffmpeg/core@0.10.0/dist/ffmpeg-core.js
It is in the output of the build script, usually we don't store it in the repository.

@Evengard
Copy link

Evengard commented Jun 30, 2021

Unfortunately ffmpeg.js doesn't support libtheora (I need encoding), and Firefox addons are supporting SharedArrayBuffers only for "privileged addons" (and I wanted to use it for an addon). Is there really no way to build it without SAB? I don't really mind being it single-threaded.

@jeromewu
Copy link
Collaborator

Acutually you can build it on your own by removing all pthread support in build script, so it is totally possible but takes time to develop.

https://github.com/ffmpegwasm/ffmpeg.wasm-core/tree/n4.3.1-wasm/wasm/build-scripts

Currently I don't have plan to do that as single thread version is slow and conflicts with ffmpeg.js.

@yahsaves
Copy link

I'm trying to build a version with pthreads removed, as I need to target ios.

Speed is not important, as my use case is to extract audio from a user video upload.

Normally I would do this with audioContext.decodeAudioData, but safari doesn't support decoding their own MOV codec.

FFmpeg.js does not recognize a video file recorded from iPhone, but this project contains the libraries to work with a MOV file.

So using wasm.core I removed all the pthread references from the build script, and removed the -j flag from the MAKE commands. However the build does not produce a dist folder with the wasm files. The terminal log is very long, and there is no immediate error's at the end so I'm not sure where to start debugging.

Any advice how to debug building this project, or anything else I should look for that could cause the build to fail?

I would even consider paying someone to help build this project, as I really need it for my use case.

@jeromewu
Copy link
Collaborator

Maybe I can help to build the single thread version and put it somewhere, but to use it, you still need to write a wrapper like ffmpeg.wasm to load the ffmpeg.wasm-core for you.

Will update here once I build the single-thread version.

@yahsaves
Copy link

That would be amazing! Like to the 10th degree of awesome.

As for the wrapper, would there be significant changes from the utils.js file provided in the core example?

I was looking through the browser examples, and I didn't see anything in particular that looks like it had to change to support a single thread. Unless the calls utils makes to the 'createFFmpegCore' function would require different parameters.

@ZYMoridae
Copy link

@jeromewu May I ask is there any updates on the latest single-thread version?

@jeromewu
Copy link
Collaborator

jeromewu commented Aug 1, 2021

I plan to release it after updating ffmpeg.wasm, but if you are in a hurry, you can download the singled threaded version here:

https://github.com/ffmpegwasm/ffmpeg.wasm-core/actions/runs/1050801687 (You can download it in artifacts section)

And you can check here for how to use it: https://github.com/ffmpegwasm/ffmpeg.wasm-core/tree/n4.3.1-wasm/wasm/tests/utils/st

(Sorry I only write a node.js version for testing, but it shouldn't be too hard to write a web worker version on your own)

@ZYMoridae
Copy link

ZYMoridae commented Aug 2, 2021

I plan to release it after updating ffmpeg.wasm, but if you are in a hurry, you can download the singled threaded version here:

https://github.com/ffmpegwasm/ffmpeg.wasm-core/actions/runs/1050801687 (You can download it in artifacts section)

And you can check here for how to use it: https://github.com/ffmpegwasm/ffmpeg.wasm-core/tree/n4.3.1-wasm/wasm/tests/utils/st

(Sorry I only write a node.js version for testing, but it shouldn't be too hard to write a web worker version on your own)

Does that means once you update the ffmpeg.wasm we could directly use it in the browser with ffmpeg.wasm?

@selmalee
Copy link
Contributor

selmalee commented Aug 2, 2021

I have build a version without SharedArrayBuffer by this build script: https://github.com/seminelee/ffmpeg.wasm-core/blob/n4.3.1-wasm/build1.sh
Hope it can help someone~

By the way, is ffmpeg-core.worker.js neccesary for a custom ffmpeg-core without pthreads? thank you @jeromewu

@ZYMoridae
Copy link

I have build a version without SharedArrayBuffer by this build script: https://github.com/seminelee/ffmpeg.wasm-core/blob/n4.3.1-wasm/build1.sh

Hope it can help someone

Do you have a example for browser version?

@jeromewu
Copy link
Collaborator

jeromewu commented Aug 2, 2021

Does that means once you update the ffmpeg.wasm we could directly use it in the browser with ffmpeg.wasm?

@ZYMoridae Yes, that is the plan, but I haven't got enough time to complete it.

@jeromewu
Copy link
Collaborator

jeromewu commented Aug 2, 2021

@seminelee Nope, I don't think ffmpeg-core.worker.js is required for non-pthread version.

@selmalee
Copy link
Contributor

selmalee commented Aug 2, 2021

@seminelee Nope, I don't think ffmpeg-core.worker.js is required for non-pthread version.

can i build a version with web worker but without sharedarraybuffer?TAT

@jeromewu
Copy link
Collaborator

jeromewu commented Aug 2, 2021

I would suggest you write your own web worker script, it is actually easier than building one.

@selmalee
Copy link
Contributor

selmalee commented Aug 2, 2021

I would suggest you write your own web worker script, it is actually easier than building one.

If I write a web worker script, can I still use this project @ffmpeg/ffmpeg ?

@jeromewu
Copy link
Collaborator

jeromewu commented Aug 2, 2021

Nope, in that case you cannot use @ffmpeg/ffmpeg as it is not compatible.

@julienbeisel
Copy link

Hi @jeromewu, do you have an example of how to use the single thread ffmpeg build? I spend some time trying to use the ffmpeg-core single-thread version but I can't make it work. I already read the test files 20 times but I don't really understand it.

Thanks for your help :)

@selmalee
Copy link
Contributor

selmalee commented Aug 31, 2021

Hi @jeromewu, do you have an example of how to use the single thread ffmpeg build? I spend some time trying to use the ffmpeg-core single-thread version but I can't make it work. I already read the test files 20 times but I don't really understand it.

Thanks for your help :)

The key is , in emcc, to move -s USE_PTHREADS=1 , and to move _proxy_main in -s EXPORTED_FUNCTIONS.
Also, I suggest that you can use web worker to run codes about ffmpeg in your project.

I have written an article about it, but in Chinese: https://juejin.cn/post/6998876488451751973
Hope it can help.^^

@zackees
Copy link

zackees commented Jan 8, 2022

The shard buffer issue is a blocker for this code.
Please have a single version wasm available!

@chicken-suop
Copy link

chicken-suop commented Jan 14, 2022

@seminelee @Aeroxander I'm getting the same error when trying your build script. Any ideas on how to resolve it?

wasm-ld: error: unable to find library -lavdevice
wasm-ld: error: unable to find library -lswresample

EDIT: I sorted above error by removing -Llibavdevice and -lavdevice from emcc args.

libx264

@seminelee Do you know how to add support for libx264? I tried adding --enable-encoder=libx264 and --enable-libx264 but I'm getting this error:

17 warnings generated.
emcc: warning: EXTRA_EXPORTED_RUNTIME_METHODS is deprecated, please use EXPORTED_RUNTIME_METHODS instead [-Wdeprecated]
wasm-ld: error: libavcodec/libavcodec.a(libx264.o): undefined symbol: x264_levels
wasm-ld: error: libavcodec/libavcodec.a(libx264.o): undefined symbol: x264_levels
wasm-ld: error: libavcodec/libavcodec.a(libx264.o): undefined symbol: x264_levels
emcc: error: '/home/ta/ffmpeg/emsdk/upstream/bin/wasm-ld -o wasm/dist/ffmpeg-core.wasm -Llibavcodec -Llibavfilter -Llibavformat -Llibavresample -Llibavutil -Llibpostproc -Llibswscale -Llibswresample /tmp/emscripten_temp__as0855l/ffmpeg_opt_0.o /tmp/emscripten_temp__as0855l/ffmpeg_filter_1.o /tmp/emscripten_temp__as0855l/ffmpeg_hw_2.o /tmp/emscripten_temp__as0855l/cmdutils_3.o /tmp/emscripten_temp__as0855l/ffmpeg_4.o libavfilter/libavfilter.a libavformat/libavformat.a libavcodec/libavcodec.a libswresample/libswresample.a libswscale/libswscale.a libavutil/libavutil.a -L/home/ta/ffmpeg/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten /home/ta/ffmpeg/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/libSDL2.a -lGL -lal -lhtml5 -lstubs -lnoexit -lc -lcompiler_rt -lc++-noexcept -lc++abi-noexcept -ldlmalloc -lc_rt -lsockets -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --import-undefined --strip-debug --export-if-defined=main --export-if-defined=__start_em_asm --export-if-defined=__stop_em_asm --export=stackSave --export=stackRestore --export=stackAlloc --export=__wasm_call_ctors --export=__errno_location --export=_get_tzname --export=_get_daylight --export=_get_timezone --export=malloc --export=memalign --export=free --export=memcpy --export=__dl_seterr --export-table -z stack-size=5242880 --initial-memory=33554432 --no-entry --max-memory=2147483648 --global-base=1024' failed (returned 1)

This is the updated build1.sh I use: https://gist.github.com/chicken-suop/f1fa1f665d699f0d20880a7ccacd7940

@chicken-suop
Copy link

Currently, there is only one way to use ffmpeg.wasm in production

Not true. You can compile a single core wasm yourself. I've done that, and it works fine in prod, as long as you can accept average speeds.

@channyeintun
Copy link

If I set headers for cross origin isolation which is needed to enable shared array buffer, I cannot download things from other site. Could anyone has a solution to that problem?

@chicken-suop
Copy link

@channyeintun You can build your own ffmpeg.wasm single thread that doesn't need shared array buffer. @seminelee made some comments above about how to do it. I shared the build1.sh file I use to build my own version. Only thing I haven't figured out is how to enable libx264 encoder

@brunoluiz
Copy link

After seeing so many replies with the same subject, I am going to publish here how to deal with this and how you can compile ffmpeg yourself (some might be afraid or lazy).

Because of the nature of ffmpeg, it is better to have multi-threading to have a better performance. This requires SharedArrayBuffer though, which is the reason many complain it doesn't work / it is not production ready.

@jeromewu already added support to single-thread in the main branch of ffmpeg.wasm-core. The trick is that it is not being published to npm, but it is ready for use basically (trust me). Saying that, to use it you have to do two things:

Build ffmpeg.wasm-core

  1. You can build manually by heading to https://github.com/ffmpegwasm/ffmpeg.wasm-core and following the instructions
  2. You can:

Fix ffmpeg.wasm

In your application, you will realise quite quickly that just using the new compiled ffmpeg will not work (some error message around main or proxy_main).

When you build without multi-threading, proxy_main is not published anymore, if i understood correctly. It is because this is used as a proxy entry point by ECMScripten when dealing with multi-threading + workers to not block the main thread (more details here). With the single-threaded version you want to use main as an entry point.

The easiest way to deal with this is (again) forking this repository, applying this patch and build a dist: #235. This one is quite straight forward to do locally (npm ci && npm build basically).

Now, in theory your application should work with the following options:

const ffmpeg = createFFmpeg({
  corePath: './<path-to-ffmpeg-core-dist>/ffmpeg-core.js',
  mainName: 'main'
});

Conclusion

While someone doesn't push these changes in a nicer packaged way, there are still work arounds.

About Chrome/Firefox extensions

It seems Chrome manifest v2 extensions have support to SharedArrayBuffer, but Firefox doesn't (as of Jan 2021). So, if you are planning to use the instructions above for browser extensions, bear in mind you will need two versions of the same thing.

@brunoluiz
Copy link

I usually vendor the libs used on my Chrome Extensions, so if anyone is looking for compiled versions of the ffmpegwasm-core single-threaded, you are welcome to copy the files at src/vendor/ffmpeg-st*: https://github.com/brunoluiz/gifsane-extension. The ffmpegwasm in there as well is patched to fix the proxy aforementioned issues.

I've written a post with the tricks and caveats around implementing FFmpeg in an extension. Mainly a bit of pain on Firefox, due to the missing SharedArrayBuffer, but I had my way: https://brunoluiz.net/blog/2022/jan/gif-sane-playback-control-ffmpegwasm/

@merri-ment
Copy link

merri-ment commented Feb 16, 2022

Hi @brunoluiz

Thanks for putting together the post.

I have followed it step-by-step (and also tried your vendor files), but i keep coming up against this error when running the single threaded version (Chrome emulator, v98);

Uncaught (in promise) RuntimeError: abort(CompileError: WebAssembly.instantiate(): expected magic word 00 61 73 6d, found 0a 76 61 72 @+0). Build with -s ASSERTIONS=1 for more info.

and similar, yet differently worded error on Safari v15.3 ;

Unhandled Promise Rejection: RuntimeError: abort(CompileError: WebAssembly.Module doesn't parse at byte 0: module doesn't start with '\0asm'). Build with -s ASSERTIONS=1 for more info.

My implementation;

import { createFFmpeg } from '@/vendor/ffmpeg.min'

load = () => {
    return new Promise(async resolve => {
      this.ffmpeg = createFFmpeg({
        log: true,
        corePath: './ffmpeg/ffmpeg-core.st.js',
        mainName: 'main',
      })
      await this.ffmpeg.load()
      resolve()
   })
}

Have you encountered this?

Any help appreciated,
Thanks Cam

@merri-ment
Copy link

Ok, managed to resolve it.
It was to do with the 'st' / 'mt' - id's within the file name.
I didn't following your symbolic link approach, therefore the ffmpeg-core.js could never locate ffmpeg-core.wasm.

@web86
Copy link

web86 commented Apr 13, 2022

Hello @brunoluiz
Thanks for the guidence, I have followed all steps, but stacked with the last one:

The easiest way to deal with this is (again) forking this repository, applying this patch and build a dist: #235. This one is quite straight forward to do locally (npm ci && npm build basically).

Sorry for maybe noob question. After changing createFFmpeg.js acording to #235 I should build a package, from this point could you explane more detaily? How to build the package? Can I build it via workflows/main.yml? If so could you share your yml script?

@brunoluiz
Copy link

Sorry for maybe noob question. After changing createFFmpeg.js acording to #235 I should build a package, from this point could you explane more detaily? How to build the package? Can I build it via workflows/main.yml? If so could you share your yml script?

To build the package, you need to:

  1. Clone the ffmpeg.wasm locally
  2. Apply the patch
  3. Call npm ci (installs dependencies)
  4. Call npm build
  5. (Optional) If you want to have your own patched version saved somewhere, you might want to push to your own fork

This will generate a new JS package. The caveat is that, if you want to install in other projects through npm, you might need to publish in npm. You can publish perhaps under a personal scope, such as @USER/ffmpegwasm or something, but PLEASE leave a README note saying that this is a fork etc. As I used it in a Chrome extension, I didn't need to go through this (copied to a local lib folder).

@jimbojw
Copy link

jimbojw commented Dec 19, 2022

What's the conclusion for this? Are we still patching and building ourselves, or is there a simple way to disable the dependency on SharedArrayBuffer?

@jimbojw
Copy link

jimbojw commented Dec 19, 2022

I added mainName: 'main' to the options to my createFfmpeg() call, but it still throws a ReferenceError about SharedArrayBuffer not being defined.

@jimbojw
Copy link

jimbojw commented Dec 19, 2022

Looks like the problem is that this fix is in 0.11.1, but the highest version on npm is 0.11.0.

@jimbojw
Copy link

jimbojw commented Dec 19, 2022

Update: Yes, if you fetch the 0.11.1 version, which is not yet available on npm, you can use the mainName option.

@FrenchGithubUser
Copy link

Any news about a potential easier way instead of having to build ourselves ?

@jimbojw
Copy link

jimbojw commented Feb 5, 2023

@FrenchGithubUser The unreleased v0.12.0-alpha works in non-SharedArrayBuffer mode. Code: https://github.com/ffmpegwasm/ffmpeg.wasm/tree/v0.12.0-alpha.1

You can install it from npm: npm install @ffmpeg/ffmpeg@0.12.0-alpha.1

However, note that the API is pretty different, and there aren't good docs yet.

@FrenchGithubUser
Copy link

I'm having an issue trying to run the latest beta version. Could someone help me figuring out what is wrong with what I did ?

Versions used :

    "@ffmpeg/core": "^0.12.0-alpha.2"
    "@ffmpeg/ffmpeg": "^0.12.0-alpha.1"

Here is my code :

import { FFmpeg } from '@ffmpeg/ffmpeg'

      let load = () => {
        return new Promise(async (resolve) => {
          const ffmpeg = new FFmpeg({
            log: true,
            corePath: './ffmpeg/ffmpeg-core.js',
            mainName: 'main',
          })
          ffmpeg.load()
          resolve()
        })
      }
      load()

And I'm getting these errors :

failed to send download progress event:  RangeError: Invalid typed array length: -1
    at new Uint8Array (<anonymous>)
    at downloadWithProgress (utils.js?0649:25:1)
    at async toBlobURL (utils.js?0649:61:1)
    at async load (worker.js?ab3b:16:1)
    at async self.onmessage (worker.js?ab3b:80:1)
1 Uncaught (in promise) TypeError: Failed to execute 'arrayBuffer' on 'Response': body stream is locked
    at downloadWithProgress (utils.js?0649:45:1)
    at async toBlobURL (utils.js?0649:61:1)
    at async load (webpack-internal:///./node_modules/@ffmpeg/ffmpeg/dist/esm/worker.js:20:19)
    at async self.onmessage (webpack-internal:///./node_modules/@ffmpeg/ffmpeg/dist/esm/worker.js:84:24)

@yagudaev
Copy link

yagudaev commented Apr 19, 2023

@FrenchGithubUser, you might want to file a different bug report to make sure the team sees it and clearly indicate the alpha version you are using

@jimbojw we are excited to give the new alpha a spin today 😁

@Boyyoka
Copy link

Boyyoka commented Jul 16, 2023

It seems that you no longer need to build any of the libraries or to rely on the alpha version as there is a hosted single threaded version on unkpg.

  const ffmpeg = createFFmpeg({
          corePath: "https://unpkg.com/@ffmpeg/core-st/dist/ffmpeg-core.js",
          mainName: "main"
  })

I'm still looking for ways to have the corePath take a local path, but it seems to raise issues that are for another thread.

So for now I'll just rely on the above solution. I hope this can help someone out there.

@Rizzist
Copy link

Rizzist commented Oct 2, 2023

It seems that you no longer need to build any of the libraries or to rely on the alpha version as there is a hosted single threaded version on unkpg.

  const ffmpeg = createFFmpeg({
          corePath: "https://unpkg.com/@ffmpeg/core-st/dist/ffmpeg-core.js",
          mainName: "main"
  })

I'm still looking for ways to have the corePath take a local path, but it seems to raise issues that are for another thread.

So for now I'll just rely on the above solution. I hope this can help someone out there.

I can confirm this is THE Solution
saved me from devleoper hell ty dude

@SpaceSaver
Copy link

It seems that you no longer need to build any of the libraries or to rely on the alpha version as there is a hosted single threaded version on unkpg.

  const ffmpeg = createFFmpeg({
          corePath: "https://unpkg.com/@ffmpeg/core-st/dist/ffmpeg-core.js",
          mainName: "main"
  })

I'm still looking for ways to have the corePath take a local path, but it seems to raise issues that are for another thread.

So for now I'll just rely on the above solution. I hope this can help someone out there.

Correct me if I'm wrong, but I believe it's now just https://cdn.jsdelivr.net/npm/@ffmpeg/core@0.12.4/ without the "-st." Could be something else though.

@Evengard
Copy link

Correct me if I'm wrong, but I believe it's now just https://cdn.jsdelivr.net/npm/@ffmpeg/core@0.12.4/ without the "-st." Could be something else though.

Wouldn't that be the multithreading version? Aka st means "single thread".

@SpaceSaver
Copy link

Correct me if I'm wrong, but I believe it's now just https://cdn.jsdelivr.net/npm/@ffmpeg/core@0.12.4/ without the "-st." Could be something else though.

Wouldn't that be the multithreading version? Aka st means "single thread".

But this is the multithreading version: https://cdn.jsdelivr.net/npm/@ffmpeg/core-mt@0.12.4/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests