-
-
Notifications
You must be signed in to change notification settings - Fork 233
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Virtual Backgrounds #635
Comments
I'm exactly searching for something like that. |
A lot of people ask for this feature.
Not really, I'll do it when had some time. |
With a lot more people WFH due to COVID-19, I'm sure this feature would be very much appreciated! 😀 |
That would be really great. So i could stop using Zoom! |
@hipersayanX, is there any ways for us to donate to you, for this specific plugin? |
see also this for background manipulation without relying on chroma key |
I'm also going to implement the "multi-chromakey", the key built from many color samples, using ramps and so on. It can give really good results - not as great as using neural networks, but faster. I was already doing this for huge commercial projects. It's just... have to find time for this ;) |
great! |
@beppe9000, the blogpost you linked contains some mistakes. I fixed them, and created a turnkey solution. It should be fairly easy to follow. https://github.com/fangfufu/Linux-Fake-Background-Webcam |
nice! |
Damn, is that much complex to do that? as complex as using Tensorflow for that? |
i always wondered if such a method can react to changing light conditions or camera noise |
@beppe9000 Camera noise can be removed with a light de-noise filter and light changes could be fixed normalizing the luma channel, but I'm talking based on my knowledge, I could be wrong. |
i hope to try it soon or later then |
@hipersayanX, you definitely don't need a convolutional neural network for that. The network in my example can segment out human without learning the background. Not learning the background is not a requirement. I only tidied up his blog post because I can't be bothered to write my own code, and I want something working quickly. |
@hipersayanX, you probably want to use a segmentation approach rather than colour replace, because the skin's RGB value can be quite close to the background RGB value, in some lighting condition. Certain parts of the skin maybe replaced. |
@hipersayanX, finally don't even think about using the bodypix network, because the model is only available in nodejs format... So in the code I gathered, it does this: And yes, it spins up two Docker containers, and they communicate with each other via HTTP... It is really stupid. |
Damn, and they forgot to include Ruby on Rails, Angular, React and JQuery on that list 🤣 |
where is the obligatory mongo database ?! |
It was not "they", it was a one-man job, done by a Google engineer... He even had time to add a hologram effect to the foreground, but he didn't have time to type up the correct command to spin up the Docker containers... I think it says a LOT about Google as a company. |
https://www.reddit.com/r/computervision/comments/g043z9/could_anyone_suggest_a_image_segmentation/ People on Reddit are recommending CNNs... It does make me wonder if people in academia are out of their mind... CNNs are so impractical for a little webcam project... |
Apparently it's possible to replicate Benjamin Elder's example in good old C++ and TensorFlow: https://github.com/floe/deepbacksub |
I think it would be quite a nightmare if webcamnoid suddenly depend on tensorflow-lite to build. I have had difficulty with getting his code working, Bodypix does not seem to be available in Python (tensorflow/tfjs#2043). |
wouln't make sense to have that implementation as a separate plugin? |
I reckon that's what needs to be done. It doesn't make sense to make webcamoid to depend on Tensorflow to build. Or at least set it as an optional plugin. |
I'm interested in this also, so far fake-chroma-key has been fastest, for my poor Haswell mobile CPU. |
Quick question: it appears that the blur is not done if no face is detected, making the whole background visible. Is this intended? Edit: could this be because of the |
@cont1nuity can you open a new bug report for that bug, please? |
Thanks for the reply. Done in #416. |
@cont1nuity thanks 😃 |
I had some time to mess around and created a PR. Reviews and tests would be very welcome! |
Adding several new feature options to adjust the recognized face area as enhancement to #250 / webcamoid#425. - Scale - Width/Height adjustment - H-/V-Offset - Rounded overlay - Overlay Width/Height adjustment - Overlay H-/V-Radius adjustment
FYI, someone just started on an OBS plugin now too that's progressing nicely, might be a good opportunity for some cross knowledge/code sharing. |
Maybe another source of inspiration : |
@hipersayanX , is there a guide on how to write plugins for Webcamnoid? @cont1nuity , Mediapipe now supports selfie segmentation: https://github.com/google/mediapipe/releases/tag/v0.8.5, more information is available at https://google.github.io/mediapipe/solutions/selfie_segmentation, however it doesn't tell you how to do it in C++. I have done an implementation in Python, it works quite well: https://github.com/fangfufu/Linux-Fake-Background-Webcam. I don't know if you are interested in writing a new plugin for Webcamnoid. |
@fangfufu I've not published any official documentation for writing plugins because I'm not sure of keeping the interface as it, but there are some very simple plugins that can be used as references for writing other plugins. |
https://git.jami.net/savoirfairelinux/jami-plugins/-/tree/master/GreenScreen |
@kno10 I will check but, a solution that depends on a privative software or hardware is never a solution, if the algorithm requires hardware acceleration I would prefer it to depend on OpenCL instead. |
I don't know if it can also use OpenCL. There is also some Android loader for the ONNX model. All I only know is that the plugin did not work for me. Jitsi also has prebuilt tensorflow models for background removal: |
Looking for virtual backgrounds, I've just installed WebCamoid. Even if the effect 'virtual background' is still missing, I like this tool very much. Thanx a lot for your work. Please, let me vote for this issue. |
Hello, any implementations for this currently available please? I checked the linked obs plugin and it is rather rudimentary, it is oval shaped and it does not pick up the shape of the head. Interested in keeping my existing background and just blurring it. |
@Svetlana-T this is what I have. It works for me. https://github.com/fangfufu/Linux-Fake-Background-Webcam |
just visiting to check for updates on this important topic. |
@AndKe nope, I have other priorities and other bugs to solve, and no one else beside me is working in the project. |
I've added the ColorKey plugin, not the exact same feature as wanted by op but it's at least least something. |
Hi, thanks. Is the ColorKey plugin documented somewhere? |
@Svetlana-T No, just launch Webcamoid daily build and the plugin is there in the effect list. |
I would like webcamoid to have a Virtual Background feature (like in Zoom) that works without a green screen. Would it be difficult to implement in webcamoid?
The text was updated successfully, but these errors were encountered: