-
Notifications
You must be signed in to change notification settings - Fork 181
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Would a cloud-accelerated export feature be usefull? #1047
Comments
Interesting idea. Personally, I feel like transferring an entire project into the cloud would take longer than just rendering it locally, even on modest hardware. Generally, the typical approach to editing on an underpowered computer is to use proxy clips locally. All of the full sized clips in the project are rendered out into smaller proxy clips (often in a smaller resolution and/or a lighter codec), the project is edited using the proxy clips, and then the real clips are used again for the render. Even with a 10 year old computer and a gigabit internet connection, you'd probably still be ahead on performance to use proxy clips instead of uploading all of the footage into the cloud. The rendering just doesn't take that long with Flowblade. 3D animation programs can definitely benefit from offloaded render farms. There, the assets are much, much smaller, and the amount of computation required to render is orders of magnitude more involved. |
Thanks for the answer! |
I can't speak for anyone myself, but I can't imagine using anything like this in my own projects. Rendering just doesn't take that long, even on an old computer, and I don't personally see any benefit to putting it in the cloud. I feel like it would just be more machinery and moving parts, only to add more delays in the process. |
I honestly don't personally feel the need for a cloud based solution, even on my underpowered machine, I feel I can get good performance on the timeline and the render, I feel most people's internet is worse than the file transfer speed, not mentioning the costs of the upkeep of running servers, especially since flowblade doesn't usually scale well with multiple cores or use a GPU, setting up render farms doesn't sound economical, but yeah maybe an idea could be finding an old PC and setting that up as a local headless render machine? Unsure if the flowblade renderer can run independently in a headless environment, but that's probably a better idea imo since it offloads the rendering to another system, and it's local, so you could transfer it quickly enough. |
Would you have usage of AV1 codec? I believe most people are exporting on the fast H.264 codec but AV1 increase the compression of your videos by a significant margin while demanding too much power on your CPU. |
I personally don't have any usage for AV1, although I had personally never used it, I tried it since, uh, I don't really think it'll be part of my workflow any time soon, I tried a 720p clip and it was a stuttery mess, and a 1080p version of the same file, just didn't run, and in both scenarios my CPU was pegged at either >70% or just at 100% all cores, Also tried transcoding some of my files over to it through ffmpeg, and either it was so dead slow it took 15 minutes to render a single frame, or ffmpeg's implementation is broken, given it warner me about the experimental-ity of this. So all I can say is, if anyone really wants that implementation, sure, it'll just not be useful for me since it already takes a night for a render for me since I render in lossless mpeg4 or h264. As for the render farms question? For me personally not, I have a fast enough internet where I could reasonably be able to upload my files in time, but yeah I honestly wouldn't think it's worth it, plus privacy concerns I would personally have and the economics behind it, I'll rather just do it locally in lower quality, but maybe I'm just a minority, we'll probably have to wait for other people's opinions maybe |
@Disaster2life Can you tell me more about your privacy concern? I'm not use to having these kind of issues and you may not be alone with this. Is it impossible for you that a cloud option becomes reliable enough to trust it with your application? |
Whenever I'm working on an editing project, the final output from Flowblade is in the highest quality that I can possibly get out of it. Right now, for me, this means the ProRes 422 HQ codec. The source files from my Fujifilm X-T3 camera take up about 1.5 GB per minute of footage. These are then transcoded into ProRes 422 HQ and synced with dual system sound that was recorded on a separate sound recorder. The transcoded ProRes 422 HQ files take up about 5 GB per minute of footage. The final export from Flowblade is also in ProRes 422 HQ, taking up 5 GB per minute of the final project. A typical project might have a 15 to 1 shooting ratio or so. That means that for every 1 minute of final output, there were about 15 minutes of source footage. On my computer (a Dell desktop machine), transcoding the footage locally, using 6 cores, is done somewhere between half and one times the speed of play. In other words, that footage (which is being transcoded and synced with a separate audio source) takes 1 hour to transcode between a half hour and one hour of source material. If I have three hours of raw footage, and a 12 minute final project, now the footage looks like this:
In order to perform this render in the cloud, here is what that would look like for me:
All in all, for me, this would amount to uploading almost 1 TB of footage to the cloud, waiting nearly a full day for the uploading and downloading, to get the cloud rendering part to do something that would have taken me 15 minutes on my local computer. When you have enough data, it has a certain level of gravity, and tends to pull computation into its orbit. |
Typically, my rendering from Flowblade is out to a high quality master. Then, from there, I may transcode to other distribution formats as necessary. For that rendering step, I'd just use ffmpeg, and bypass Flowblade altogether. I will say that Flowblade is, at heart, a desktop video editing application geared toward more professional use. I'm not sure that a bunch of integrations with Internet services is desirable, as they tend to come and go, and could likely complicate the codebase anyway. @bbasics it sounds like you're really interested in the possibility of adding an AV1 cloud rendering option to Flowblade. Can you expand on how you see this benefiting Flowblade users? I'm having trouble imagining how someone might benefit from this within the context of a typical Flowblade editing workflow. |
It looks like some may be interested in rendering using render farms.
I've started to work on that a long time ago but never finished it or got
it to a clean enough state to submit it, basically because I got a faster
computer.
It worked fine, though.
This is how my version works:
1. There is a checkbox in fb that determines local render or farm
2. Setup tab allows you to enter the ip of all render slaves (I use local
ones, but remote is possible). Source file location is specified (could be
local access on a shared drive or remote) and relative rendering speed of
each machine is entered so your own 386 can also be used
3. Render is launched. Based on speed fb cuts the render in chunk and
assigns each chunk to render a slave.
The slave (a headless Melt server) renders it and sends the result to the
output directory.
4. All rendered chunks are merged for the final render.
There is a fairly crude autobalance mechanism you can use to update the
renderspeed of each slave automatically.
If there is interest I can pick this project back up. It worked pretty
neat, I thought.
Thoughts?
Regards
Steven
…On Wed, 16 Feb 2022, 18:05 Nathan Rosenquist, ***@***.***> wrote:
Typically, my rendering from Flowblade is out to a high quality master.
Then, from there, I may transcode to other distribution formats as
necessary. For that rendering step, I'd just use ffmpeg, and bypass
Flowblade altogether.
I will say that Flowblade is, at heart, a desktop video editing
application geared toward more professional use. I'm not sure that a bunch
of integrations with Internet services is desirable, as they tend to come
and go, and could likely complicate the codebase anyway.
@bbasics <https://github.com/bbasics> it sounds like you're really
interested in the possibility of adding an AV1 cloud rendering option to
Flowblade. Can you expand on how you see this benefiting Flowblade users?
I'm having trouble imagining how someone might benefit from this within the
context of a typical Flowblade editing workflow.
—
Reply to this email directly, view it on GitHub
<#1047 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AB4TCP6UBG6SGM4VZVHCBVDU3PKOJANCNFSM5OAYJUUA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
I mean, uploading to an external server, I'm worried on how long it's staying there, what if a data leak happens, man in the middle attacks, I haven't done anything that requires a real level of privacy, but just generally adding the step of uploading my data onto some servers I do not physically maintain ... Is a gamble for me. I mean, I wouldn't say impossible, but I generally think not, unless it's run super openly, and i basically live next to it, I'm pretty sure I am better off just spending that cash upgrading my current system, plus I honestly am not into the whole, "let's shift everything to the cloud" philosophy, I don't see the point of adding all these points of failures and new issues when instead i could just render this out locally, why I suggested local headless machines at first. As ratherlargerobot mentioned, I don't really have their scale of work since I'm broke and just starting off and would definitely love to geek out about their hardware, even on my smaller scale projects I capture off of my phone become hard to do on the cloud really quickly, not to mention data caps and stuff that are limitations for many, looking at my case I will probably move elsewhere and might be stuck with really small data plans, I feel there is a reason why the industry has stuck to local render farms at large studios and production places. Also Steve, wish i could see it work, sound really cool and sorta like what i had in my head about headless servers, wonder if we could just have a "Flowblade Media Encoder" like adobe has, which could just handle the rendering and distributing all the render jobs between like a cluster of computers or maybe even Pis (sounds inefficient but super cool) |
I wonder if a cloud-accelerated export feature could improve the video edition workflow? I’m trying to get opinions here before implementing such a feature.
How I see it, this could mean:
However I suppose the service could not be free as it requires a dedicated infrastructure, but it would only be an option.
I'm not talking about the technical difficulty for now, but I'm trying to know if it would be used or not.
What do you think?
The text was updated successfully, but these errors were encountered: