Hardware Acceleration for RPi4 under 64-bit OS #134
Comments
|
This is interesting, it always makes me happy seeing all the different applications people find for Weylus!
Yeah, that should be possible. Anything ffmpeg can encode, Weylus can also be made to encode :). But first try to check if ffmpeg itself works fine with v4l2, the following command closely resembles what Weylus needs to do internally: Lines 359 to 383 in 67500d8 As quick and dirty hack you could just adapt above code to work with |
|
Wow! nice! I will try that, and conclude it here I tried running the above said command, and it seemed to work! although the video was a black screen (i think that was because of missing codecs), which indicated that it found the right video encoder, and it seemed pretty fast when I ran compared with the default one!
I then tried to recompile the whole package with that portion you mentioned altered, but when connecting it says that the codec was not found.. I'll dig a bit deeper on another day, as it takes nearly 20 min to compile everything on my RPi, but it seems promising |
|
Hi! Any thoughts on this? hule-ka:/usr/local/src/Weylus/target/release $ ./weylus --access-code=123 --auto-start --no-gui
2022-01-21T18:46:40.241783Z INFO weylus::web: Webserver listening at 0.0.0.0:1701...
2022-01-21T18:46:43.285481Z INFO weylus::websocket: Client connected. address=192.168.6.3:54286
2022-01-21T18:46:43.303811Z INFO weylus::websocket: WS-Client authenticated! address=192.168.6.3:54286
2022-01-21T18:46:44.115261Z INFO weylus::log: Using device /dev/video11
2022-01-21T18:46:44.115423Z INFO weylus::log: driver 'bcm2835-codec' on card 'bcm2835-codec-encode' in mplane mode
2022-01-21T18:46:44.115524Z INFO weylus::log: requesting formats: output=YU12 capture=H264
2022-01-21T18:46:44.165597Z WARN weylus::log: Failed to set timeperframe
2022-01-21T18:46:44.166003Z WARN weylus::log: Failed to set gop size: Invalid argument
2022-01-21T18:46:44.178114Z INFO weylus::log: Video: 1920x1080@h264_v4l2m2m pix_fmt: yuv420p
2022-01-21T18:46:44.485334Z INFO weylus::log: ff_v4l2_buffer_buf_to_avpkt
2022-01-21T18:46:44.485408Z WARN weylus::log: Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2022-01-21T18:46:44.485442Z WARN weylus::log: Encoder did not produce proper pts, making some up.
2022-01-21T18:46:44.485760Z INFO weylus::log: ff_v4l2_buffer_buf_to_avpkt
2022-01-21T18:46:44.644296Z INFO weylus::log: ff_v4l2_buffer_buf_to_avpkt |
|
Is the video playable at all (e.g. via ffplay or mpv)? It looks like h264_v4l2m2m doesn't support all the options Weylus uses. You might want to play around with these parameters: Lines 121 to 126 in 33cf5f1 It's also possible that h264_v4l2m2m doesn't like encoding at a variable framerate. Weylus sets a so called time base to 1/1000, which means timestamps of frames are recorded as precisely as 1/1000th of a second. Weylus then encodes frames as fast as possible and sets the timestamp accordingly: Line 476 in 33cf5f1 |
|
Thank you for your support. I'm working on this and try to get encoding working with different parameters. Is there a way to try the values for time_base, framerate and gop without recompiling the software? |
|
It's possible but a little complicated. Instead I'd suggest abusing environment variables for testing. Once you are done you can remove this "hack". Just add something like int gop_size = atoi(getenv("GOP_SIZE"));to read a value from the environment. Then make sure to start Weylus like this |
|
I'm sorry for being delayed but I'm not a developer and even more not familiar with c code. I've tried to read the environment variables during runtime but it's not working. weylus is complaining that GOP_SIZE is not set correctly. Can you please elaborate the necessary code changes a bit more. Sorry for my missing understanding and thank you in advance. |
|
Don't worry. This is how you could do it, if you want to play with gop size and framerate: diff --git a/lib/encode_video.c b/lib/encode_video.c
index e2f0a4c..8316b34 100644
--- a/lib/encode_video.c
+++ b/lib/encode_video.c
@@ -115,13 +115,19 @@ void init_ffmpeg_logger() { av_log_set_callback(log_callback); }
void set_codec_params(VideoContext* ctx)
{
+
+ int gop_size = atoi(getenv("GOP_SIZE"));
+ int frame_rate_num = atoi(getenv("FRAME_RATE_NUM"));
+ int frame_rate_den = atoi(getenv("FRAME_RATE_DEN"));
+
/* resolution must be a multiple of two */
ctx->c->width = ctx->width_out;
ctx->c->height = ctx->height_out;
ctx->c->time_base = (AVRational){1, 1000};
- ctx->c->framerate = (AVRational){0, 1};
+ ctx->c->framerate.num = frame_rate_num;
+ ctx->c->framerate.den = frame_rate_den;
- ctx->c->gop_size = 12;
+ ctx->c->gop_size = gop_size;
// no B-frames to reduce latency
ctx->c->max_b_frames = 0;
if (ctx->oc->oformat->flags & AVFMT_GLOBALHEADER)Make sure to start Weylus with |
|
Thank you for the patch. I can build weylus with the patch and I've tried different settings for the newly created environment variables. Actually I have the impression that this is not the right direction because the error messages regarding GOP_SIZE is unchanged. I can see a changing number regarding fps but in chromium I get the error message that the video stream can't be decoded. |
|
You can get all ffmpeg log messages by increasing the log level of Weylus like this: Thanks for doing some more research! It looks like ffmpeg contains a bug which prevents proper encoding. There is a patch linked in the report. I included the patch in this branch, lets hope it works: https://github.com/H-M-H/Weylus/tree/rpi_v4l2 |
|
Thank you again for your effort and your time. I'm working with the new rpi branch. Today my sd-card with the 32-bit os died so I switched to the sd-card with an 64-bit os. Actually on the 64-bit raspbian I get an linker error in the last step of building weylus. At the moment a build with the "--release" option is running and I'll report back if this is successful. |
|
The error persists: I believe this is related to the rust or cargo version. Unfortunately I'm not sure how to update to a newer version within rasbian. Update |
|
I tried with Update |
|
Looks like building ffmpeg failed. Please check the contents of If they are not, navigate into |
|
As I understand the build process I can pas --features ffmpeg-system to the build command to use the system libraries? Is this the wrong way? Is it mandatory in the rpi_4vl branch to build ffmpeg in the weylus build process? Update |
|
Oh, now I see the problem! Yes, it is mandatory as ffmpeg needs to be patched. The system version won't work. |
|
Sorry but I thing the rpi branch is not ready. I see two issues
My next attempt was to recompile ffmpeg out of the debian source package and before compiling I patched v4l2_m2m_enc.c as suggested in the report (it's the same what the patch in the weylus internal build of ffmpeg does). To start with the good part: The patch works and I get "Encoder: repeat parameter sets = 1" in the weylus trace. The bad part I still have no video and a new error: Any suggestions on that invalid, non monotonically ... ? |
|
Here is the output of v4l2-ctl -L -d /dev/video11 |more `Codec Controls |
|
I'm still working on a way to use Weylus with the raspberry pi but without success. What I tried: These are the parameters I set for weylus /* resolution must be a multiple of two */
ctx->c->width = ctx->width_out;
ctx->c->height = ctx->height_out;
ctx->c->time_base = (AVRational){1, 100};
//ctx->c->time_base = -1;
ctx->c->framerate = (AVRational){1, 25};
//ctx->c->framerate = 120;
//av_opt_set(ctx->c->priv_data, "qp", "23", 0);
//av_opt_set(ctx->c->priv_data, "sequence_header_mode", "1", 0);
//ctx->c->gop_size = 12;
// no B-frames to reduce latency
ctx->c->max_b_frames = 0;This is the log file: Any help is appreciated. |
|
Thanks for further investigating here @saturn-hh ! I'd recommend trying to get encoding with the This invocation of ffmpeg is pretty close to what Weylus does, check if you can get this working: I hope this is at least of some help! |
|
I've tried the command you mentioned and at the end of the encoding process I got a file and no error message, BUT the file isn't playable. I found this post witch is maybe related to the issue. Investigating further with ffprobe give me this report: This is the report of the encoding process: |
Does this mean no video at all or a corrupted file like in the linked issue? Please check if this fails too: If it does and you can't get ffmpeg itself to properly encode a test video, I'd suggest asking the people at the #ffmpeg irc channel on libera.chat. |
|
We are going in the right direction! First time that I got a picture in the browser. The key is to change the movflags. Without the flag empty_moov its working. Patching ffmpeg or libavcodec is NOT neccessary. System is an Raspberry pi OS 64 bit bullyeys with packages from the repos. Picture is choppy but I'll try some tuning. Cheers Allexander |
|
Nice! In that case adjusting Line 441 in 31780cd |
|
I can't make it happen. Probably there is a regression in the actual version of h264_v4l2m2m in 32 and 64 bit version. I haven't found a way to produce a valid fragmented mp4 file. With the movflag "empty_moov" the mp4 is not playable and without this movflag the steam given by weylus to the browser is not playable. I've tried different combinations of movflags here. I've opened a report here and here but unfortunately none responded. In the depreciated version of Rasbian Buster I can produce a correct mp4 file but I can't compile weylus. No matter what I try, in the final step I get this linker error:
I've updated gcc and ld manually but the error persists. So now I'm out of options. Any ideas about this? |
|
Hi. Instead of RPi, I am working with a Jetson Nano 2G. With a modified version of Edit: I find out I was building a static library but asking weylus to use the system shared library. I am rebuilding with the static library to see if there any problem persists. Edit: I decided to build ffmpeg as shared library instead and this time it works, although the performance is not good. I am still tuning the parameters to see if I can get any improvement. |
Hey! first of all, thank you for developing this wonderful tool, it has so many purposes, and is portable to various platforms at the same time.
I decided to push it a little bit further, tho.. I built from source and installed the app in my RPi4 8G running under Pop!_Os 21 for arm64, and it works flawlessly. Using the full KMS, i get around a stable 15fps after some seconds upon connecting.
I've been trying to push it to at least 30 though, so I tried fixing the VAAPI binding, creating a symlink from /usr/lib/aarch64.../dri to /usr/lib/dri, and that seemed to make it load a bit further, but still no luck. after some research, I found that VAAPI isn't fully compatible with 64-bit arm, from what I understand..
So my question is: Is it possible to implement v4l2 someway to enable it to Weylus? maybe under ffmpeg?
Here's the error output:
Thanks!
The text was updated successfully, but these errors were encountered: