Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: send encoded h264 packet to webrtc #11

Closed
TzuHuanTai opened this issue Nov 14, 2022 · 1 comment · Fixed by #12
Closed

perf: send encoded h264 packet to webrtc #11

TzuHuanTai opened this issue Nov 14, 2022 · 1 comment · Fixed by #12

Comments

@TzuHuanTai
Copy link
Owner

TzuHuanTai commented Nov 14, 2022

Capturing the h264 packets directly from the camera module, written into mp4 can reduce by 80% space compared to mjpeg.

Use webrtc::VideoEncoderFactory to produce some custom "encoder"
The encoder subclass passes encoded frames (e.g. H264) to the webrtc::EncodedImage::OnEncodedImage callback.
The encoder doesn't really have to encode anything, it just delivers pre-encoded frames.

@TzuHuanTai TzuHuanTai linked a pull request Nov 15, 2022 that will close this issue
@TzuHuanTai
Copy link
Owner Author

TzuHuanTai commented Nov 24, 2022

obviously, directly capturing the h264 format from cameras in v4l2 is not a good idea in the webrtc framework.

  1. If captured h264 in AdaptedVideoTrackSource and then sent frames to VideoEncoder, webrtc drops frames sometimes, it will make the NAL unit not continuous. So the decoding order is wrong making the broken real-time video on the client side.
  2. If captured h264 in VideoEncoder. Although it will not interrupt the sequence, it would not be triggered when webrtc drops frames. We can not drop the delayed buffer because of keeping it in sequence, therefore, the delay will increase over time.

Try v4l2m2m encode raw frames could be a better idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant