Decode H264 frames

[mihail-ivanov]

Hi,

Is there some solution, which decodes the received frames and then use them in GoCV?
For example in client-read.go to decode the received buffer and load it in img := gocv.NewMat():
https://github.com/aler9/gortsplib/blob/2c7dbe200908f3484fd43ed937fd609d2b2b912e/examples/client-read.go#L24

Thanks,
Mihail

[aler9]

Hello, i never tried to decode H264 (is it H264?) from RTSP in Go, but i think it can be done - i have to warn you, the procedure will require a lot of trial and error, and is the following:

1. start a client, start reading
2. decode RTP/H264 frames into H264 frames
3. decode H264 frames

actually, i don't know how to perform the 3rd point in Go. It probably requires interfacing with x264 or libav.

a draft could be (see comment below)

[mihail-ivanov]

Thank you for your answer. I am new in Go (but I have experience with Python and C/C++), so I will try to create an example based on your guidelines and if everything is fine, I will make a pull request.

[Sean-Der]

@mihail-ivanov I did something similar with gocv-receive it takes the incoming RTP, packages it into a ivf file and then sends to ffmpeg using stdin/stdout!

You could also send the RTP straight in (over the network) and then read from ffmpeg stdout into gocv. Something like rtp-forwarder

[aler9]

An example on how to obtain raw H264 frames from RTSP has been added here:

https://github.com/aler9/gortsplib/blob/main/examples/client-read-h264/main.go

But i can't go any further due to possible patent infringement. Anyway (for anyone interested) it's enough to find out how to use Gstreamer with Go to proceed and decode the frames.

[mihail-ivanov]

Thank you, @aler9. I think that would help me a lot.

[mderit]

@aler9, I was check your example client-read-h264/main.go
and I found an error, when i check log on err variable,
are that have a problem with start bit?

first NALU does not contain the start bit

nalus, _, err := dec.Decode(buf)
			if err != nil {
				fmt.Println(err) 
				return
			}

[aler9]

@mderit replace return with continue

[aler9]

now there's a working example that shows how to decode raw frames with gortsplib:
https://github.com/aler9/gortsplib/blob/main/examples/client-read-h264-decode/main.go

[rpatidar]

the link is moved here https://github.com/aler9/gortsplib/blob/main/examples/client-read-format-h264/main.go