Non-monotonically increasing PTS, frame index # #312
Comments
|
I think I understand what is happening here. The RTP packets are being sent in monotonically increasing DTS order, the order in which frames are decoded. And this differs from PTS order when there're B-frames present. When I disable B-frames in the ffmpeg command, the PTS are in the right order: Probably when we have a real live stream, B-frames aren't used anyways, so this shouldn't be a problem. The second question persists though: how do I get the frame number? Is there a proper way to do this via the API? Thanks! |
|
Looks like we can store frame index # in userdata field in the session: type MyUserData struct {
Index int64
Path string
}
//.....................................................................................................................................................................................
func (sh *serverHandler) OnSetup(ctx *gortsplib.ServerHandlerOnSetupCtx) (*base.Response, *gortsplib.ServerStream, error) {
log.Printf("setup request %s", ctx.Path)
ctx.Session.SetUserData(MyUserData{Path: ctx.Path, Index: 0})
return &base.Response{
StatusCode: base.StatusOK,
}, nil, nil
}
//.....................................................................................................................................................................................
func (sh *serverHandler) OnRecord(ctx *gortsplib.ServerHandlerOnRecordCtx) (*base.Response, error) {
log.Printf("record request %s", ctx.Path)
userData := ctx.Session.UserData().(MyUserData)
// called when receiving an RTP packet
ctx.Session.OnPacketRTP(sh.media, sh.format, func(pkt *rtp.Packet) {
nalus, pts, err := sh.rtpDec.Decode(pkt)
if err != nil {
return
}
// decode H264 NALUs into image.Image
for _, nalu := range nalus {
// convert NALUs into RGBA frames
img, err := sh.h264RawDec.decode(nalu)
if err != nil {
panic(err)
}
// wait for a frame
if img == nil {
continue
}
log.Printf("Image session_path: %s, index: %d, pts: %s", userData.Path, userData.Index, pts)
userData.Index = userData.Index + 1
// convert frame to JPEG and save to file
//err = saveToFile(img)
//if err != nil {
// panic(err)
//}
//saveCount++
//if saveCount == 5 {
// log.Printf("saved 5 images, exiting")
// os.Exit(1)
//}
}
})
return &base.Response{
StatusCode: base.StatusOK,
}, nil
}I just don't know if it's the best way 😅 |
alllexx88
changed the title
|
Hello, video streams have two timestamps: PTS (presentation time stamp) and DTS (decode time stamp). PTS can be unordered when there are B-frames, while DTS is ordered. There's an utility that allows to convert PTS into DTS that is |
|
This issue is being locked automatically because it has been closed for more than 6 months. |
Hello! I made this code mix from
server-h264-save-to-diskandclient-read-format-h264-convert-to-jpegto implement an RTSP server that converts input h264 stream into images (comments not updated):If I uncomment the
err = saveToFile(img)part inOnRecord(), I do get frames saved as pictures. What confuses me is that pts don't seem to grow monotonically, I get something like this:I'm sending RTSP stream with
ffmpeg:Is this behavior normal? I need to process frames in the correct order to apply a Computer Vision algorithm.
As a side note, is there a way to get the frame number? Or I should compute it from pts and fps?
Thanks!
The text was updated successfully, but these errors were encountered: