Ignore SPS that contains scaling lists

[juvasquezg]

Hi,

I have a source that delivers SPS with scaling list and I get a problem in

line 85 - rtspvideocapturer.cpp

rtc::Optional<webrtc::SpsParser::SpsState> sps = webrtc::SpsParser::ParseSps(buffer+sizeof(marker)+webrtc::H264::kNaluTypeSize, size-sizeof(marker)-webrtc::H264::kNaluTypeSize);

I get:

SPS contains scaling lists, which are unsupported.
cannot parse sps

My sprop-parameter-sets is

sprop-parameter-sets=Z2QAKK2ECSZuIzSQgSTNxGaSECSZuIzSQgSTNxGaSECSZuIzSQgSTNxGaSEFTS69fX5P6/J9eutVCCppdevr8n9fk+vXWqtAUBf8uAqkAAADAAQAAAMCWYEAD6AABGUb3vheEQjU,aO48sA==

The problem is in the file webrtc/common_video/h264/sps_parser.cc

      // We don't support reading the sequence scaling list, and we don't really
      // see/use them in practice, so we'll just reject the full sps if we see
      // any provided.
      if (seq_scaling_list_present_flags > 0) {
        LOG(LS_WARNING) << "SPS contains scaling lists, which are unsupported.";
        return OptionalSps();
      }

I want to skip sps with scaling list but I don't a master in H.264. Maybe you could help me.

Before going to the stackoverflow I want to ask you :)

Thank you so much.

[mpromonet]

Hi Julian,

An alternative could be to use h264bitstream if it could parse the SPS of your source.

Regards,

Michel.

[juvasquezg]

Hi :)

Thank you,

I saw http://www.live555.com/liveMedia/faq.html#testRTSPClient-how-to-decode-data

And with live555 say

''
If you are receiving H.264 video data, there is one more thing that you have to do before you start feeding frames to your decoder. H.264 streams have out-of-band configuration information ("SPS" and "PPS" NAL units) that you may need to feed to the decoder to initialize it. To get this information, call "MediaSubsession::fmtp_spropparametersets()" (on the video 'subsession' object). This will give you a (ASCII) character string. You can then pass this to "parseSPropParameterSets()" (defined in the file "include/H264VideoRTPSource.hh"), to generate binary NAL units for your decoder.
'''

Fuctions like

MediaSubsession::fmtp_spropparametersets()
parseSPropParameterSets()

Parse before, and then send to onData callback.

Maybe work? What do you think?

Thank you so much.

[mpromonet]

Hi,
This method just split the strings with coma as separator and decode bas64, but it doesnot decode the SPS in order to get width & height.
Then I think this wil not work.
I guess you have 2 options:

• wait WebRTC support these SPS or modify WebRTC
• use h264bitstream if it decodes these SPS (there is an example of doing this in v4l2writer.cpp )

Regards,
Michel.

[juvasquezg]

Hi @mpromonet

Thank you so much,

I get an error so I sent a issue to h264bitstream #19

Regards,
Julián.

[juvasquezg]

Hi @mpromonet

The problem was when I built with h264bitstream, the commit 62261fcc1715174216cb07f0c0c38173f4cf48bb

not allow build like v4l2rtspclient

h264bitstream/h264_stream.c h264bitstream/h264_nal.c h264bitstream/h264_sei.c ....

I built h264bitstream and in the Makefile I added

LDFLAGS += h264bitstream/.libs/libh264bitstream.a

I view the video, but now I always get

 NOTIFY failed

from

void RTSPConnection::SessionSink::afterGettingFrame(unsigned frameSize, unsigned numTruncatedBytes, struct timeval presentationTime, unsigned durationInMicrose\
conds)                                                                                                                                                          
{                                                                                                                                                               
    if (numTruncatedBytes != 0)                                                                                                                                 
    {                                                                                                                                                           
        delete [] m_buffer;                                                                                                                                     
        envir() << "buffer too small " << (int)m_bufferSize << " allocate bigger one\n";                                                                        
        allocate(m_bufferSize*2);                                                                                                                               
    }                                                                                                                                                           
    else if (m_callback)                                                                                                                                        
    {                                                                                                                                                           
        if (!m_callback->onData(this->name(), m_buffer, frameSize+m_markerSize, presentationTime))                                                              
        {                                                                                                                                                       
            envir() << "NOTIFY failed\n";                                                                                                                       
        }                                                                                                                                                       
    }                                                                                                                                                           
    this->continuePlaying();                                                                                                                                    
}                                                                                                                                                               

onData

    webrtc::EncodedImage input_image(buffer, size, 2*size);
    int res = m_decoder->Decode(input_image, false, NULL);

    return (res == 0);

This is a problem? or it is Normal?

What do you think?

Thank you so much!!

[mpromonet]

Hi juvasquezg,

In this commit 3927b59, I improved usage of SPS/PPS, ffmpeg that is under the hood to decode don't like to receive SPS, PPS alone, it expected to get it with an I frame.

If I understood well, h264bitstream is not able to decode your special SPS ?

Best Regards,
Michel.

[juvasquezg]

Hi @mpromonet

Thank you,

No. h264bitstream decodes my special SPS, the problem was when compile it, but I resolved it when I added

LDFLAGS += h264bitstream/.libs/libh264bitstream.a in Makefile.

I will try new commit.

Thank you!!

[juvasquezg]

Hi @mpromonet

With h264bitstream I decode my special SPS but now when I run the program, I get

#
# Fatal error in ../../webrtc/modules/video_coding/codecs/h264/h264_decoder_impl.cc, line 305
# last system error: 0
# Check failed: input_image._size >= input_image._length + EncodedImage::GetBufferPaddingBytes(kVideoCodecH264) (12 vs. 14)
#
#

I comment

else {
	LOG(INFO) << "===========================" << nalu_type;
	//webrtc::EncodedImage input_image(buffer, size, 2*size);
	//res = m_decoder->Decode(input_image, false, NULL);
}

But I get

NOTIFY failed

Thank you so much!!

[mpromonet]

Hi,

As it is reported in the assert the encoded image size should be : size + webrtc::EncodedImage::GetBufferPaddingBytes(webrtc::VideoCodecType::kVideoCodecH264)
When I added this problems, I just double the size, but this was a lazy fix.
This commit ae80bbf should helps you.

Best Regards,
Michel.