Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get frames without setting the format #1494

Closed
LairCortex opened this issue Jun 24, 2024 · 5 comments
Closed

Get frames without setting the format #1494

LairCortex opened this issue Jun 24, 2024 · 5 comments

Comments

@LairCortex
Copy link

Good afternoon!
Our stream doesn't have sprop parameter attached so we can't pass processH264FmtpAttribute RtspMediaTrack check.
Found a similar problem and solution in google/ExoPlayer#10971 (comment) , but the video failed without setting default parameters instead of checking processH264FmtpAttribute - formatBuilder.setWidth(1920).setHeight(1080); We can get parameters from the first frame, but we need to set default format, how can we get frames without setting the format

@microkatz
Copy link
Contributor

Hi @LairCortex,

The RtspMediaTrack method still needs to provide a Format, just not with the sps parameters provided by the processH264FmtpAttribute method. You would then need to update the format information in the RtpH264Reader class with those values you get from that first frame. It is difficult to provide advice without seeing how you are customizing the code. Would you be able to provide an example repo showcasing the changes you are making?

@LairCortex
Copy link
Author

Hi @microkatz,
We have done the experiment and realise that we can catch the first packet in RtpExtractor.read it turns out we just need to update the Format Can you tell us how to do it correctly?

@microkatz
Copy link
Contributor

@LairCortex

That's great! It would be in the processSingleNalUnitPacket(or other processing Nal unit method in RtpH264Reader). After receiving the sps data, you would need to call trackOutput.format(Format format) with a Format built from the sps data. Similar to the code below:

List<byte[]> initializationData = new ArrayList<>();
        initializationData.add(Arrays.copyOf(sps.nalData, sps.nalLength));
        initializationData.add(Arrays.copyOf(pps.nalData, pps.nalLength));
        NalUnitUtil.SpsData spsData = NalUnitUtil.parseSpsNalUnit(sps.nalData, 3, sps.nalLength);
        NalUnitUtil.PpsData ppsData = NalUnitUtil.parsePpsNalUnit(pps.nalData, 3, pps.nalLength);
        String codecs =
            CodecSpecificDataUtil.buildAvcCodecString(
                spsData.profileIdc,
                spsData.constraintsFlagsAndReservedZero2Bits,
                spsData.levelIdc);
        trackOutput.format(
            new Format.Builder(payloadFormat.format)
                .setSampleMimeType(MimeTypes.VIDEO_H264)
                .setCodecs(codecs)
                .setWidth(spsData.width)
                .setHeight(spsData.height)
                .setColorInfo(
                    new ColorInfo.Builder()
                        .setColorSpace(spsData.colorSpace)
                        .setColorRange(spsData.colorRange)
                        .setColorTransfer(spsData.colorTransfer)
                        .setLumaBitdepth(spsData.bitDepthLumaMinus8 + 8)
                        .setChromaBitdepth(spsData.bitDepthChromaMinus8 + 8)
                        .build())
                .setPixelWidthHeightRatio(spsData.pixelWidthHeightRatio)
                .setInitializationData(initializationData)
                .setMaxNumReorderSamples(spsData.maxNumReorderFrames)
                .build());

Do you have an sample RTSP server that we could use to test this as well?

@google-oss-bot
Copy link
Collaborator

Hey @LairCortex. We need more information to resolve this issue but there hasn't been an update in 14 weekdays. I'm marking the issue as stale and if there are no new updates in the next 7 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

@google-oss-bot
Copy link
Collaborator

Since there haven't been any recent updates here, I am going to close this issue.

@LairCortex if you're still experiencing this problem and want to continue the discussion just leave a comment here and we are happy to re-open this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants