OvenMediaEngine, when used in conjunction with NGINX RTMP Push or Nimble Streamer RTMP Republisher, is unable to determine the parameters of the incoming stream

[mehdi-qencode]

There was a problem when using OvenMediaEngine in conjunction with NGINX RTMP, or Nimble Streamer. The need for this solution is dictated by the task of organizing parallel transcoding of live streams to create a solution taking into account the requirements of fault tolerance.

An approximate solution scheme is given below

According to this scheme, the NGINX RTMP instance or Nimble Streamer instance receives the incoming stream and transmits it to two separate OvenMediaEngine instances for transcoding. OBS and Vmix were used for RTMP pushing to the NGINX or Nimble. When using this scheme, OvenMediaEngine is unable to determine the parameters (resolution, frame rate, bitrate) of the incoming stream, as a result of which all streams (OvenMediaEngine is used for ABR transcoding) are encoded with the same resolution.

We investigated network parameters between instances of NGINX RTMP / Nimble Streamer and OvenMediaEngine, parameters of incoming and outgoing packets to and from NGINX RTMP and NImble Streamer using tcpdump, but no any networking issues were found. The research was conducted with OvenMediaEngine versions 0.11.3 and 0.12.1. Both versions behave the same, the problem occurs in both cases. Also, for the test we used Nimble Streamer Transcoder as a replacement for OvenMediaEngine, and in this case the circuit works flawlessly.

[getroot]

OME encodes as set, regardless of the parameters of the input stream. Therefore, it is normal behavior for all streams to be encoded with the same set resolution. Or am I misunderstanding? Please upload your Server.xml.

[mehdi-qencode]

We are trying to test OvenMediaEngine as a ABR transcoder, which mean there is at least 4 different resolution need to be encoded. My Server.xml file is here https://drive.google.com/file/d/13v3131U3EwiaJN0pKU9YLqg6xMfRqcpj/view?usp=sharing

[getroot]

If you input "app/stream" stream into OME, OME will create app/stream_1080p and app/stream_720p according to your settings. Do app/stream_1080p and app/stream_720p both have the same resolution?

If you want to play this with webrtc you can use a url like this:

ws://host:3333/app/stream_1080p
ws://host:3333/app/stream_720p
ws://host:3333/app/stream_360p
ws://host:3333/app/stream_240p

[mehdi-qencode]

Yes, in this case all streams has a same resolution which is 1920x1080.

[getroot]

If so, there seems to be a bug there. Thanks for the reporting.
The developer of that part will look into that issue.

[getroot]

I would like to ask you one question to quickly analyze the problem.

The output encoding profile is independent of the parameters of the input stream. Inputting the stream with OBS should give the same result.

You mentioned that you relayed the stream via nginx/nimble. Does this mean there are no issues when using OBS to send directly to OME?

[mehdi-qencode]

Definitely yes. When we use OBS or Vmix with direct push to OME, all working fine. The issue occurs only when we want to relay or republish using NGINX RTMP or Nimble Streamer.

[getroot]

Thanks for the detailed information. We'll build that environment and analyze it.

[mehdi-qencode]

Thanks to you. If you need my assistance for making environment, just let me know.

[Keukhan]

@mehdi-qencode

Hi.

Can you set the MediaRouter tag to debug in the Logger.xml file as shown below and send the collected logs?

<Tag name="MediaRouter" level="debug" />

I am interested in the timing of Incoming Stream from Nimble or NginX Relay. If you send the log, it will be very helpful in analyzing the cause.

Thanks.

To output the debug log, you need to recompile and run OME in debug mode. :)

[mehdi-qencode]

About recompiling, no problem, I'll do it. But what I want exactly say is in those part of logs that below:

`
[2021-06-28 08:55:32.385] I [SckPoolRTMP:212030] RTMPProvider | rtmp_provider.cpp:117 | A RTMP client has connected from <ClientSocket: 0x7f16b8000c50, #77, state: Connected, TCP, 35.199.56.45:51234>
[2021-06-28 08:55:33.126] W [SckPoolRTMP:212030] RTMPProvider | rtmp_stream.cpp:827 | Unknown Type - Type(4)
[2021-06-28 08:55:33.127] W [SckPoolRTMP:212030] RTMPProvider | rtmp_stream.cpp:970 | Unknown Amf0DataMessage - Message(onMetaData)
[2021-06-28 08:55:33.482] I [SckPoolRTMP:212030] MediaRouter | mediarouter_application.cpp:243 | Trying to create a stream: [#default#qlive/obs(77)]
[2021-06-28 08:55:33.482] I [SckPoolRTMP:212030] MediaRouter | mediarouter_application.cpp:244 |
[Stream Info]
id(77), output(obs), SourceType(Rtmp), Created Time (Mon Jun 28 08:55:32 2021)

Video Track #0: Bypass(false) Bitrate(0b) codec(1, H264) resolution(0x0) framerate(0.00fps) timebase(1/1000)
Audio Track #1: Bypass(false) Bitrate(0b) codec(6, AAC) samplerate(0) format(s16, 16) channel(unknown, 0) timebase(1/1000)

[2021-06-28 08:55:33.482] I [SckPoolRTMP:212030] MediaRouter | mediarouter_stream.cpp:57 | Trying to create media route stream: name(obs) id(77)
[2021-06-28 08:55:33.482] I [SckPoolRTMP:212030] Monitor | application_metrics.cpp:56 | Create StreamMetrics(obs) for monitoring
`

As you see, there is resolution(0x0) and framerate(0.00fps) issues. And this occurs when we want to use RTMP Relay.

[getroot]

The message output like that is just the metadata of RTMP (and it's not actually used, only for log output), and we extract the actual resolution from sps of H264.
It's not the cause of the problem. Could you please upload the whole log?

[mehdi-qencode]

I need recompile OME in debug mode. After, I can send you whole log.

My question is about the case, when we want to detect input resolution and define output resolutions based on input resolution. For example, if we want to make dynamic output resolution such as 1/1, 1/2, 2/3 and etc. resolutions sets.

[getroot]

Recognizing the input resolution as RTMP metadata is dangerous. Some encoders do not send this information. So we use this information only for log output.

The most accurate way is to parse the codec bitstream. See the codes below as examples.

https://github.com/AirenSoft/OvenMediaEngine/blob/master/src/projects/mediarouter/mediarouter_stream.cpp#L153
https://github.com/AirenSoft/OvenMediaEngine/blob/master/src/projects/mediarouter/mediarouter_stream.cpp#L352
https://github.com/AirenSoft/OvenMediaEngine/blob/master/src/projects/mediarouter/mediarouter_stream.cpp#L590
https://github.com/AirenSoft/OvenMediaEngine/blob/master/src/projects/mediarouter/mediarouter_stream.cpp#L590

[mehdi-qencode]

When is this sps parsing run? I have written the code that make dynamic calculation of output resolutions and framerates sets that works at the transcoder initialization point. But at this point I can't get resolution and framerate of the input stream.

[getroot]

Is it because of the code you added that all outputs (stream_1080p, stream_720p, stream_360p) have 1080p? Doesn't this happen with existing code?

OnStreamPrepared is called when a stream has been parsed. If you want to develop a dynamic profile, you can start here.

OvenMediaEngine/src/projects/transcoder/transcoder_application.cpp

Line 99 in ad69081

bool TranscodeApplication::OnStreamPrepared(const std::shared_ptr<info::Stream> &stream)

Since OME is now creating the transcoder stream in OnStreamCreated, quite a few things need to change.

[mehdi-qencode]

The code running perfectly when I push stream from OBS to OME directly.

I'll try to develop code at those point. Thank you for help.

[getroot]

There are many softwares, such as nginx, that either don't send RTMP codec metadata or send it in a different format. Because it is not a standard specification. So using that information as real data creates a lot of problems. Of course, since we use OBS often, we are parsing all the information OBS sends.

Framerates cannot be extracted from codec bitstreams such as SPS. Because FPS can be variable like bitrates. You should keep this in mind when developing the feature.

If you run into a problem with the code you added, please let us know. We will be able to get to the cause of the problem faster.

[mehdi-qencode]

Thank you. I will.

[getroot]

This issue has been closed since it has been inactive for quite some time. If you want to continue discussing this issue, please feel free to reopen it.