[Q] Is TSDuck able to modify the time interval between 2 consecutive PCR values in a VBR TS?

[moving-digital]

Hello, Thierry @lelegard

I'm having a VBR TS stream where the time interval between 2 consecutive PCR values is ~ 100-200ms, vs the standard stating the max should be = 40ms.

I was wondering if TSDuck is able to modify the time interval between 2 consecutive PCR values?

I am not using pcradjust plugin as it only works for CBR TS. Therefore, I was curious if there was a solution for VBR TS stream that I missed?

[lelegard]

Hi @moving-digital,

Modifying a PCR is technically easy. The real question is which value do you want to store in each individual PCR packet? Which PCR computation algorithm would you like to implement?

Each algorithm probably deserves a specific plugin.

Your question is a bit ambiguous. Modify the time interval between two PCR? The PCR is a program reference clock. It makes complete sense with synchronous CBR streams, typically broadcast, where MPEG-TS comes from. A VBR TS (full TS, not VBR service inside a CBR TS) is typically sent over asynchronous transports (UDP, TCP) with bursts - network packets - and non-uniform latencies. What does Program Clock Reference mean in that context? Unlike synchronous broadcast TS, the clock may change from relay to relay. And the interval between two PCRs at the departure from a point-to-point link may not be the same at the arrival to the next hop.

In short, I haven't the slightest idea on how to precisely define the right PCR value in a VBR TS.

Except maybe defining it as the "clock of the initial source", in which case you should not modify it. Unless the source is wrong, in which case you have to define how wrong the source is in order to specify how it should be adjusted.

That being said, if you want to modify one individual PCR value, you can use the craft plugin.

[moving-digital]

Thanks a lot @lelegard

Except maybe defining it as the "clock of the initial source", in which case you should not modify it.

I fully agree with you. I'm currently not modifying it (= the input is left intact) but Sencore CMA1820 is throwing the below ETR 101 290 errors:

Unless the source is wrong, in which case you have to define how wrong the source is in order to specify how it should be adjusted.

Indeed, that's what I have in mind, and was wondering if something could be done with TSDuck for my specific use-case.
As you can see from the screenshot, the time interval between 2 consecutives PCR values is not constant, probably due to the fact the input TS is VBR?

[lelegard]

So, your problem is not the PCR value, it is a burst & latency issue. If the source is TSDuck, you may need to do some fine tuning on all buffering parameters of the tsp command. RTFM for more details

TSDuck is a test & lab tool, not a production tool. It has not been designed to meet production constraints, here timing constraints.

[moving-digital]

Thanks @lelegard

The source is a UDP stream, that is fed into TSDuck.

I have tried playing with the "UDP Receive Buffer" many times, but unfortunately it does not seem to improve the situation.

By the way, in the FM, it only states the following:

--udp-buffer-size value
Specifies the UDP socket receive buffer size (socket option).

or

--buffer-size value
Specify the UDP socket receive buffer size (socket option).

However, unlike --buffer-size-mb whose the FM gives a lot of information and guidance as to how to use it, it unfortunately does not say much for --buffer-size value: We do not know if it this is in bytes, kbytes or mbytes, what is the default value, and what is the range of acceptable values we should be looking at when playing with that option.

Are you able to advise for the same?

[lelegard]

Have a look at options --max-flushed-packets, --max-input-packets, --max-output-packets.

[lars18th]

Hi @lelegard ,

Any way, what's the most simple (and reliable) method using TSDuck to convert a TS stream in VBR to CBR?
I'm asking this for multiple reasons, but for this particular case, after the conversion to CBR it will be simple to recompute the PCR values, right?

[moving-digital]

I second @lars18th

1. If the TS is a pure VBR TS (without any null packet), then definitely there is no way to recompute the PCR.
2. However, most of the time, when VBR is used, the services themselves are indeed VBR ES, but the TS itself is still CBR TS (containing all needed null packet / stuffing to output a constant bitrate).

So for use-case 2, I believe it should be possible to recompute the PCR.

[lars18th]

Hi @moving-digital ,

The ES data for the service could be CBR and/or VBR. For example, it's common that audio tracks are CBR and video tracks be VBR. And this is not a problem using MPEG-TS encapsulation because every ES packet has his timestamps. Futhermore, the entire TS has after the PCR timestamps. Therefore, after muxing the PCR is the only required information to maintain the correct flow timing. However, the problem is when the resulting TS is VBR and the PCR timestamps haven't sufficient repetition intervals to handle fast bitrate changes. So for this reason in DTV standards is common to use around 40ms for it (PCR). But for streaming or with non-professional tools it's very common to use 100-200ms values for PCR repetition values. And what's then the most reliable solution to "convert" these streams? From my experience the best is to convert it to CBR, and after recompute the PCR values. This will produce the best results without remuxing. And you need to know that the remuxing is the only definitive solution. I hope this will help you.

However for @lelegard : For a lot of time I'm searching for the most reliable method to convert VBR to CBR using TSDuck. More or less, this could be easy if the bitrate of the original stream never raises the CBR value. However, until now I haven't found any robust way. So please @lelegard could you propose (from your experienced perspective) the best method?
Thank you.

[moving-digital]

The ES data for the service could be CBR and/or VBR.

Yes, indeed, I agree.

But now, I'm further confused as to what is/is not supported by TSDuck:

1. I know that the ES data in CBR encapsulated with CBR TS is supported as this is the most standard/"basic" use-case.

2. But, how about ES data in VBR encapsulated with CBR TS (Stuffing with null packets)?
   Question mark here for this use-case.

3. To me, from experience, the most tricky one is when the use-case is ES data in VBR encapsulated with VBR TS (No stuffing at all)?
   That use-case may not be supported as it's extremely difficult to implement it.

[lelegard]

@lars18th, @moving-digital

You probably come from different backgrounds.

In the broadcast world, a TS is always globally a CBR MPTS with stuffing, simply because the modulation parameters logically produce a given fixed bitrate. Knowing that bitrate gives an external clock reference that you can use to do some transformation.

In the streaming world, a TS is almost always a VBR SPTS. When processed offline or in an asynchronous parallel environment such as tsp, there is no external clock reference which you can rely on.

[lars18th]

Hi @lelegard ,

Yes, you know it very well: in broadcasting TS are CBR (MPTS or SPTS). But not every time, because when you consider the pid filtering, then the CBR is transformed in a VBR TS if the content is not real CBR.

So in this particular use case, I repeat my request: What's from your experience the best method to convert a VBR stream to one CBR using the TSDuck tools only?

[moving-digital]

Yes, you know it very well: in broadcasting TS are CBR (MPTS or SPTS). But not every time, because when you consider the pid filtering, then the CBR is transformed in a VBR TS if the content is not real CBR.

I can't agree more, and this is the most extreme use-case unfortunately.

Some Professional IRD (that I won't name) are unfortunately doing this quite frequently:

• the input is a CBR MPTS containing multiple StatMuxed VBR services
• the operator cherry picks 1 single Service and outputs it, but instead of the TS to be CBR containing the cherry picked VBR service, this PIRD won't bother itself and output a dirty VBR TS

Edit: in that case, I also believe that to make this VBR TS a proper CBR TS at TSDuck level, the best way is to start with an initial bitrate > than the maximum bitrate that the service will ever reach, with the condition that the user must know this bitrate in advance in order to set it manually in the tsp input bitrate setting when launching the command.

[lelegard]

Yes, you know it very well: in broadcasting TS are CBR (MPTS or SPTS). But not every time, because when you consider the pid filtering, then the CBR is transformed in a VBR TS if the content is not real CBR.

In that situation, the best way is to never use VBR as intermediate state. With TSDuck, PID filtering and other plugins which remove packets usually have an option --stuffing to clear filtered packets as null packets instead of deleting them. In that case, the TS globally keeps its CBR. As a final stage, the plugin reduce can reliably remove null packets to transform an input CBR into a lower output CBR.

Doing variable null packet insertion in a VBR TS requires to evaluate instant bitrate. There is currently no way to do so in tsp. Implementing this is certainly possible but it requires some investment and I no longer have time to invest on this kind of development. At some point, other developers will have to take over.

[lars18th]

Hi @lelegard ,

I'm aware about the "limitation" (in fact not a real limitation, but a good compromise to be optimal) inside the TSDuck chain processing. The toolkit can't introduce non-deterministic new packets, only remove them (deterministic packets are for example from --input-stuffing). And therefore the use of the --stuffing parameter with a lot of plugins is the answer to mainting the bitrate. So every time we can, it's preferable to work with CBR TS streams. Initially it might appear to be inefficient because of the number of null packets. However, it simplifies a lot the processing with the only cost of a higher bitrate. And if your platform could handle this bitrate, then the processing will be robust.

However, in some circunstances the SOURCE stream is not CBR. You can imagine a lot of causes. Only to list some of them:

• The output is from a 3rd party tool that generates a VBR TS.
• The source is filtered from another processing that removes the padding stuff.
• etc.

More or less all cases are from outside the TSDuck processing. And for this reason I ask about the best and simple method to transform these sources to a CBR TS. The idea then to use it with TSDuck to minimize troubles. My last tests have focused on generate an empty CBR stream with TSDuck and add to it then the VBR stuff with the merge plugin. However, I found that this method is not really robust. You know something more reliable?