Passthrough original PAT/PMT packets to the passthrough muxer

[linuxstb]

This is a work-in-progress and is not yet ready for pulling.

The passthrough muxer currently generates its own PAT/PMT packets and inserts these into the output stream at regular intervals. This PR replaces that code with code which includes the original PAT/PMT packets from the DVB broadcast, in their original positions.

A new DVR config option allows the user to specify if these are passed through unmodified, or if they are rewritten to exclude services (in the case of the PAT) or streams (in the case of the PMT) not included in the output stream. This option is for cases where the user needs to use players or processing tools which expect a more compliant PAT/PMT.

However, this rewriting is not 100% robust, and we may decide a better option is to just rebuild the PAT/PMT as we were doing in the past for the "rewrite" case.

NOTE: This code also currently only works when the adapter is running in Full Mode - in filtered mode, no TS filters are currently set for the PAT and PMT packets, so the resulting transport stream will be missing them.

One way to solve that is to change the filtered mode code to also use TS filters and perform section parsing in software. The advantage of this would be more consistent behaviour between raw and filtered mode, but it may use more PID filters, and (at least when using the passthrough muxer) would prevent tvheadend temporarily pausing a filter for the active PMT and reusing it for something else (e.g. EIT scanning).

[EricV]

Oops have been too quick. If its selectable why not. However, as explained this breaks TNT as no client is able to play correctly full mux and full mux by itself on USB tuners may well kill the machine for low end recorders with no PCI bus. And btw as you may know libav TS muxer as it is now leaks more than 1 GB per hour.

[john-tornblom]

TNT?

[linuxstb]

John, I'm guessing Eric is referring to French DVB-T which goes by the name TNT.

Eric, you seem to have misunderstood this PR. It doesn't record a full mux. It doesn't use the libav TS muxer. Did you look at the code?

[EricV]

Right TNT is the french word for DVB-T. You write in your description you "need a full Mux" and that you replace "PAT/PMT by original one in original position". I said then we need to rewrite PAT/PMT for DVB-T because the PAT/PMT contains multiple programs and cannot be used directly. Why not fix the actual code to support "more compliant PAT/PMT" using better PAT/PMT re-generator (from mmudvb or else?). Or is it what your code tries to do? And reading code in diff mode is not so easy ;-)

For the sake of history. Original code did generate only one PAT/PMT packet and this broke several STB hardware that missed it and were supposed to find other ones in the stream. We fixed that by adding periodic generation like other available code did and tested the new code on available hardware. Of course we could try fix them on the fly by letting the demux code transmit them will full program description (unfixed but no need for full mux then) and fix them before real ouput. If this is what your code does fine.

And for libav, this is just a reminder that current TS passthrough code when compiled with libav leaks and that it is more important to fix that

[linuxstb]

Eric,

tvheadend can use DVB adapters in one of two modes - "Full Mux mode" (where the entire mux is read from the card, and processed inside tvheadend) and "Filtered mode" (where TS or section filters are set in the driver/hardware and tvheadend only reads the data it needs). My comment related to the fact that my code currently only works in Full Mux mode, but just be clear, I also want this feature when tvheadend is running in Filtered mode, and is the main reason I said that this PR isn't ready to be accepted.

My experience in using DVB for about 12 years is that players don't care if the PAT contains references to programs which don't exist in the stream - everything I've ever tried just ignores any services listed in the PAT where the PMT can't be found (players such as mplayer, vlc, xbmc and the PS3, post-processing tools like pvastrumento or Project X). What player are you using that rejects such a recording?

But my code does attempt to correct this, with the "Rewrite PAT/PMT" option. My only concern is that are corner cases such as when the contents of the PAT or PMT change in the middle of a recording which are untested.

[EricV]

My comment are: Full mux mode for low end hardware just breaks on USB 2 adapters especially if you have up to 3 hd streams in a single mux like in france. I've been bitten on the PI and on my other low end recorder because default was "Full Mux" unconditionally for a while.

The comment about player where about full mux dump where most software and hardware players are totally confused and select a random stream, or refuse to play. Fixing PAT/PMT is better for hardware decoders. For software decoders anyway generating a single PAT/PMT at the beginning of the file is enough ...

Maybe fixing PAT is not needed but is it that complicated? The corner case you mention will break if you select a particular program id in filtering mode anyway? And in my experience, in filtered mode you still have PAT and PMT packets that contains the full mux description and not only the program you selected. The adapter filters the data from non selected program id but let PAT/PMT packet pass through unmodified (at least this is what I see). And this is different than the case you mention where the PAT contains all info and PMT packet can't be found.

[linuxstb]

Yes, I would expect some players to get confused with full mux dumps, but this PR isn't about full mux dumps, it's about improving the TS passthrough muxer when it's recording a single service.

I really don't understand what you are asking me. Is there something in my proposed code you don't agree with?

[EricV]

First : do you have actual devices where you have real problems?
Second: a solution relying on full mux dump is KO for many users and I would prefer to have a general solution
Third why not let PAT/PMT flow and rewrite them on the fly. This would work for non full mux dump AFAICT.

I will have no internet access for a week so do not be surprised if I'm silent for a week...

[linuxstb]

First: No.
Second: I'm happy you agree with me.
Third: Yes, that's what I've said needs to be done.

[EricV]

If First no then I'm a bit puzzled... Not that I do not agree that PAT/PMT packet generation could not be improved!

[adamsutton]

This is now included.