EIT rewrite is missing some packets

[doub]

tl;dr: EIT rewriting cannot be done easily at the TS packet level.

I've been trying to decode EIT sections from a partial TS streamed by MuMuDVB, and I frequently encounter invalid sections.

I've looked at the code, and it appears that the EIT rewriting code (eit_sort_new_packet) makes two assumptions that are not true in my situation (DVB-T in the UK).

First it assumes that there can be at most only two (partial) sections in a TS packet, ie. one before the pointer_field and one after. It's only checking for the EIT header once at the get_ts_begin offset. But I have TS packets that have several EIT headers (ie. 3 or more section are spanned on the TS packet).

Second, it assumes that if there is a EIT header present, then whatever is before it belongs to the same service. In other words if the TS packet has an EIT header for service X, it won't send the TS packet to another channel. But in my case I have TS packets with an EIT header for service X, but before the header there are some bytes belonging to another EIT section for service Y.

So all in all, the EIT rewriting needs much more work. Packets should be split, with the beginning of a packet sent to whatever channel the previous packet ended with. The end of such a packet should be re-stuffed before being sent to the first channel (and the payload_unit_start_indicator flag cleared). Then each EIT section (and not only the first one) should be independently forwarded to the right channel, which for the second and subsequent EIT headers implies rewriting the pointer_field of the TS packet.

Maybe a simpler solution would be to do the rewriting at the section level, ie. parse the TS packets and drop them, and completely recreate them once a section is completed (which may happen several times during the processing of a single packet).

[braice]

I remember this bug now. It appears only with uk providers... Which decided
to pack lot of eit together

Could you provide me a stream dump so I can see what I could do?

I agree there is two solutions, I have to think on which one is simpler

Thanks
Le 28 mai 2013 23:07, "Jérôme Vuarand" notifications@github.com a écrit :

tl;dr: EIT rewriting cannot be done easily at the TS packet level.

I've been trying to decode EIT sections from a partial TS streamed by
MuMuDVB, and I frequently encounter invalid sections.

I've looked at the code, and it appears that the EIT rewriting code
(eit_sort_new_packet) makes two assumptions that are not true in my
situation (DVB-T in the UK).

First it assumes that there can be at most only two (partial) sections in
a TS packet, ie. one before the pointer_field and one after. It's only
checking for the EIT header once at the get_ts_begin offset. But I have TS
packets that have several EIT headers (ie. 3 or more section are spanned on
the TS packet).

Second, it assumes that if there is a EIT header present, then whatever is
before it belongs to the same service. In other words if the TS packet has
an EIT header for service X, it won't send the TS packet to another
channel. But in my case I have TS packets with an EIT header for service X,
but before the header there are some bytes belonging to another EIT section
for service Y.

So all in all, the EIT rewriting needs much more work. Packets should be
split, with the beginning of a packet sent to whatever channel the previous
packet ended with. The end of such a packet should be re-stuffed before
being sent to the first channel (and the payload_unit_start_indicator flag
cleared). Then each EIT section (and not only the first one) should be
independently forwarded to the right channel, which for the second and
subsequent EIT headers implies rewriting the pointer_field of the TS packet.

Maybe a simpler solution would be to do the rewriting at the section
level, ie. parse the TS packets and drop them, and completely recreate them
once a section is completed (which may happen several times during the
processing of a single packet).

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/24
.

[doub]

Here is a 40MB extract that show the problem: http://piratery.net/temp/aligned1.ts MuMuDVB is streaming that extract: http://piratery.net/temp/aligned2.ts which has a packet with 3 sections, and invalid sections.

[braice]

Thank you; I'll have a look

For the moment EIT is just sorting, I think it will be better to turn it in demuxing and rewriting as SDT, it will give us better control on what is hapenning

[braice]

I am writing the EIT/sorting rewriting code, it's a bit harder than before because EIT sections easily go over one TS packets. But should be done without much too much difficulties, just a matter of time.

[braice]

Just an update on this issue, the storing and the sorting of the complete EIT for each service works well, now I need to write the sending part

[braice]

I have some code, which needs cleanup but ready for testing. Tell me if you are interested.

Best

[braice]

Hello,

It's now done at the TS level, demuxing the sections etc...

I close the issue, reopen it if you see bugs on this feature