Add muxer #16

barbashov · 2021-03-02T12:17:26Z

I've created a MPEG-TS muxer based on your demuxer library, as requested in #10 .

I must say here that I had no intention for muxer to support everything demuxer supports. What I'm trying to do is to solve my tasks with the library (I guess that's how open source works anyway). So it's not full-featured, but it still helps a lot.

TODOs and limitations:

Most important: It doesn't really support PSI sections yet. Though it knows how to write tables, each table must fit in one TS packet. Mostly the limitation is there because I don't see the easy and clean way to do it now. And I don't really need it right now.
Only PAT and PMT tables are supported. No SDT, NIT, EIT, etc. Though it won't be hard to add the support if needed.
Muxer doesn't support more than one program (it's not hard to do). Number of elementary streams is not restricted.

Additionally while working on the muxer I added some little things here and there, most notably:

autoDetectPacketSize now knows how to work with bufio.Reader correctly
~~packetPool.add got an option flushOnPIDChange to overcome behaviour when one-packet tables doesn't get returned to demuxer at once.~~
I've changed CRC32 API slightly to be able to update it incrementally.

Sorry for the big pull request, but I don't see a way to do it in smaller parts :)

…ting

barbashov · 2021-03-06T02:49:14Z

I've removed flushOnPIDChange since it doesn't seem to be necessary - I think I overdebugged something a couple days ago.

FYI you forgot to remove the comment here

Thanks! Cleaned that up

asticode

A few remarks regarding last changes.

data_psi.go

asticode

In all honesty, this PR is really awesome.

crc32.go

data_pes.go

data_pat.go

data_pes.go

data_psi.go

asticode · 2021-03-22T09:35:46Z

muxer.go

+	return nil
+}
+
+func (m *Muxer) WritePayload(pid uint16, af *PacketAdaptationField, ph *PESHeader, payload []byte) (int, error) {


I think the public API should be func (m *Muxer) WriteData(af *PacketAdaptationField, d *Data) (int, error) instead. I don't mind implementing it for PES only for now. *Data would be sufficient to pass pid, ph and payload

I gotta remember why I haven't done that in the first place, will try to remember in next couple of days and will get back to you :)

Aha! I remember now why I haven't done it that way.

Data has FirstPacket with PID in its header. At the same time Data has PID as a separate member. While it's perfectly okay and handy for demuxer API user, that is confusing for muxer API user

FirstPacket has a bunch of fields which have no effect setting in WriteData case - I will rewrite CC, AF/payload indicators, etc. That could confuse API user as well.

So the main point here is: I've designed WritePayload to require arguments that really make sense here and are really used.

I get your point as well - it's not consistent with demuxer currently. And it's not especially pretty :) Though I don't really know a way to make it perfect now - seems like we're choosing between two tradeoffs (well, a position I constantly find myself in :))

Also I think muxer logic is a bit different by its nature.

Demuxer usage is like "tell me everything you can parsing a TS stream".

While muxer usage is more like "okay, I got a bunch of elementary streams, now let's write payloads. And handle all the low level stuff like PAT/PMT for me, I don't want to be bothered by that".

Love your explanation and I get your point about FirstPacket, that would indeed be confusing.

What about renaming the current Data to DemuxerData and creating :

type MuxerData struct { AdaptationField *PacketAdaptationField PES *PESData PID uint16 }

I think that would be a good compromise.

Let me think about it a little bit.

Well I've done it that way and I like it.

Though don't rush to resolve this thread just yet. I'd like to do some additional testing today to make sure I haven't broken anything.

I've tested new WriteData with my project I'm working on. All is ok, no issues.

Nice! I'll merge the PR then

muxer.go

packet.go

packet_buffer.go

barbashov · 2021-03-23T09:58:57Z

@asticode can you please resolve threads I've already addressed? I'm starting to get confused as there's a lot of them :)

asticode · 2021-03-23T10:11:47Z

@asticode can you please resolve threads I've already addressed? I'm starting to get confused as there's a lot of them :)

Done.

Be careful as there are open threads that Github hides in the middle. You have to load them manually.

barbashov · 2021-03-23T10:23:55Z

@asticode can you please resolve threads I've already addressed? I'm starting to get confused as there's a lot of them :)

Done.

Be careful as there are open threads that Github hides in the middle. You have to load them manually.

Thanks.

Yeap, I'd found that hidden threads yesterday. Quite controversial UX decision if you ask me :)

asticode · 2021-03-24T08:22:49Z

I've resolved threads that are now fixed. Only 3 open threads remaining!

asticode · 2021-03-24T14:45:18Z

As I said previously, it was a very nice PR ❤️

Thanks for sticking with it.

I've created a v1.7.0 tag

Cheers

barbashov · 2021-03-24T14:52:56Z

Whoa. Finally after two months, 4k lines of code and 57 commits it happened.

Will open a bottle of wine for that on Friday :)

Thanks for the throughout review and nice feedback!

barbashov · 2021-03-28T00:42:00Z

Well I've opened a bottle for that on saturday. I hope you'll appreciate that one even thought it's italian ;)) It was very nice to work with you on that PR, looking forward to write some code in the future :)

asticode · 2021-03-28T09:22:18Z

Haha hope it was a good one!

It was a pleasure too, let's connect on LinkedIn as well (I've sent you an invite).

Cheers

tmm1 · 2021-04-01T17:54:55Z

Congrats! Thanks for all the great work here! 🎉

Ilya Barbashov added 30 commits January 25, 2021 20:17

starting

d07a182

pcr write

0798d32

go-astikit replacement until PR is merged

c9e1bff

packet header, adaptation field, dts write

9c4fa08

dependency cleanup

8741f0c

get rid of panic/recover for writing

905064d

make writePCR code more adequate

f298e1e

writePacket and adaptation field length calculation

5c71b25

writePacket and adaptation field length calculation

278fce9

s/188/MpegTsPacketSize/

4047b66

writePATSection

7dc14a4

Try* -> BitsWriterBatch

28edb16

descriptors WIP

b4a87e9

go-astikit version bump

e7a5601

descriptors WIP

3dae8fc

descriptor parsing tests refactored to allow reuse for descriptor wri…

b06993f

…ting

descriptors writing tested

3e5341d

descriptors writing refactoring

8aa5b6c

descriptors done, PMT WIP

e4788ba

write PMT section fix

28a686e

writePSIData works

fc82353

WIP

f82bc07

PES WIP

db031e6

PES functions pass tests

05b66f0

minor fix

705a8d3

muxer: pat & pmt

9391d51

muxer: more tests

78bebcd

muxer: payload writing

2475c9b

es-split WIP

05cec28

es-split seems to work

69ec742

use PSITableTypeId more instead of comparing strings

72ce363

comment cleanup

40fd6af

barbashov mentioned this pull request Mar 6, 2021

Add a muxer #10

Closed

asticode requested changes Mar 8, 2021

View reviewed changes

data_psi.go Outdated Show resolved Hide resolved

data_psi.go Outdated Show resolved Hide resolved

Ilya Barbashov added 3 commits March 9, 2021 19:00

PSITableTypeId -> PSITableTypeID

6b7966e

PESIsVideoStreamId -> PESIsVideoStreamID

a6249bb

PSITableTypeID.String() -> PSITableTypeID.Type()

5064bcb

barbashov requested a review from asticode March 10, 2021 10:36

asticode requested changes Mar 22, 2021

View reviewed changes

Ilya Barbashov added 2 commits March 22, 2021 23:23

review fixes: first pack: constants and stuff

2abf745

packet_buffer read() renamed to peek()

5d7277f

Ilya Barbashov added 5 commits March 24, 2021 03:08

tools are moved to cmd directory

6af7b0b

correct prefixes for muxer and demuxer opts

a5c45f3

SetPCRPID instead of isPCRPID of AddElementaryStream

18725af

test fix and some comments

432588a

muxer WritePacket export

135fae5

Ilya Barbashov added 3 commits March 24, 2021 13:30

astits.New renamed to NewDemuxer

f2a492a

astikit version bump; writeBytesN removed in favor of one in BitsWriter

e943295

WritePayload -> WriteData with MuxerData

bc4c488

asticode merged commit 5caf397 into asticode:master Mar 24, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add muxer #16

Add muxer #16

barbashov commented Mar 2, 2021 •

edited

Loading

barbashov commented Mar 6, 2021

asticode left a comment

asticode left a comment

asticode Mar 22, 2021

barbashov Mar 22, 2021

barbashov Mar 24, 2021

barbashov Mar 24, 2021

asticode Mar 24, 2021

barbashov Mar 24, 2021

barbashov Mar 24, 2021

barbashov Mar 24, 2021

asticode Mar 24, 2021

barbashov commented Mar 23, 2021

asticode commented Mar 23, 2021

barbashov commented Mar 23, 2021

asticode commented Mar 24, 2021

asticode commented Mar 24, 2021

barbashov commented Mar 24, 2021 •

edited

Loading

barbashov commented Mar 28, 2021

asticode commented Mar 28, 2021

tmm1 commented Apr 1, 2021

Add muxer #16

Add muxer #16

Conversation

barbashov commented Mar 2, 2021 • edited Loading

barbashov commented Mar 6, 2021

asticode left a comment

Choose a reason for hiding this comment

asticode left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

barbashov commented Mar 23, 2021

asticode commented Mar 23, 2021

barbashov commented Mar 23, 2021

asticode commented Mar 24, 2021

asticode commented Mar 24, 2021

barbashov commented Mar 24, 2021 • edited Loading

barbashov commented Mar 28, 2021

asticode commented Mar 28, 2021

tmm1 commented Apr 1, 2021

barbashov commented Mar 2, 2021 •

edited

Loading

barbashov commented Mar 24, 2021 •

edited

Loading