Audio drop-outs since recent updates

[neilkinnish]

Following on from ... #142

The silence changes appear to have solved that issue, but I am now getting drop-outs in the audio (for audio only files).

I can supply files if required, but I am able to re-create on every recording.

[frostbyte73]

can you give more details on how to reproduce? I'm not seeing this in my testing

[neilkinnish]

@frostbyte73 yes sure, I'm using startTrackEgress as you know this outputs an ogg – thats really about it. I test by just recording and counting with natural pauses and the audio then has broken parts.

This only started in the recent update.

I did test with trackComposite set to MP4 (audio only) and the results were better, but a longer recording did have missing parts (although the room recording was fine).

In all cases we are recording the room at the same time.

I have spent some time looking around your changes today and reading more about gstreamer (not used extensively before). I made a small tweak that seems to fix my issue, but I'll test more before I post details to make sure I am on right path.

[neilkinnish]

@frostbyte73 I see you've released the egress today, I'll also run that build to ensure it's nothing to do with our build from master.

[neilkinnish]

@frostbyte73 testing with your build (v.1.5.0) and still experiencing the issue, quite easy for us to re-create. Have had a couple of recordings where it took a little longer to start happening (2 mins in for example) but larger parts of audio drop out.

Regarding the settings and use-inband-fec will this require plc?

[liautaud]

Hi! I've also been experiencing this issue. I'm using startTrackEgress to output .ogg audio files for every track in my room, and I systematically end up with audio cut outs. They seem to worsen the longer the recording lasts.

Here's a sample file to illustrate. Notice the cuts at at 01:59, 02:08, 02:16, 02:29, 02:38, 02:41 etc.
glitch.ogg.zip

[frostbyte73]

@neilkinnish I've rewritten most of the track composite handling - I found a number of issues related to corner cases with timestamping, track sync, etc. The updates are released in v1.6.3

[moksh-samespace]

@frostbyte73 Hey, still facing the issue of audio drop for v1.6.3. After running in debug mode I get the following logs

2023-04-26T05:22:51.822Z WARN egress sdk/synchronizer.go:272 timestamping issue

Let me know if you need any more details.

[blombosi]

@frostbyte73 I am facing some issues probably related to this issue too. In my case I want to record audios and I start track_egress, but after the file is uploaded to S3 it either completely silent or stats with some audio but after a certain point it remains silent even if the participant was unmuted and talked.

What I could debug is that after the track remains silent in the file the same time this logs come up in the egress, and it never stops coming.

2023-05-24T14:21:13.734Z WARN egress synchronizer/track.go:181 timestamping issue {"nodeID": "NE_kC7PH64ww9Fo", "handlerID": "EGH_h4hKyi9uNTMd", "clusterID": "", "egressID": "EG_6uAtZDxTAb64", "expected": "36.785090237s", "calculated": "20.240083333s"} 2023-05-24T14:21:13.754Z WARN egress synchronizer/track.go:181 timestamping issue {"nodeID": "NE_kC7PH64ww9Fo", "handlerID": "EGH_h4hKyi9uNTMd", "clusterID": "", "egressID": "EG_6uAtZDxTAb64", "expected": "36.805558238s", "calculated": "20.2800625s"} 2023-05-24T14:21:13.843Z WARN egress synchronizer/track.go:181 timestamping issue {"nodeID": "NE_kC7PH64ww9Fo", "handlerID": "EGH_h4hKyi9uNTMd", "clusterID": "", "egressID": "EG_6uAtZDxTAb64", "expected": "36.894333806s", "calculated": "20.3400625s"} 2023-05-24T14:21:13.854Z WARN egress synchronizer/track.go:181 timestamping issue {"nodeID": "NE_kC7PH64ww9Fo", "handlerID": "EGH_h4hKyi9uNTMd", "clusterID": "", "egressID": "EG_6uAtZDxTAb64", "expected": "36.904834187s", "calculated": "20.3200625s"} 2023-05-24T14:21:13.864Z WARN egress synchronizer/track.go:181 timestamping issue {"nodeID": "NE_kC7PH64ww9Fo", "handlerID": "EGH_h4hKyi9uNTMd", "clusterID": "", "egressID": "EG_6uAtZDxTAb64", "expected": "36.915577552s", "calculated": "20.3400625s"} 2023-05-24T14:21:13.879Z WARN egress synchronizer/track.go:181 timestamping issue {"nodeID": "NE_kC7PH64ww9Fo", "handlerID": "EGH_h4hKyi9uNTMd", "clusterID": "", "egressID": "EG_6uAtZDxTAb64", "expected": "36.930152905s", "calculated": "20.3600625s"} 2023-05-24T14:21:13.904Z WARN egress synchronizer/track.go:181 timestamping issue {"nodeID": "NE_kC7PH64ww9Fo", "handlerID": "EGH_h4hKyi9uNTMd", "clusterID": "", "egressID": "EG_6uAtZDxTAb64", "expected": "36.955069971s", "calculated": "20.3800625s"} 2023-05-24T14:21:13.914Z WARN egress synchronizer/track.go:181 timestamping issue {"nodeID": "NE_kC7PH64ww9Fo", "handlerID": "EGH_h4hKyi9uNTMd", "clusterID": "", "egressID": "EG_6uAtZDxTAb64", "expected": "36.965507488s", "calculated": "20.4000625s"} 2023-05-24T14:21:13.945Z WARN egress synchronizer/track.go:181 timestamping issue {"nodeID": "NE_kC7PH64ww9Fo", "handlerID": "EGH_h4hKyi9uNTMd", "clusterID": "", "egressID": "EG_6uAtZDxTAb64", "expected": "36.996372924s", "calculated": "20.4200625s"} 2023-05-24T14:21:13.967Z WARN egress synchronizer/track.go:181 timestamping issue {"nodeID": "NE_kC7PH64ww9Fo", "handlerID": "EGH_h4hKyi9uNTMd", "clusterID": "", "egressID": "EG_6uAtZDxTAb64", "expected": "37.018427034s", "calculated": "20.4400625s"}

[frostbyte73]

Hey all, I've done a lot of work in v1.7.2 to address timestamping and sync issues. Let me know if anyone still has issues with this, otherwise I'll close it as fixed

[frostbyte73]

Audio loss fixed in v1.7.5 - was CPU related (fix: livekit/protocol#409)