Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inverted samples in WAV recording when clipping #971

Open
mumble-voip opened this issue Mar 15, 2013 · 43 comments

Comments

@ghost
Copy link
Collaborator

commented Mar 15, 2013

I'm not sure whether this is related to Mumble or libsndfile, but when I record into a WAV file I get really heavy noise when samples are clipped. Instead of just being clipped to value 1.0 or -1.0 they somehow change sign and "jump" to the other side. To demonstrate this, I have attached a screenshot from Audacity where on the top I did a direct audio recording using Audacity and on the bottom you see the recorded WAV from Mumble of the same voice.

So, on the top this is how I would expect the waveform to look like and on the bottom you see the inverted samples. This makes the clipping noise much worse than it should be. Instead of sounding just a bit distorted, you get a very loud click noise.

I have tried and switched line 198 in VoiceRecorder.cpp from

sfinfo.format = SF_FORMAT_WAV | SF_FORMAT_PCM_24;

to

sfinfo.format = SF_FORMAT_WAV | SF_FORMAT_FLOAT;

and the inverted samples are gone. But this is probably just a work-around.

Any clue?

This ticket has been migrated from sourceforge. It is thus missing some details like original creator etc.
The original is at https://sourceforge.net/p/mumble/bugs/971/ .

*The following attachments were added on the original item:

@ghost ghost assigned hacst Nov 7, 2013
@hacst

This comment has been minimized.

Copy link
Member

commented Nov 7, 2013

That looks funky. It's very possible that using the float format is actually the fix as that's what we use internally. I'll have to take a proper look at the source-code to make sure.

This might actually be the source of the other issue described in #1024 .

hacst added a commit to hacst/mumble that referenced this issue Nov 8, 2013
When libsndfile is handed float data but asked to write it in a
non-float format by default wrapping behavior occurs. This lead to
very audible artifacts in recordings for those file types.

This fix enables hard-clipping behavior in libsndfile for non-float
file formats by setting the SFC_SET_CLIPPING property to true.
@ghost

This comment has been minimized.

Copy link

commented Dec 23, 2013

@dD0T Hello! I've had these crackling issues on 1.2.4. Is this fix implemented in the latest snapshot on the official site?

@hacst

This comment has been minimized.

Copy link
Member

commented Dec 24, 2013

Unfortunately not. I applied the fix to this branch after refactoring so I wasn't able to selectively take it over into our master before cleaning it up. It is scheduled to be in 1.2.5 though. As temporary workaround you can record in AU or Vorbis format. Those use in floating point and should not clip.

@ghost

This comment has been minimized.

Copy link

commented Dec 28, 2013

@dD0T Oh. :/ So the issue persist with .flac as well?
I prefer to record in .wav or .flac as they arent compressed..

@hacst

This comment has been minimized.

Copy link
Member

commented Dec 28, 2013

@thernztrom FLAC takes in PCM24 data so it is affected. While AU is a pretty unusual format it is using float and is uncompressed so you can convert it to a format of your liking afterwards.

@ghost

This comment has been minimized.

Copy link

commented Dec 28, 2013

@dD0T Okay. Thanks for the fast reply! :)
Hopefully we'll have a fix for this soon. In the mean time I'll be lazy and record in .ogg.

hacst added a commit to hacst/mumble that referenced this issue Feb 12, 2014
When libsndfile is handed float data but asked to write it in a
non-float format by default wrapping behavior occurs. This lead to
very audible artifacts in recordings for those file types.

This fix enables hard-clipping behavior in libsndfile for non-float
file formats by setting the SFC_SET_CLIPPING property to true.
hacst added a commit that referenced this issue Feb 13, 2014
When libsndfile is handed float data but asked to write it in a
non-float format by default wrapping behavior occurs. This lead to
very audible artifacts in recordings for those file types.

This fix enables hard-clipping behavior in libsndfile for non-float
file formats by setting the SFC_SET_CLIPPING property to true.
@hacst

This comment has been minimized.

Copy link
Member

commented Feb 13, 2014

@thernztrom I just merged the fix as 8e22f9a . Snapshots are building currently (should be 1.2.5-240-gea165cd). Would be great if you could check to make sure this fixes your issues.

@hacst

This comment has been minimized.

Copy link
Member

commented Feb 23, 2014

I'm closing this in the assumption that the issue is resolved. Please reopen if this isn't the case.

@hacst hacst closed this Feb 23, 2014
@flyser

This comment has been minimized.

Copy link

commented Feb 9, 2015

I am experiencing this issue in flac or wav multi-track recordings with version 1.3.0 e469bd6
Please reopen

@mkrautz mkrautz reopened this Feb 9, 2015
@hacst

This comment has been minimized.

Copy link
Member

commented Feb 9, 2015

@flyser Hi. Can you provide a recording with the issue? Also which platform and architecture architecture are you running on? I can't reproduce the issue with the current snapshot (was pretty easy before). If it's just clipping that would be expected behavior. This issue produced wraparound with a very audible crackle.

@flyser

This comment has been minimized.

Copy link

commented Feb 25, 2015

I analyzed the problem a bit further and found very weird behaviour.
Take a look at the following screenshot: http://i.imgur.com/y7uJey8.png
The first track is the flac file generated by mumble, the second track is the same flac file transcoded to .wav with ffmpeg and the third track is an ogg file generated by a different mumble instance.

I would be happy to provide a sample file, but it seems I can't cut out the example portion without creating the artifacts you see on track two.
I am wondering why audacity is able to decode it properly, but neither sox, flac-utils nor ffmpeg are ...

EDIT: system and architecture: a x86_64 Linux system

@flyser

This comment has been minimized.

Copy link

commented Feb 25, 2015

Here is another screenshot showing inversed clipping in mumble: http://i.imgur.com/Pg8faIW.png
The first track is the flac file generated by mumble and the second track is a recording done by sox via pulseaudio simultaniously

hacst added a commit that referenced this issue Feb 25, 2015
In my patch for #971 I tried to enable clipping for formats that
are not float and not vorbis (which is float by default) but checked
them like they were flags. This caused clipping to never actually
enable for formats whose submask contained at least one bit similar
to either the float or vorbis type.

This patch uses SF_FORMAT_SUBMASK and a straight comparison instead
and should fix this issue.
@hacst

This comment has been minimized.

Copy link
Member

commented Feb 25, 2015

Ah dammit. I just reviewed my patch (8e22f9a) and it looks like I was working under the invalid assumption that SF_FORMAT_* subtypes were flags and not an enumeration. That screwed up the comparison. No idea why I didn't catch that in testing.

@flyser If you can compile from our master please check if 1c00533 resolves the issue. Thanks a lot for reporting this.

@flyser

This comment has been minimized.

Copy link

commented Feb 25, 2015

Thanks for the fast response, I will test this as soon as I can.
What about the inverted clipping that occured in the ogg file as well?

@hacst

This comment has been minimized.

Copy link
Member

commented Feb 26, 2015

Not sure what's up with that. From my understanding vorbis only does floating point encoding and shouldn't be constrained to [-1.0, 1.0] either. Are you sure that spike is an inversion and not just a pop + clipping introduced in another way?

That being said: If it does turn out to be an inversion I'd be interested to find out where in our processing pipeline that is coming from...

@flyser

This comment has been minimized.

Copy link

commented Feb 26, 2015

The inverted clipping you see in http://i.imgur.com/Pg8faIW.png is also present in the ogg file and not present in the sox recording, so yes I am pretty sure

@flyser

This comment has been minimized.

Copy link

commented Mar 1, 2015

I suspect the inverted clipping occurs before the opus encoding step, as surrounding samples seem to contain encoding artifacts. Maybe the compressor/amplifier is causing the clipping?

@hacst

This comment has been minimized.

Copy link
Member

commented Mar 6, 2015

Sounds reasonable. The output path from decoder into the recorder should all be float. In the input we still have a PCM16 conversion due to the Speex filter stage (those should all be scale + clamp though).

Wonder if AGC would produce sth. like this if gain is too high...that would suck...(I doubt that though)

@flyser

This comment has been minimized.

Copy link

commented Mar 10, 2015

Is there anything I can do to debug this issue? I am happy to test patches or provide debugging output if it helps. This bug means a lot of extra work for me and would love to see it fixed.

@notthetup

This comment has been minimized.

Copy link

commented Mar 19, 2015

👍 Didn't realise this was a Mumble issue. Have been trying to fix stuff with level for almost 1.5yrs.

I'm gonna try to build of 1c00533 on saturday. Will report if it fixed the issues.

@notthetup

This comment has been minimized.

Copy link

commented Mar 19, 2015

Wow, compiling Mumble on my OSX is complicated. Are the development snapshots off the master branch?? So would Mumble-Universal-1.3.0547g657f9e8 which was built on 15-Mar-2015 01:40 have picked up 1c00533?

@notthetup

This comment has been minimized.

Copy link

commented Mar 19, 2015

OK. Got it. I'll try this fix and let you know in a couple of days!

@flyser

This comment has been minimized.

Copy link

commented Mar 19, 2015

1c00533 does not fix this completely. I am not sure how to explain this ... somehow the waveform is always clipped to the positive (i.e. the upper maximum in audacity) value even if they should be clipped to the lowest value. Does that make sense to you?

@notthetup

This comment has been minimized.

Copy link

commented Mar 20, 2015

@flyser I think a waveform image would explain this the best.

@notthetup

This comment has been minimized.

Copy link

commented Mar 20, 2015

Also, it seems all builds in the snapshot repository from 1c00533 forward are Universal. What's the process of creating snapshots?

P.S. I'm failing horribly at building mumble on OSX. After working around some QT SSL version issues (which seems to be fixed in the latest commit) now there is a linker issue.

@hacst

This comment has been minimized.

Copy link
Member

commented Mar 20, 2015

@notthetup I think @mkrautz is currently working on some issue with the normal OSX builds that prevent new snapshots for them from being created.

@notthetup

This comment has been minimized.

Copy link

commented Mar 21, 2015

Yup. I talked to @mkrautz on IRC. I'm stuck at the same place as him.

@mkrautz

This comment has been minimized.

Copy link
Member

commented Mar 21, 2015

@notthetup It's fixed now.

@notthetup

This comment has been minimized.

Copy link

commented Mar 21, 2015

👍

@notthetup

This comment has been minimized.

Copy link

commented Mar 22, 2015

So I was using a g657f9e8 build for my podcast recording yesterday, and I can confirm that the inversion issue is fixed.

Here is a screenshot of a bit where the audio clipped because it was too loud. Before this, it would have inverted.

screenshot 2015-03-22 17 43 47

@flyser

This comment has been minimized.

Copy link

commented Mar 22, 2015

This is what happened to some really bad clipping in my recording: corruption1
The first track is the original mumble file, the second track shows how the clipping should look like.

@flyser

This comment has been minimized.

Copy link

commented May 6, 2015

Note that #1651 fixes some inverted clipping that was shown in this issue. inverted clipping in FLAC (and possibly WAV) recording as shown in my last comment is still present though

fwaggle added a commit to fwaggle/mumble that referenced this issue Jul 8, 2015
In my patch for mumble-voip#971 I tried to enable clipping for formats that
are not float and not vorbis (which is float by default) but checked
them like they were flags. This caused clipping to never actually
enable for formats whose submask contained at least one bit similar
to either the float or vorbis type.

This patch uses SF_FORMAT_SUBMASK and a straight comparison instead
and should fix this issue.
@ghost

This comment has been minimized.

Copy link

commented Nov 18, 2015

Any progress on this issue? @hacst
Havent tested the new versions since 1.2.4, but looking at this its at the same state in 1.2.10?

Some clipping exmaples from when we recorded audio in 1.2.4 (.wav):
https://youtu.be/2i35f6XBvCg?t=15s
https://youtu.be/2i35f6XBvCg?t=1m

@flyser

This comment has been minimized.

Copy link

commented Nov 18, 2015

1.2.10 should still be affected by this bug. I haven't tested the 1.3 prereleases for a while though

@hacst

This comment has been minimized.

Copy link
Member

commented Nov 19, 2015

@thernztrom As flyser says 1.2.10 does not have the later patches. We changed numbering schemes http://blog.mumble.info/version-numbering-change/ which makes the version numbers mentioned in this issue a bit confusing. TL;DR: Since 1.2.4 the 1.2.X branch only receives essential fixes for security and breaking issues. Pretty much all the new features as well as most bugfixes are currently only available in our 1.3 snapshots.

W.r.t to whether it is completely fixed by now: I don't know. Looking at flysers last reports it doesn't look like it. I think I had some IRL distractions and simply forgot about following up with this one again...

@flyser Steps to reproduce is simply recording something in say WAV and ensuring clipping occurs? That's on a Linux box with Pulseaudio?

@flyser

This comment has been minimized.

Copy link

commented Nov 20, 2015

It should be reproducable in any recording that takes several minutes. It might me easier if you allow mumble to amplify your microphone a lot (I don't remember what this setting was called; the automatic amplification thingy) and when you don't talk constantly, but do a lot of pauses, so the automatic amplification has time to raise the volume while you don't talk.

But even without these preconditions, it should be easy to reproduce.

To find the clipping in the recording, I opened the file in audacity, doubled the track and applied a 20kHz high-pass filter to the second one. When you zoom near the peaks of the filtered track, the result should be similar to the screenshots I posted.
EDIT: Also apply a normalization filter to the high-pass track to make the peaks more visible.

I also use Linux and pulseaudio, but I doubt it makes any difference, since .au recording is fine.

@flyser

This comment has been minimized.

Copy link

commented Dec 4, 2015

@hacst Did you manage to reproduce it?

@hacst

This comment has been minimized.

Copy link
Member

commented Dec 6, 2015

Unfortunately not yet. I'm having a real hard time to even get it to clip in the first place and the occurances I've had were all clean. I'll have to inject clipping audio into the input stage I guess...

@mkrautz

This comment has been minimized.

Copy link
Member

commented Dec 13, 2015

We believe the commit 23fa9b3 might have something to do with this.

Could you guys please test this? It is in the latest snapshot, 1.3.0883g2a31708~snapshot.

Thanks.

@flyser

This comment has been minimized.

Copy link

commented Dec 13, 2015

I didn't do much recording lately, but I will try to test this in the coming weeks.

@flyser

This comment has been minimized.

Copy link

commented Jan 9, 2016

I just did a one hour recording with 2 participants (me using a git build of today on linux and the other one using the most recent snapshot on windows). Both tracks did not exhibit any inverted clipping when recording to floating-point .au, but did invert-clip at least 15 times during this hour when recording to integer .flac.

unascribed added a commit to unascribed/mumble that referenced this issue Apr 7, 2016
When libsndfile is handed float data but asked to write it in a
non-float format by default wrapping behavior occurs. This lead to
very audible artifacts in recordings for those file types.

This fix enables hard-clipping behavior in libsndfile for non-float
file formats by setting the SFC_SET_CLIPPING property to true.
unascribed added a commit to unascribed/mumble that referenced this issue Apr 7, 2016
In my patch for mumble-voip#971 I tried to enable clipping for formats that
are not float and not vorbis (which is float by default) but checked
them like they were flags. This caused clipping to never actually
enable for formats whose submask contained at least one bit similar
to either the float or vorbis type.

This patch uses SF_FORMAT_SUBMASK and a straight comparison instead
and should fix this issue.
@Kissaki Kissaki modified the milestones: 1.3.0, 1.3.1, 1.4.0 Aug 23, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.