Skip to content

Commit

Permalink
src/mumble/AudioInput: Document a reproducible test for verifying the…
Browse files Browse the repository at this point in the history
… correct operation of the echo canceller
  • Loading branch information
fedetft committed May 20, 2020
1 parent 428e555 commit 0c07744
Showing 1 changed file with 31 additions and 8 deletions.
39 changes: 31 additions & 8 deletions docs/AudioInputDebug.md
Expand Up @@ -46,14 +46,6 @@ The audio dumps have an additional property that is fundamental for debugging th

This is fundamental for an echo canceller, which can break simply because the microphone data arrives before the speaker one (how can the echo canceller predict an echo from the future?), or if the speaker data is so ahead that exceeds its limited filter length.

### Example of a bug in the echo canceller detected using the dumps

![](AudioInputDebugFiles/bug.png)

### Exampe of the fixed echo canceller

![](AudioInputDebugFiles/fix.png)

### The `--print-echocancel-queue` option

Now that I've mentioned the requirement for the echo canceller to have well aligned inputs, maybe it's best to introduce the `--print-echocancel-queue` option. When running Mumble with this option, the current state of the queue in the Resynchronizer class used to align the microphone and speaker readback streams is printed on the command line. Moreover, if packets are dropped (which is necessary to keep the signals aligned if the OS/pulseaudio/audio card is playing tricks to us), those will be printed as well.
Expand Down Expand Up @@ -85,3 +77,34 @@ With a 10ms chunk, this queue should introduce a ~20ms lag to the voice.
![](AudioInputDebugFiles/fsm.png)

Here _m_ means a microphone chunk was received, _s_ a speaker chunk was received, and the number in the state is the queue fill level. The design tries to keep the limit cycle of the queue add/remove pattern between 1 and 4 elements, preventing the queue to operate in a limit cycle between 0 and 1 elements (queue too empty, the speaker data may risk arriving after the microphone) and in a limit cycle between 4 and 5 elements (too full, we're wasting some precious filter length to cancel real echo just because some delay accumulated).

### A reproducible test for verifying the correct operation of the echo canceller

To avoid regressions being introduced in the echo cancellation feature, it is beneficial to have a controlled test that can be easily reproduced to test whether the echo canceller works.

You will need:

* Low quality headphones that cause echo. The in-ear type that's used with smartphones and has a combined microphone/headphones jack works best. If you don't have them or your PC lacks a combined microphone/headphones jack, do the test with your speakers, but keep the volume relatively quiet. Some echo is unavoidable at high volume levels, especially if it makes the microphone clip.
* A quiet Mumble server to connect to. Just join an empty room with no other users.

Here's the step by step guide:

1. Make sure Mumble echo cancellation is enabled. You may also need to repeat this test twice with mixed and multichannel echo cancellation.
1. Run Mumble with the `--dump-input-streams` option
2. Join the quiet server
3. Play the first 15 or so seconds of a youtube video that contains both a relatively periodic note and voice, such as this one: https://www.youtube.com/watch?v=im9z8NT96Iw
4. Say a phrase, such as "Testing 1 2 3"
5. Close Mumble
6. Open the three dumped streams in Audacity. Don't forget to select the correct number of channels for the `speaker_dump` when testing multichannel echo cancellation
7. Play the raw microphone stream, you should hear the echo of the youtube video clearly above the noise, and it should be less loud than you saying "Testing 1 2 3". If you hear no echo, increase you headphones volume, switch to worse headphones or use your speakers and repeat. If the echo is as loud as or louder than you speaking, reduce your audio volume and repeat.
8. Now listen to the processed microphone stream: the echo should be almost gone, both the note and the voice coming from youtube, while your voice should remain. It is acceptable that after a silence gap, the first part of the echo can reappear, but it should quickly be cancelled. If not, there's a bug in Mumble.
9. Play the speaker dump. It should sound as well as the youtube video itself. If not, there's a bug in Mumble.
9. As a final check, take a transition from silence to noise as a reference and zoom in in Audacity: the speaker dump should precede the microphone dump by 20ms or so (0 to 50ms is acceptable). If not, there's a bug in Mumble.

Example of an echo canceller bug: the speaker data lags compared to the microphone one. As a result, only the note is cancelled, but voice is not.

![](AudioInputDebugFiles/bug.png)

Exampe of the a working echo canceller.

![](AudioInputDebugFiles/fix.png)

0 comments on commit 0c07744

Please sign in to comment.