Feature: Opus as additional audio codec

[eXpl0it3r]

SFML Tasks

• Fix conflict and rebase onto master
• Visual Studio builds play the audio at a slightly slower rate than the original
• Conversion warnings between std::size_t, unsigned int, sf::Uint64, opus_uint64, ...
• MinGW builds don't play a sound with Opus
• Opus can't be found on some system (see CI builds)
• MinGW builds have undefined references to some function
• Test on OS X
• Test on Linux

---

Original PR by @susnux #1077.

I've added the Windows binaries, however there are still some issues.

[eXpl0it3r]

@susnux Does the example opus file from the official opus website playback at the right speed for you on your system?

For me the Paper Lights by Ehren Starks example is about 20s slower than when played back on the website, in VLC or MPC-HC.

[susnux]

Will look into this as quick as possible, but the next few days I am quite busy with work.

[SergioRAgostinho]

FYI: I just compiled this and the sound example is working but it's as you reported the number of samples per second seems to be incorrect. Vs the original, the song is playing at a lower tone. It feels like the number of samples per sec were incorrectly parsed from 48k to 44.1k. That justifies the 20 second difference.

OS: Ubuntu 16.04
Audio Interface (USB): Focusrite Scarlett 2i4

Side note: The graphics module is throwing a linker error which is unrelated to this PR.

[SergioRAgostinho]

Here's the problem, there was a misinterpretation of the sample_rate.

All Opus audio is coded at 48 kHz, and should also be decoded at 48 kHz for playback (unless the target hardware does not support this sampling rate). However, this field may be used to resample the audio back to the original sampling rate, for example, when saving the output to a file.

[eXpl0it3r]

@susnux Any chance you get around to invest some more time into this?

[SergioRAgostinho]

I'm interested in seeing in this feature pushed so if it's just some final cleanup stuff I can lend a hand.

This fixed the slowed play issue

diff --git a/src/SFML/Audio/SoundFileReaderOpus.cpp b/src/SFML/Audio/SoundFileReaderOpus.cpp
index 095113c..2dfa949 100644
--- a/src/SFML/Audio/SoundFileReaderOpus.cpp
+++ b/src/SFML/Audio/SoundFileReaderOpus.cpp
@@ -119,7 +119,7 @@ bool SoundFileReaderOpus::open(InputStream& stream, Info& info)
     // Retrieve the music attributes
     const OpusHead* opusHead = op_head(m_opus, -1);
     info.channelCount = opusHead->channel_count;
-    info.sampleRate = opusHead->input_sample_rate;
+    info.sampleRate = 48000;
     info.sampleCount = static_cast<std::size_t>(op_pcm_total(m_opus, -1) * opusHead->channel_count);
 
     // We must keep the channel count for the seek function

There's of course the downside that I'm not familiar with the inner details of the library and code style, so the review process might be more involved.

[eXpl0it3r]

Would be great if you could create a pull request and base it onto the feature/opus branch.

Also you may want to add a comment with the link from above to the why it needs to be 480000.
Also what's the opusHead->input_sample_rate used for then?

I'll look into getting the merge conflict solved and get it up to speed with the latest master.

[SergioRAgostinho]

Sure.

Also what's the opusHead->input_sample_rate used for then?

I assume it's relevant from a transcoding perspective. Although all opus sound uses a 48kHz sample rate raw audio, if you want to transcode it to another format and assuming the original sample rate was at a lower sample rate, you might want to resample the audio first to it's "original format" before encoding again, saving you some unnecessary samples from the encoding process.

[SergioRAgostinho]

Based on the docstring for SoundFileReader::seek

SFML/include/SFML/Audio/SoundFileReader.hpp

Lines 79 to 91 in e31b925

	////////////////////////////////////////////////////////////
	/// \brief Change the current read position to the given sample offset
	///
	/// The sample offset takes the channels into account.
	/// Offsets can be calculated like this:
	/// `sampleNumber * sampleRate * channelCount`
	/// If the given offset exceeds to total number of samples,
	/// this function must jump to the end of the file.
	///
	/// \param sampleOffset Index of the sample to jump to, relative to the beginning
	///
	////////////////////////////////////////////////////////////
	virtual void seek(Uint64 sampleOffset) = 0;

and the docstring for op_pcm_seek.

should it not also be divided by the sample rate, in this case 48000.

[eXpl0it3r]

I don't think so. It wouldn't be working at all if this was the case, plus Opus uses the same API as OGG (and FLAC) and there it isn't needed either, plus dividing by 48000 sound like a huge number, not surprising if this would introduce odd rounding issues.

[LaurentGomila]

The function wants a number of samples, that's what we give it. Dividing by the sample rate would give a number of seconds.

[SergioRAgostinho]

Then I don't get the message

/// Offsets can be calculated like this: 
/// `sampleNumber * sampleRate * channelCount` 

Disregarding the channelCount

sampleNumber still has dimension [samples]
sampleRate is [samples/sec]

according SoundFileReader::seek "offset" expression I would get [samples^2/sec]. What exactly is this? I'm I making a mistake somewhere?

[LaurentGomila]

This part of the documentation was added later and is obviously wrong, indeed. Feel free to remove or correct it (sampleNumber should be "timeInSeconds").

[Foaly]

Seeing the last commit made me wonder. From my knowledge SFML uses 44100 Hz for playback internally. If Opus is read out at 48000 Hz the sound should be pitched down when played back at 44100. Is there a resampling happening somewhere in the encoder or in OpenAL? I tried to answer it by looking at the code, but couldn't find anything.

[LaurentGomila]

Is there a resampling happening somewhere in the encoder or in OpenAL?

Sure there is! You can play samples with any source sample rate, it's a parameter (see SoundBuffer::loadFromSamples, or SoundStream::initialize for example).

[Foaly]

Ah I did not know that! Very interesting! Thank you

[JonnyPtn]

This works well on macOS 10.13, using opus/file installed via brew (And tested with SFML sound example). I'll hopefully add the precompiled libs soon, and give iOS a test

[eXpl0it3r]

I'll move this to 2.6 for now, but if someone is willing to invest the required time to get it setup and tested across platforms before 2.5 it can be changed again.

[ChrisThrasher]

https://github.com/ChrisThrasher/SFML/tree/feature/opus

I rebased this branch on the master branch and made a few fixes to get it to compile.

[ChrisThrasher]

Looks like there are still plenty of lossy conversions to add static_cast to. Not a difficult problem to fix but it will be tedious to fix all those.