Add support for the Opus codec.

[sebma]

Hi,

The opus codec in gstreamer should be able to play opus files in Sailfish 2.0.0.10 at last, please see https://together.jolla.com/question/5556/upgrade-gstreamer-to-support-opus-codec/?answer=119918#post-id-119918

Can you add support for the Opus codec in your application ?

God bless you.

[florianjacob]

I'm also interested in this and checked the current state. While gstreamen on Jolla supports opus just fine, the problem is that QAudioEncoder used by sailfish-recorder doesn't, even in the dev version of Qt. See the backend class which interfaces with gstreamer: https://github.com/qtproject/qtmultimedia/blob/dev/src/plugins/gstreamer/mediacapture/qgstreameraudioencode.cpp#L72

The patch for QtMultimedia would actually be a single line, I think:

m_elementNames["audio/opus"] = "opusenc";

…
Even with this change integrated in the next Qt version, the Qt version on SailfishOS probably won't be updated anytime soon. 😦 So the only real option for sailfish-recorder to support opus would be to interface with gstreamer directly, either via C API or command line.

[florianjacob]

Nevertheless I just wrote a Qt suggestion for gstreamer opusenc support. Maybe the SailfishOS Qt version will be upgraded some day, or some desktop application will gain opus support through that. 🙏

[cornedor]

Thanks for this research, I'm not really experienced with gstreamer or C overall. So while I don't think I will get very far with this, I will look into this. Might even learn something :-)

[florianjacob]

Yeah, as much as I like opus, I'm really not sure whether this will be possible. 😢

Neither the gstreamer Qt bindings or gstreame itself seem to be [libraries allowed in harbour][https://github.com/sailfish-sdk/sdk-harbour-rpmvalidator/blob/harbour-qa/allowed_libraries.conf], though. 😞 Maybe some other library in there has some audio recording / encoding capabilities as an alternative to gstreamer?…

The FAQ states

GStreamer APIs and plug-ins
We are currently investigating the best way to support GStreamer API usage. Our current
packages are from GStreamer 0.10, for API and ABI stability, the GStreamer 1.x series of
releases would be preferable for Harbour. Consider using QtMultimedia instead, which is already
a supported API in Harbour, and has C++ bindings as well as QML components ready to use.

, though. So, are they allowed or not?…

What I meant with CLI by the way, there's the “gst-launch-0.10“ CLI application. The man page states that it is not intended for developing against, only for debugging and one should use the C api, but as you can use the pipeline descriptor in a C call as well.

Two days ago, I toyed around with something like this:

# gst-launch-0.10 alsasrc ! audioconvert ! audioresample ! opusenc ! oggmux ! filesink location=mic.ogg

Thanks for creating and still caring about sailfish-recorder by the way. 😄 If you need any code reviews, let me now.

[cornedor]

Thanks for this comprehensive reaction, sadly there are quite a few lib's/api's that we can't use. Would also be cool if I could share recordings like I do in the share branch. The FAQ is indeed not very clear about it, if it is allowed or not, but I could always try.

CLI would indeed be a but weird and hard to debug 😄

If you like to do some code reviews, that would be really nice, I usually only dev in JavaScript etc ;-)

[sebma]

Hi,

@florianjacob

The Harbour FAQ has not been updated since 24.09.2015

Opus has been pushed to mer and libav has been updated so GStreamer now supports the opus codec : https://together.jolla.com/question/5556/upgrade-gstreamer-to-support-opus-codec/?comment=135655#comment-135655

Is it now possible to include the Opus support in SailFish Recorder for SailFish OS 2.0.1.11 ?

[florianjacob]

@sebma all of my comments were made after 2.0.1.11 was already released with gstreamer opus support.

So the situation has not changed since then: Any Sailfish app wanting to support opus has to talk to gstreamer directly instead of using the Qt abstractions. As long as Qt does not get that feature and SailfishOS will get an update to that Qt version (which would need at least a few years, I guess), the only option would be to write a recording core for this app that uses gstreamer directly. 😞

[sebma]

@florianjacob Thanks for your explanation, now I understand it right :)

I'd like to create an issue on the qtmultimedia github project to tell them to add the opus support but it seems creating issues is not allowed.

Is there another way to communicate with them ?

[florianjacob]

@sebma: The Qt issue tracker is the official way to communicate, the github project is a mirror.

As mentioned in #5 (comment), I already created QTBUG-50567 for opus support, but it has not seen much activity since then. You could add a comment there that this feature is important to you, too. 👍

[sebma]

@florianjacob It seems your modification suggestion is not enough, these lines would have to be modified (or copied and modified :) ) for opus support :

$ curl -Ls https://github.com/qtproject/qtmultimedia/raw/dev/src/plugins/gstreamer/mediacapture/qgstreameraudioencode.cpp | grep -n speex
55:    codecCandidates << "audio/AAC" << "audio/mpeg" << "audio/vorbis" << "audio/speex" << "audio/GSM"
58:    codecCandidates << "audio/mpeg" << "audio/vorbis" << "audio/speex" << "audio/GSM"
73:    m_elementNames["audio/speex"] = "speexenc";
80:    m_codecOptions["audio/speex"] = QStringList() << "mode" << "vbr" << "vad" << "dtx";
225:            } else if (codec == QLatin1String("audio/speex")) {

[sebma]

@cornedor Can you also post a comment on the QTBUG-50567 ?
The more people are interested, the more chances there will be that they start coding the opus support for QtMultimedia.

Btw: There is C++ binding for Gstreamer : https://gstreamer.freedesktop.org/bindings/cplusplus.html (just in case :) )

[florianjacob]

@sebma You're right! In case you have the time, you could massively increase our chances to get this into Qt by preparing a pull request and testing the change. 👍

[florianjacob]

opus is now supported in Qt 5.7.1 and 5.9 alpha! https://bugreports.qt.io/browse/QTBUG-50567
Whenever SailfishOS will get another Qt upgrade after the recent upgrade to 5.6, this will (hopefully) just work. 😄