0.9 beta

The project continues to get to steadier releases, so here we are with 0.9 beta, just over 4 months after 0.8.2. The amount of changes is comparable with 0.8, and we haven't finished yet; expect a few more things to land before 0.9 gets out, hopefully in time for the very first Matrix Conference later in September!

Toolchain and compatibility

Quotient 0.9 will only build and run with Qt 6 - no Qt 5 any more (#709 by @TobiasFella). In fact, the oldest supported version is Qt 6.4, and we already use quite a few things introduced in that version. The compiler and standard library versions have been bumped up as well: it's GCC 13, Clang 16, MSVC&nbsp:2022 - we now require solid C++20 support and first elements of C++23 are already used, too (#744 and #748 by @KitsuneRal). We still don't use C++ modules.

If things go as we'd like, 0.9 will be the last version using Olm for E2EE. Although still solid, Olm is now officially deprecated by the Matrix project in favour of Vodozemac, and that will likely be our new E2EE foundation, too. This will mean adding Rust to our dependencies - while this is unfortunate (we try to keep deps to a minimum, to facilitate porting), our project is not equipped to take over Olm or otherwise reimplement E2EE in C++. On the other hand, Rust code is usually very good at performance and memory footprint, which is what we at Quotient care about. As of this writing, valiant @TobiasFella already commenced the work - if you know something about Rust-C++ bindings, you're welcome to help.

Highlights

The most prominent feature added in this cycle is (finally!) cross-signing support (#630 and #765 by @TobiasFella, #755 by @KitsuneRal). Tobias has been driving and polishing it for quite some time, using NeoChat as the testbed - all thanks go to him for making it happen.

The library is now based on Matrix API 1.11 (#763 by @KitsuneRal), and thanks to that it is possible to support authenticated media requests (#776 by @KitsuneRal and #784 by @TobiasFella). You don't need to do anything in the client, the library will automatically add the access tokens to media requests if it detects a compliant homeserver.

While not a new feature, one major refactoring in the library stands out that started back in 0.8.x: taking the room member API from somewhat heavy, QObject-based User to the separate much lighter RoomMember class. @nvrWhere has done most of the heavy-lifting - many thanks for that! With this refactoring the library won't make thousands of User instances on the heap any more, one for almost each and every MXID it learns of, without much use for it as most room members are never interacted with or even displayed. Overall, that means faster processing of sync batches (especially initial syncs). Most of the new API was introduced in 0.8.x, with the old API deprecated; and in 0.9, this deprecated API will be entirely gone (#743 and #746 by @nvrWhere). If you're missing some calls, bring up an issue and we'll sort it out.

With the library entirely flying Qt 6 now, another refactoring is in progress, and will likely continue into post-0.9 cycle: using futures instead of signals for asynchronous communication, in particular in the context of network requests (aka jobs). Using job pointers and signal-slot connections has always had caveats: e.g. you have to take extra care to connect to the signal before it has any chance of being emitted; the job type is not passed in the signal (to avoid making every single job type a Q_OBJECT, which would inflate build times greatly) so you have to use side channels to pass that type into the handler; signals for non-job single-shot processes (e.g. file downloads) need extra filtering for the identifier associated with the process (e.g. file path or a Matrix event); and so on. Futures provide a more straightforward, typesafe interface to achieve the same goal. To facilitate the transition, a new JobHandle template class is introduced that behaves both as a future and as a QPointer to the underlying job (#754 and #764 by @KitsuneRal). Eventually, we will likely stop using job pointers completely and switch to futures (especially as and when Qt gets better support for futures in QML).

Other notable changes

• Quotient_ENABLE_E2EE CMake flag is not there anymore; the library is always built with E2EE support, use Connection facilities to turn it on/off (#741 by @TobiasFella)
• With the new toolchains, std::optional is good enough that Omittable is not needed any more (#745 by @KitsuneRal)
• RoomMember is a bit more convenient to use now (#747 by @KitsuneRal)
• Matrix API structures are now exported with QUOTIENT_API (#749 by @KitsuneRal)
• Cleanup/deprecation of old code and backport of Room::requestedHistorySize() from Quaternion (#750, #778 and #786 by @KitsuneRal)
• The library doesn't run database queries twice any more (#751 by @TobiasFella)
• Auth data can have non-objects now (#752 by @KitsuneRal)
• Update membersTyping by @nvrWhere in #753
• E2EE tweaks and improvements (#756, #757, #758 and #761 by @TobiasFella)
• Small convenience facilities: findIndirect() range algorithm, editSubobject()/replaceSubvalue() for JSON manipulation, QUO_CHECK, QUO_ALARM and QUO_ALARM_X macros for data checking (#760 by @KitsuneRal, with additional fixes in #778 already mentioned above)
• Exposing member power levels in RoomMember (#768 by @nvrWhere, #769 by @KitsuneRal)
• Deserialize CBOR data without one allocation per value (#772 by @vkrause)
• Add a promise/future pair to PendingEventItem (#767 by @KitsuneRal)
• Code modernisation + newer Synapse for autotests (#775 by @KitsuneRal)
• Prepare libQuotient for operating without being able to reach the server (#777 by @TobiasFella)
• Unified event id accessor for pending and normal events (#787 by @nvrWhere)

New Contributors

Thanks, and congratulations with the first contribution, go to @teohhanhui for fixing a dead link to the NeoChat page (#785).

Full Changelog: 0.8.2...0.9-beta1

0.8.2

A new 0.8.x release is here; unlike the previous one, it includes more changes than usual due to significant contributions from NeoChat community (big thanks to @nvrWhere and @TobiasFella!).

This is meant to be the last (significant) 0.8.x release; active work on 0.9 will commence from now on the development branch, with the first big change - dropping Qt 5 support - coming immediately after 0.8.2 is released. Incremental, compat-preserving changes can still arrive for 0.8.x in the dedicated branch, with 0.8.3 potentially rolling them up in a few months.

Most notable changes:

• Historical encrypted messages can now be decrypted with old megolm keys retrieved from SSSS (@TobiasFella, #687)
• The User library API is getting prepared to be split into that for room members (RoomMember) and user profiles (stays in User for now) in 0.9, deprecating most of User methods that accept Room for the member context (@nvrWhere, #695, #724, and #725)
• Along the same lines: functions to (un)ignore users by userid rather than User* are introduced; the old signatures are deprecated now and will be removed in 0.9 (@TobiasFella, #718)
• More efficient (less cache-wasting) avatar retrieval (@KitsuneRal, #711)
• Property for querying connection's account_data (@TobiasFella, #719)
• Fix brain-split when checking event types on some Linux distributions (@KitsuneRal, #726, fixes #692)
• A new event class for m.room.server_acl (@nvrWhere, #729)
• The library supports defaulting direct chats to E2EE now (@nvrWhere, #730)

Full Changelog: 0.8.1.2...0.8.2

0.8.1.2

Thanks to the vigilant eye of @antonio-rojas, it turned out that version 0.8.1 is not entirely ABI compatible with 0.8.0 as it should. This release is made solely to fix it; there are no other changes.

0.8.1(.1)

(Version 0.8.1 missed an important PR that actually changes the advertised version and updates the documentation - so it was re-released as 0.8.1.1 with the PR merged.)

The first maintenance release in the 0.8.x branch is here now. Notable changes:

• @KitsuneRal got rid of deprecated RoomPinnedEvent name in #680
• @TobiasFella fixed session verification in #682
• @nvrWhere constified Room::userIdsAtEvent() in #686 (first contribution - thanks, and looking forward to more!)
• @TobiasFella enhanced logging around key verification in #690
• @KitsuneRal made it possible to use the key backup client-server API in the backend (#693)
• @KitsuneRal also constrained the e-mail logic in linkifyUrl() so that it doesn't trigger in the middle of another URL (#698)

Full Changelog: 0.8.0...0.8.1.1

0.8.0

Took much less than 0.7, right? Just over half a year passed since 0.7.0 has been released, and 0.7.2 came out just a couple months ago. The whole changeset since 0.7.2 is just over a hundred commits, compared to more than a thousand between 0.6.x and 0.7.0. Hopefully, we'll be able to continue this steadier, more evolutionary pace from now on, and then 0.9 will come out by the end of the year.

Most of the changes are already described in release notes for 0.8 beta - you're welcome to read them.
Since RC went out, we’ve got just two substantial changes: @vkrause fixed AccountRegistry::invokeLogin() in #675 to still add Connection objects after successfully connecting them to the homeserver; and @KitsuneRal made pinned events actually follow the specification (and therefore, interoperable) in #677.

On top of that - a few formal things that we used to add to announcements: the toolchain updates, and notes on compatibility and maintenance.

Toolchain and compatibility

Well, in fact there are not many updates here. We're still using C++20 and building with the same compiler versions as libQuotient 0.7.0 used: GCC 11, Clang 11, MSVC 2019 and Apple Clang 12 are our baseline. We also continue building with Qt 5 as well as Qt 6 - but as we warned before, Qt 5 may be dropped any time soon, with Qt 6.5 already out there and being in quite a good shape. By now we have to jump through some hoops and limit our use of the new Qt 6 API - so the pressure is already there to switch over to Qt 6 entirely as soon as version 0.9 starts coming together - meaning that 0.8.x will very probably be the last one to officially build with Qt 5.

Also, the next version will bump compiler requirements - in particular, we're going to require Clang 13 and Apple Clang 14, to take advantage of even more C++20 features (particularly the ranges framework). If your toolchains are older, take effort to update them.

Despite not a lot of time behind, libQuotient 0.8 API is not compatible with 0.7; if you didn't try 0.8 beta before, you now have to rebuild your code with the new version and see what breaks. We try to play soft and deprecate things before dropping them; but that's not always possible. Be aware that if the compilation already spits out deprecation warnings with an older version, these are the most likely places to get broken when building with the next "sub-major" (0.8->0.9, that is) version.

Understanding that the install base for libQuotient 0.6 is comparatively large, 0.6 remains on "sustained support" (i.e. updating it on request). There were not many requests to update it since 0.7 though so this is likely to only be there for another month or two.

Thanks

As always, many thanks to all who use libQuotient and help us with the development!

@KitsuneRal and @TobiasFella

0.8 RC

The release candidate for 0.8 is a relatively incremental change, as beta seemed to be doing generally fine, with only minor (and not even that necessary) tweaks here and there and the documentation update.

What's changed in code

• asKeyValueRange: support accepting rvalues (by @KitsuneRal in #671)
• Tighten up logging categories stuff (by @KitsuneRal in #672)

Full Changelog: 0.8-beta1...0.8-rc1

0.8 beta

After enormous waits for 0.6 and 0.7, the project is finally getting to make more frequent releases, and 0.8 beta is here less than half a year after 0.7.0. The amount of changes is somewhat smaller too, so you don't need to spend extended time just to figure out what's changed (and I don't need to spend even longer to write the release notes!).

This is a good moment for packagers to double-check that their builds are still good with the new library but please hold off from packaging 0.8, just yet. More goodness is coming soon, with the final release planned to happen before the end of June.

As always, thanks to all the contributors, new and seasoned - you rock!

Main news

The highlight of this release is that you can switch E2EE at each Connection object individiually (#609 by @KitsuneRal). By default, E2EE is now switched off; you have to call Connection::setE2eeDefault() to enable it for all Connections created further. On top of this class-scope setting, you can call Connection::enableEncryption() on any Connection object before it is logged in, to alter the mode for that one object.

As of this beta, there's no graceful handling of situations when encryption is enabled but is unusable (e.g. if there is a database but the pickling key doesn't match it or is unavailable). To preserve user data, Debug builds will fail on assertion; Release builds will just disable encryption on the object and keep running is if it was not enabled at all. By the final release, Debug builds will do the same as Release builds.

Other news

• The backend Client-Server API code is generated from Matrix v1.7 files (#669), enabling access, among else, to asynchronous uploads (MSC2246), content redirects (MSC3860), and using an existing session to log in another (MSC3882); these are not wired further into Connection/Room yet but the first step is made
• Thanks to @TobiasFella (#634), Doxygen documentation for the library is now generated and published automatically at https://quotient-im.github.io/libQuotient/ - reports on any inaccuracies/typos/bitrot etc. are welcome!
• @vkrause made another round of build tightening and improvement PRs:
  • Qt5 and Qt6 versions of libQuotient can now be installed side-by-side (#646, #650) - this required to change soname of the Qt6 variant but no known distro shipped that variant anyway
  • Global constants for key names etc. now have one prefered variant (the one without L at the end), with L variants getting deprecated (#648); L-less variant is now QLatin1String, too
  • Fixed dependency finding in CMake (#656, #658)
• @KitsuneRal also gave the redactions code a hard look and fixed things around redacting state events that surfaced (#666)
• Connection::assumeIdentity() emits networkError() (which it never did before) instead of loginError() when the actual problem is with the network (e9c3d37 by @KitsuneRal)
• Connection no more automatically adds and removes itself from the Accounts singleton, and this singleton itself is deprecated now (c2c9bf4 by @KitsuneRal); client authors have to create an AccountRegistry instance for themselves (if they wish, still as a singleton, but be aware that it's considered an anti-pattern)
• BaseJob::aboutToSendRequest() carries a pointer to the modifiable network request object along with it, allowing to adjust the request before sending (389f365 by @KitsuneRal). Use with care.
• Previously deprecated Room::reset*Count() functions are entirely removed (#661) after they were found out to crash NeoChat in obscure ways
• Omittable::ensure() has been removed as it was sitting unused for quite some time (14f31f4 by @KitsuneRal)
• Event::type() is deprecated now and is planned for removal in 0.9 (#667), to prevent the long-standing confusion between type() and matrixType(), with even the library's own code misusing type() even though it doesn't necessarily return the actual type stored in the event JSON
• @TobiasFella fixed aliases not to remap back from the replacement room to its predecessor accidentally (#649)
• @KitsuneRal fixed image requests from QML to be thread-safe (#659)
• @chmeeedalf fixed Matrix URI parsing for colon characters that are percent-encoded, i.e. %3A instead of : (#651) - first contribution to the project!
• @CyberTailor fixed E2EE builds with LibreSSL (#654) - another first-time contributor here!

Full Changelog: 0.7.2...0.8-beta1

0.7.2

Another round of fixes and small improvements has arrived in 0.7 branch. Special thanks go to @vkrause (with @TobiasFella as a supporting act) for a small pile of building and packaging cleanup PRs (#617, #619, #620, #621, #622, #623, #624, #632/#633, and #626/#645).

Notable changes:

• Within #626 mentioned above, the top-level source directory name was changed from lib to Quotient. Although it's not exactly backwards-compatible if you use libQuotient as a submodule, client build scripts were not supposed to rely on the existence of lib directory in the first place. On the other hand, the new top-level directory name allows to use a common Quotient/ prefix in #include statements, regardless of the way libQuotient is consumed (a submodule or an installed package).
• MxcReply has got some love, fixing crashes (#628) and broken WebP images (#627) in NeoChat (by @redstrate - first contribution, congrats and thank you!)
• Send megolm key when there is no new olm session (#629 by @TobiasFella) - fixing the "encrypted messages fail after the first 100" problem
• Fix the library failing to process to-device events beyond the first one in the sync batch (#631 by @TobiasFella)
• Two new utility functions to help client authors figure out E2EE status of a given device (#635 by @TobiasFella)
• A few tweaks and fixes in Connection and AccountRegistry (#636 by @KitsuneRal):
  • AccountRegistry::invokeLogin() is deprecated
  • AccountRegistry::isLoggedIn() actually checks that an account is logged in, not just exists in the registry
  • Connection::assumeIdentity() doesn't use the passed device id any more, taking it from the access token instead
• You can now get the whole list of room account data event types (#638 by @TobiasFella)
• Verification of the user's devices is a bit more reliable thanks to #642 by @TobiasFella
• The library now prefers the correct Base64 algorithm (brought in by MSC3783) for MAC calculation (#640 by @KitsuneRal)

Full Changelog: 0.7.1...0.7.2

0.7.1

Happy New Year! Hot on the heels of 0.7.0, the first bugfix release in 0.7.x series arrives. Most important changes:

• Fix compilation with e2ee on windows (#604 by @TobiasFella)
• To prevent things like the one above from happening CI now covers E2EE on all platforms, not just Linux (#606 by @KitsuneRal)
• Fix SSO flows failing to complete when a client uses a proxy server (#607 by @BLumia)
• Fix exposing QtKeychain dependency to clients (#611 by @TobiasFella and #612 by @KitsuneRal) - particularly nasty in that libQuotient itself builds and installs just fine but an attempt to build a client using certain parts of that installed libQuotient (AccountRegistry, namely) will always fail.

New Contributors

• @BLumia made their first contribution in #607 - many thanks!

Full Changelog: 0.7.0...0.7.1

0.7.0

It took another long wait since 0.6 was released but here we are, the next stable libQuotient is out! As usual, some numbers for statistics - 0.7.0 is getting released 2+ years after 0.6.0 (yikes...) and more than a year since the latest stable release, 0.6.11. Well... there was a reason to that, and it is all the work on E2EE finally getting to beta status! That, together with all the other features, fixes and cleanup, resulted in a monstrous chain of 1100+ commits in Git. See more details below.

This release is the first since a long time with the bus factor for libQuotient becoming more than 1 again. Between 2017 and 2020, I (@KitsuneRal) was the only active developer who knew the library code. Since 2020, the NeoChat leads, @TobiasFella and @CarlSchwan, reimplemented the E2EE foundations in the library that were previously unfinished in 0.6 - learning most of the library code along the way. Not that I plan to retire or anything but it's much better to be in a team. Their contributions are also the biggest that the library has ever seen from a non-project lead, by a large margin. I owe you beverages, guys.

That said, with a really big chunk of work behind further changes will (hopefully!) come in smaller and more rapid batches...

Toolchain updates

Quite a bit of time passed since 0.6, so we've had an opportunity to switch to C++20, now widely adopted. Respectively, the earliest known-to-work compiler versions are GCC 11, Clang 11, MSVC 2019 and Apple Clang 12. Older ones are very unlikely to build the library without considerable patching. CMake (3.16+) is the only build configuration tool supported from now; qmake support has been dropped.

As for Qt, libQuotient 0.7 can be built either with Qt 5 or Qt 6; however, only Qt 5.15 (the only still officially supported) is accepted for the older major version. It is likely that in 0.8 or 0.9 we will stop building with Qt 5 altogether; Qt 6 is already mature and widespread enough, and Qt 5.15 will get less and less attention as The Qt Project collectively moves along.

New dependencies: Qt Keychain (always), Olm and OpenSSL (for E2EE)

Quotient punted the problem of storing secrets (such as access tokens - passwords should never be stored anywhere at all) to clients; both Quaternion and NeoChat used QtKeychain to accomplish that. With introduction of E2EE, there's one more secret to store - a pickling key that is used to decrypt pretty much anything sensitive in the SQLite database that Quotient now has. Therefore, libQuotient is now explicitly and unconditionally dependent on QtKeychain - please make sure it's installed or CMake will complain early on.

When E2EE is switched on, the library will also depend on Olm and OpenSSL. Version 0.6 used QtOlm but this is no more the case; instead, libQuotient incorporated a rewrite of QtOlm pieces and now uses Olm directly. OpenSSL is also necessary to allocate secure heap and use standard crypto algorithms not exposed through Qt, such as AES.

New (optional) submodule: GTAD

Since libQuotient 0.7, you can add and manage GTAD as a submodule within the library's Git tree (#561). Normally you won't need that and even if the submodule is checked out, GTAD building process is intentionally not integrated into the library building in any way. Be mindful that GTAD build requirements may be different from those of the library; as for now, they happen to be the same but there's a high probabilty that GTAD will adopt C++23 (and require newer compilers therefore) before the library does.

Compatibility and maintenance

As before, 0.7 is not backwards compatible with 0.6; fundamentally it's the same API but there are quite a few breaking changes. Micro-versions (0.7.z) will maintain API and ABI compatibility, until 0.8 breaks it again.

With this release, 0.5 becomes officially unmaintained (it didn't receive a single commit for the last couple of years anyway, so it's a pure formality). 0.6 is now on "sustaining support", only receiving critical fixes (e.g. to keep the older clients somehow workable as the Matrix standard changes).

CI changes

After changing their owner, Travis CI abruptly went to a new billing scheme at the end 2020, leaving (non-paying) open-source projects they had long welcomed with no allowance to run builds for more than two months, effectively pushing them to find a new home. Fortunately, GitHub Actions matured just in time and had all the minimally necessary features, so it was adopted for Linux and macOS pipelines (#434). And since GHA also supported Windows as a CI platform, we have migrated from AppVeyor as well (#495) once the migration from Travis CI proved successful. Turns out that Travis CI really lost their integrity under the new owner, suffering from breaches one after another - so, not missing them at all and recommending to move over to anyone who's somehow still with them, paid or not.

To improve on the integrity of our own code, we moved from LGTM - barely maintained after GitHub bought them - to GitHub's own action using the same CodeQL technology LGTM used (cc69883, #495 again). Much bigger static analysis fun came with SonarCloud adoption (originally in 0a46049, with many updates since). The first analysis came up with 1300+ "code smell" warnings; after massaging the Sonar profile the number of warnings halved; and after some massive cleanup effort the number went down to below 300. You can see the whole story at the libQuotient's page in SonarCloud. Another thing brought in with Sonar is coverage tracking (ebabb2f); that triggered me (@KitsuneRal) to actually execute autotests (originally championed by @CarlSchwan) in CI just before running Quotest (#523). The coverage numbers still don't impress: even after adding a few more tests we're still at around 42% so far (and the number will go down if we keep adding things without adding tests - hint, hint!).

Changes

Notable changes since 0.6.11 are listed below; if you want to go through all small things as well, check out the full commit log and the list of closed issues.

E2EE

As mentioned above, E2EE is the biggest part of this release, and @TobiasFella made most of the heavy-lifting. The following parts of E2EE are known to work:

• foundations (#506, #552; additional contributions and refactoring in #550, #556, #571, #581, #582, #589 by @KitsuneRal)
• Olm/Megolm signalling with our parties/devices (#534, #544, #560, #595)
• managing device and one-time keys (#535, #542)
• sending (#540, #572) and receiving (monstrous #477, together with @a-andreyev, @CarlSchwan and @KitsuneRal; #536, #537, #538) new encrypted messages ;
• encrypting/decrypting attachments (#505, #543, #553);
• retrieval and decryption of historical messages keys for which are already in the database (#583, #590)
• device verification (#541, #547, #566, #568, #573, #575, #577; #594 by @nicolasfella)

What is not there yet:

• historical Megolm sessions are not requested from other devices/parties, so you will see quite a few undecryptable messages in rooms with past communication - this might be fixed (if possible without breaking the API) in further 0.7.x releases
• secure server-side storage (SSSS) is not supported at all for now, and will likely require API breakage so will come in 0.8
• soft logout is not supported; this is also a subject of 0.8

Because there's no soft logout the database for a given account is completely reset upon a successful login attempt. This may come as a very unpleasant surprise if you don't keep your login session between client restarts; but this is a necessary trade-off for the current feature set (see #546 for the discussion). Together with the rest of the above not-there-yet list, this means that you MUST NOT use E2EE with libQuotient-backed clients as your only device(s) on the account - there's quite a risk of losing encrypted conversations if anything happens to the database libQuotient keeps key material in, or if you lose your session and have to log in again. Did I mention the whole E2EE functionality is still in beta? Client authors are strongly recommended to show big scary warnings against E2EE for now.

Read markers -> read receipts + fully read markers

First, a bit of history. Originally, Matrix only had a single kind of read marker, m.read; and back in older times clients diverged on when this read marker should be updated. Quaternion and libQuotient (back then libQMatrixClient), for one, took that m.read signified the "fully read" state and the user should scroll back to the marker and read the messages successively in order for the marker to slide over them (or they can force the whole room as "fully read"). Element and some other clients took an approach where merely opening the room updated m.read to the latest currently displayed message. This led to confusion when both kinds of clients were used on the same account and depending on which one displayed the timeline last, m.read was updated differently.

At a certain point the spec received the m.fully_read marker, making it possible for the user to track their progress through the room both ways at the same time. However, libQuotient 0.6 chose to coalesce the two into a single "read marker" to avoid a massive rewrite, late in th...