Dropping legacy QtWebKit (and Python <= 3.4, Qt <= 5.6) in v1.0

[The-Compiler]

I've partly talked about this before in #1775, but I think with v1.0 it's time to do this.

tl;dr: If you're using legacy QtWebKit, or Python 3.4, or Qt < 5.7, this will probably affect you.

Here is what I'd like to drop, and - unless someone convices me otherwise - probably will with v1.0:

Legacy QtWebKit

First of all: I won't drop QtWebKit completely, only the outdated version currently coming with Qt.

This is probably the biggest one, and to my knowledge it'll affect anyone using QtWebKit on distributions other than Archlinux and Gentoo. Those people won't be able to use qutebrowser anymore.

Unfortunately, people with Python 3.4 won't really have a good upgrade path to either QtWebKit-NG or QtWebEngine.

From what I've seen, this mostly affects Debian Jessie (now oldstable) and Ubuntu 14.04 (Trusty). It'd effectively mean dropping support for those distributions in qutebrowser.

Still I think this is the way to go, even if it hurts. Looking at crash reports, I still see a lot of people using legacy QtWebKit (around 1 report per day in average).

qutebrowser might work fine for those (and not work anymore afterwards) - and while that sucks, those people really should find some way to upgrade. With QtWebKit (especially on Qt < 5.4; see below), qutebrowser crashes quite a lot, and various websites recently stopped working as well (GitHub, Mastodon, Recaptcha/Google captchas, new Google sign-in page come to mind). It's also terrible from a security standpoint, so I think it's time to pull the plug.

The next release (v0.11) will already show a one-time warning when it detects legacy QtWebKit. I hope that gets those numbers down a bit, and gets people to migrate away before dropping support.

Python 3.4

I originally thought nobody using QtWebEngine/QtWebKit-NG would use Python 3.4 anymore, so it'd be possible to get rid of that. That'd mainly mean we could drop qutebrowser.utils.typing and start using type annotations freely, which opens the way for #1456.

Crash reports tell me another story: Lots of Gentoo users (around one report all 2-3 days) still seem to be on Python 3.4. I'd still like to find out the story here - apparently it's easy to upgrade, but nobody does it? Looking at the Gentoo package it looks like Python 3.5 and 3.6 are still in testing (which means Gentoo is now officially more outdated than Debian stable)?!

Qt < 5.6

Again looking at crash reports:

• Qt 5.2: 0-2 per month, last one a month ago
• Qt 5.3: 0-4 per month, same
• Qt 5.4: one all 2 months or so, last one 3 months ago
• Qt 5.5: around one all 2-3 days. Looking at OS, this is the Windows build (which will be updated with the next release), Ubuntu Xenial (which will be affected by the legacy QtWebKit drop above), and stuff based on that (elementary OS 0.4.1, Linux Mint 18.1). I can't find a single report using QtWebKit-NG or QtWebEngine with that, so it seems safe to drop it.
• Qt 5.6: Looks like this is the default on Gentoo. That means it'd probably make the live of Gentoo users easier as they can only upgrade to QtWebKit-NG and not Qt (which might affect other things on their system). That being said, I can't find any reports of people doing that right now, but I only have 1 crash report of someone using QtWebKit-NG on Gentoo. Should I drop it?
• Qt 5.7: Default on Debian Stretch, definitely going to stay.

As for the benefits, see #1775.

Legacy PyYAML

See #2745 - I don't think anything not affected by the above is affected by this though.

[ckeen]

As I see it, people with QtWebKit distributions are stuck with that either way, regardless of upstream support. A lot of the bugs causing the crashes cannot be fixed in qutebrowser anyway since they are due to webkit limitations. So moving forward seems like the way to go.

[jgkamat]

For gentoo:

Here's the bug for stabilizing python 3.5. Hopefully they'll get it to stable by the time 1.0 is released.

Upgrading qt to 5.7 in gentoo is a pain, but it is rock solid right now. I hope it gets stabilized quickly, but I'm not holding my breath.

AFAIK, you can use type annotations in python3.4, they just won't actually do anything:

[jay@laythe ~]$ py 
Python 3.4.5 (default, Apr 18 2017, 00:45:03) 
[GCC 5.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from typing import Union
>>> 

EDIT: never mind, I had typing installed via pip. Installing dev-python/typing has the same effect as well.

Maybe support for annotations were added in a minor release (and in that case, you could support something like python > 3.4.5

I'm 👍 on dropping legacy webkit, it's very buggy and a bad experience overall. I think it's fair to let those people use an older version of qb, esp if it's easy to upgrade to webkit-ng on their platform.

[The-Compiler]

Looks like there's a stabilization request for Qt 5.7.1 in Gentoo too, FWIW.

The main reason I'd like to drop Qt 5.6 and Python 3.4 isn't really code - I don't think there are many changes there (as long as typing is installed). The only ones coming to mind are #2038 (comment), #1736 and #2106 (plus a workaround or two for 5.6 bugs).

The reason is that I don't know how to test it easily on Travis. The only distribution I've seen using Qt 5.6 (and not older) with Python 3.4 so far is Gentoo. I can't easily build a docker image of that on Travis because I presume not everything is available as a binary package (if anything?), and Travis' build timeout is 45mins IIRC.

And without any testing, I can almost guarantee it'll break sooner or later, so I'd rather not support it at all in that case.

[jgkamat]

Also, opensuse's latest stable version is on qt 5.6 and python 3.4. I think that they are currently trying to stabilize a new version though.

[The-Compiler]

qutebrowser already requires Qt 5.7 for QtWebEngine though (and I don't think I can easily extend that to 5.6, it's just too much missing there). Since I'm assuming OpenSUSE doesn't package QtWebKit-NG, that'd already be affected by the "no legacy QtWebKit" change...

(edit: see a86170f for what was dropped when dropping Qt 5.6 support for QtWebEngine)

[The-Compiler]

Also see #2745 - this should't affect anyone not affected by the above, but if you're using something other than Archlinux, please do a quick check whether that's packaged (as ruamel.yaml or ruamel-yaml mostly) for your distribution and let me know if you can't find it.

[1-61803]

Could you make sort of a table with the versions of each dependency for each qutebrowser release, past and future?

[blyxxyz]

QtWebEngine doesn't work on one of my systems (running Debian Unstable) because of Nouveau, so it would become impossible to run qutebrowser without relying on the QtWebkit from http://repo.paretje.be/. That's not a huge issue because that version is better anyway, but it would stop me from running qutebrowser with just dependencies from the Debian repositories.
Another system I run qutebrowser on is 32-bit, so it can't use the @paretje package, because it's only for amd64. Luckily, it can handle QtWebEngine, but it would be very inconvenient otherwise.
Dropping legacy QtWebkit might be worth it, but it'll cause problems even for some people not running outdated distros.

[The-Compiler]

@1-61803 No need for a table I think - see the readme for the current dependencies. PyYAML was added in v0.2 I think, other than that (and QtWebEngine related things) there never were any big changes.

What'd change with this:

• Any QtWebKit since Qt 5.2 -> only the revived QtWebKit
• Qt/PyQt 5.2 or newer -> 5.6 (or maybe 5.7) and newer
• Python 3.4 or newer -> 3.5 or newer

---

@blyxxyz Oh, thanks for letting me know - I wasn't aware the package is only available for 64bit.

I don't think this is something which should hold me back though, I'm afraid 😆

FWIW you should still be able to use QtWebEngine with export LIBGL_ALWAYS_SOFTWARE=1 - whether the performance is still usable seems to depend on your hardware, though.

[The-Compiler]

@jgkamat FWIW, I will keep Python 3.4 support if and only if:

• Gentoo still didn't stabilize 3.4 until then
• Someone (e.g. you 😉) comes up with a Dockerfile which allows me to test this on Travis and which requires less than 50 minutes to build on Travis (ideally, less than 10 or so). See the existing files for inspiration.

I tried getting something to run with Sabayon (which comes with binary packages), and after some struggling I almost got it to work - then I noticed it builds PyQt without the webengine useflag. It might be possible to install Qt from binaries with equo (their Entropy package manager with binary packages) and then build PyQt from source against it (which should take some 20min or so) - but I'm not going to try, I have no idea how Gentoo works.

[The-Compiler]

On a second thought, Python 3.5 has another big benefit: It's much better when resolving circular imports. We could probably get rid of most imports which are inside functions when switching to it...

[The-Compiler]

And another update I haven't thought about yet:

tox < 2.3.1

Looking at:

• https://repology.org/metapackage/python:tox/versions
• https://repology.org/metapackage/tox/versions

Fedora 26, Ubuntu 16.04 and some others use 2.3.1, but we can drop everything that's older. (note that the 0.x versions are probably the messenger application "tox", not the Python thing)

See the tox changelog - this will help a lot with #2341.

[lupinix]

For Fedora this should be fine, Fedora 24 (which ships Qt 5.6) is end of life in one month. Newer Fedora releases match the new criteria.

[The-Compiler]

Cleanups to be done after dropping support

Python 3.5

• Drop qutebrowser.utils.typing
• Move imports in functions to the top of modules where possible
• ...

Legacy QtWebKit

• Drop qtutils.is_qtwebkit_ng()
• Drop legacy WebKit history reading
• Drop HTML history page
• Various tests only running with legacy QtWebKit
• ...

Old Qt versions

• Drop lots and lots of workarounds (grep for WORKAROUND)
• Drop PyQt/Qt conditional code (grep for version_check, VERSION)
• Drop TypeError in except (RuntimeError, TypeError): for deleted objects
• Drop @pyqtSlot annotations maybe?
• ...

[abbradar]

Hi! From NixOS side:

1. We already use newer Python (3.6) and Qt 5 (5.9) by default and have no problem in updating various Python packages if needed (haven't checked closely on them).
2. (Personally) all for dropping QtWebKit. I'll see into packaging QtWebKit-NG later; for now I think we already have QtWebEngine by default on NixOS so it's okay.

[The-Compiler]

Python < 3.5, Qt < 5.7.1, PyQt < 5.7 and legacy QtWebKit are now dropped in master, resulting in about 1000 lines less code! 🎉