New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dropping legacy QtWebKit (and Python <= 3.4, Qt <= 5.6) in v1.0 #2742

Closed
The-Compiler opened this Issue Jun 20, 2017 · 16 comments

Comments

Projects
None yet
7 participants
@The-Compiler
Collaborator

The-Compiler commented Jun 20, 2017

I've partly talked about this before in #1775, but I think with v1.0 it's time to do this.

tl;dr: If you're using legacy QtWebKit, or Python 3.4, or Qt < 5.7, this will probably affect you.

Here is what I'd like to drop, and - unless someone convices me otherwise - probably will with v1.0:

Legacy QtWebKit

First of all: I won't drop QtWebKit completely, only the outdated version currently coming with Qt.

This is probably the biggest one, and to my knowledge it'll affect anyone using QtWebKit on distributions other than Archlinux and Gentoo. Those people won't be able to use qutebrowser anymore.

Unfortunately, people with Python 3.4 won't really have a good upgrade path to either QtWebKit-NG or QtWebEngine.

From what I've seen, this mostly affects Debian Jessie (now oldstable) and Ubuntu 14.04 (Trusty). It'd effectively mean dropping support for those distributions in qutebrowser.

Still I think this is the way to go, even if it hurts. Looking at crash reports, I still see a lot of people using legacy QtWebKit (around 1 report per day in average).

qutebrowser might work fine for those (and not work anymore afterwards) - and while that sucks, those people really should find some way to upgrade. With QtWebKit (especially on Qt < 5.4; see below), qutebrowser crashes quite a lot, and various websites recently stopped working as well (GitHub, Mastodon, Recaptcha/Google captchas, new Google sign-in page come to mind). It's also terrible from a security standpoint, so I think it's time to pull the plug.

The next release (v0.11) will already show a one-time warning when it detects legacy QtWebKit. I hope that gets those numbers down a bit, and gets people to migrate away before dropping support.

Python 3.4

I originally thought nobody using QtWebEngine/QtWebKit-NG would use Python 3.4 anymore, so it'd be possible to get rid of that. That'd mainly mean we could drop qutebrowser.utils.typing and start using type annotations freely, which opens the way for #1456.

Crash reports tell me another story: Lots of Gentoo users (around one report all 2-3 days) still seem to be on Python 3.4. I'd still like to find out the story here - apparently it's easy to upgrade, but nobody does it? Looking at the Gentoo package it looks like Python 3.5 and 3.6 are still in testing (which means Gentoo is now officially more outdated than Debian stable)?!

Qt < 5.6

Again looking at crash reports:

  • Qt 5.2: 0-2 per month, last one a month ago
  • Qt 5.3: 0-4 per month, same
  • Qt 5.4: one all 2 months or so, last one 3 months ago
  • Qt 5.5: around one all 2-3 days. Looking at OS, this is the Windows build (which will be updated with the next release), Ubuntu Xenial (which will be affected by the legacy QtWebKit drop above), and stuff based on that (elementary OS 0.4.1, Linux Mint 18.1). I can't find a single report using QtWebKit-NG or QtWebEngine with that, so it seems safe to drop it.
  • Qt 5.6: Looks like this is the default on Gentoo. That means it'd probably make the live of Gentoo users easier as they can only upgrade to QtWebKit-NG and not Qt (which might affect other things on their system). That being said, I can't find any reports of people doing that right now, but I only have 1 crash report of someone using QtWebKit-NG on Gentoo. Should I drop it?
  • Qt 5.7: Default on Debian Stretch, definitely going to stay.

As for the benefits, see #1775.

Legacy PyYAML

See #2745 - I don't think anything not affected by the above is affected by this though.

@ckeen

This comment has been minimized.

Show comment
Hide comment
@ckeen

ckeen Jun 20, 2017

As I see it, people with QtWebKit distributions are stuck with that either way, regardless of upstream support. A lot of the bugs causing the crashes cannot be fixed in qutebrowser anyway since they are due to webkit limitations. So moving forward seems like the way to go.

ckeen commented Jun 20, 2017

As I see it, people with QtWebKit distributions are stuck with that either way, regardless of upstream support. A lot of the bugs causing the crashes cannot be fixed in qutebrowser anyway since they are due to webkit limitations. So moving forward seems like the way to go.

@jgkamat

This comment has been minimized.

Show comment
Hide comment
@jgkamat

jgkamat Jun 20, 2017

Collaborator

For gentoo:

Here's the bug for stabilizing python 3.5. Hopefully they'll get it to stable by the time 1.0 is released.

Upgrading qt to 5.7 in gentoo is a pain, but it is rock solid right now. I hope it gets stabilized quickly, but I'm not holding my breath.

AFAIK, you can use type annotations in python3.4, they just won't actually do anything:

[jay@laythe ~]$ py 
Python 3.4.5 (default, Apr 18 2017, 00:45:03) 
[GCC 5.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from typing import Union
>>> 

EDIT: never mind, I had typing installed via pip. Installing dev-python/typing has the same effect as well.

Maybe support for annotations were added in a minor release (and in that case, you could support something like python > 3.4.5

I'm 👍 on dropping legacy webkit, it's very buggy and a bad experience overall. I think it's fair to let those people use an older version of qb, esp if it's easy to upgrade to webkit-ng on their platform.

Collaborator

jgkamat commented Jun 20, 2017

For gentoo:

Here's the bug for stabilizing python 3.5. Hopefully they'll get it to stable by the time 1.0 is released.

Upgrading qt to 5.7 in gentoo is a pain, but it is rock solid right now. I hope it gets stabilized quickly, but I'm not holding my breath.

AFAIK, you can use type annotations in python3.4, they just won't actually do anything:

[jay@laythe ~]$ py 
Python 3.4.5 (default, Apr 18 2017, 00:45:03) 
[GCC 5.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from typing import Union
>>> 

EDIT: never mind, I had typing installed via pip. Installing dev-python/typing has the same effect as well.

Maybe support for annotations were added in a minor release (and in that case, you could support something like python > 3.4.5

I'm 👍 on dropping legacy webkit, it's very buggy and a bad experience overall. I think it's fair to let those people use an older version of qb, esp if it's easy to upgrade to webkit-ng on their platform.

@The-Compiler

This comment has been minimized.

Show comment
Hide comment
@The-Compiler

The-Compiler Jun 20, 2017

Collaborator

Looks like there's a stabilization request for Qt 5.7.1 in Gentoo too, FWIW.

The main reason I'd like to drop Qt 5.6 and Python 3.4 isn't really code - I don't think there are many changes there (as long as typing is installed). The only ones coming to mind are #2038 (comment), #1736 and #2106 (plus a workaround or two for 5.6 bugs).

The reason is that I don't know how to test it easily on Travis. The only distribution I've seen using Qt 5.6 (and not older) with Python 3.4 so far is Gentoo. I can't easily build a docker image of that on Travis because I presume not everything is available as a binary package (if anything?), and Travis' build timeout is 45mins IIRC.

And without any testing, I can almost guarantee it'll break sooner or later, so I'd rather not support it at all in that case.

Collaborator

The-Compiler commented Jun 20, 2017

Looks like there's a stabilization request for Qt 5.7.1 in Gentoo too, FWIW.

The main reason I'd like to drop Qt 5.6 and Python 3.4 isn't really code - I don't think there are many changes there (as long as typing is installed). The only ones coming to mind are #2038 (comment), #1736 and #2106 (plus a workaround or two for 5.6 bugs).

The reason is that I don't know how to test it easily on Travis. The only distribution I've seen using Qt 5.6 (and not older) with Python 3.4 so far is Gentoo. I can't easily build a docker image of that on Travis because I presume not everything is available as a binary package (if anything?), and Travis' build timeout is 45mins IIRC.

And without any testing, I can almost guarantee it'll break sooner or later, so I'd rather not support it at all in that case.

@The-Compiler The-Compiler changed the title from Dropping legacy QtWebKit (and Python < 3.4, Qt < 5.6) in v1.0 to Dropping legacy QtWebKit (and Python <= 3.4, Qt <= 5.6) in v1.0 Jun 20, 2017

@jgkamat

This comment has been minimized.

Show comment
Hide comment
@jgkamat

jgkamat Jun 20, 2017

Collaborator

Also, opensuse's latest stable version is on qt 5.6 and python 3.4. I think that they are currently trying to stabilize a new version though.

Collaborator

jgkamat commented Jun 20, 2017

Also, opensuse's latest stable version is on qt 5.6 and python 3.4. I think that they are currently trying to stabilize a new version though.

@The-Compiler

This comment has been minimized.

Show comment
Hide comment
@The-Compiler

The-Compiler Jun 20, 2017

Collaborator

qutebrowser already requires Qt 5.7 for QtWebEngine though (and I don't think I can easily extend that to 5.6, it's just too much missing there). Since I'm assuming OpenSUSE doesn't package QtWebKit-NG, that'd already be affected by the "no legacy QtWebKit" change...

(edit: see a86170f for what was dropped when dropping Qt 5.6 support for QtWebEngine)

Collaborator

The-Compiler commented Jun 20, 2017

qutebrowser already requires Qt 5.7 for QtWebEngine though (and I don't think I can easily extend that to 5.6, it's just too much missing there). Since I'm assuming OpenSUSE doesn't package QtWebKit-NG, that'd already be affected by the "no legacy QtWebKit" change...

(edit: see a86170f for what was dropped when dropping Qt 5.6 support for QtWebEngine)

@The-Compiler

This comment has been minimized.

Show comment
Hide comment
@The-Compiler

The-Compiler Jun 20, 2017

Collaborator

Also see #2745 - this should't affect anyone not affected by the above, but if you're using something other than Archlinux, please do a quick check whether that's packaged (as ruamel.yaml or ruamel-yaml mostly) for your distribution and let me know if you can't find it.

Collaborator

The-Compiler commented Jun 20, 2017

Also see #2745 - this should't affect anyone not affected by the above, but if you're using something other than Archlinux, please do a quick check whether that's packaged (as ruamel.yaml or ruamel-yaml mostly) for your distribution and let me know if you can't find it.

@1-61803

This comment has been minimized.

Show comment
Hide comment
@1-61803

1-61803 Jun 22, 2017

Could you make sort of a table with the versions of each dependency for each qutebrowser release, past and future?

1-61803 commented Jun 22, 2017

Could you make sort of a table with the versions of each dependency for each qutebrowser release, past and future?

@blyxxyz

This comment has been minimized.

Show comment
Hide comment
@blyxxyz

blyxxyz Jun 25, 2017

Contributor

QtWebEngine doesn't work on one of my systems (running Debian Unstable) because of Nouveau, so it would become impossible to run qutebrowser without relying on the QtWebkit from http://repo.paretje.be/. That's not a huge issue because that version is better anyway, but it would stop me from running qutebrowser with just dependencies from the Debian repositories.
Another system I run qutebrowser on is 32-bit, so it can't use the @paretje package, because it's only for amd64. Luckily, it can handle QtWebEngine, but it would be very inconvenient otherwise.
Dropping legacy QtWebkit might be worth it, but it'll cause problems even for some people not running outdated distros.

Contributor

blyxxyz commented Jun 25, 2017

QtWebEngine doesn't work on one of my systems (running Debian Unstable) because of Nouveau, so it would become impossible to run qutebrowser without relying on the QtWebkit from http://repo.paretje.be/. That's not a huge issue because that version is better anyway, but it would stop me from running qutebrowser with just dependencies from the Debian repositories.
Another system I run qutebrowser on is 32-bit, so it can't use the @paretje package, because it's only for amd64. Luckily, it can handle QtWebEngine, but it would be very inconvenient otherwise.
Dropping legacy QtWebkit might be worth it, but it'll cause problems even for some people not running outdated distros.

@The-Compiler

This comment has been minimized.

Show comment
Hide comment
@The-Compiler

The-Compiler Jun 25, 2017

Collaborator

@1-61803 No need for a table I think - see the readme for the current dependencies. PyYAML was added in v0.2 I think, other than that (and QtWebEngine related things) there never were any big changes.

What'd change with this:

  • Any QtWebKit since Qt 5.2 -> only the revived QtWebKit
  • Qt/PyQt 5.2 or newer -> 5.6 (or maybe 5.7) and newer
  • Python 3.4 or newer -> 3.5 or newer

@blyxxyz Oh, thanks for letting me know - I wasn't aware the package is only available for 64bit.

I don't think this is something which should hold me back though, I'm afraid 😆

FWIW you should still be able to use QtWebEngine with export LIBGL_ALWAYS_SOFTWARE=1 - whether the performance is still usable seems to depend on your hardware, though.

Collaborator

The-Compiler commented Jun 25, 2017

@1-61803 No need for a table I think - see the readme for the current dependencies. PyYAML was added in v0.2 I think, other than that (and QtWebEngine related things) there never were any big changes.

What'd change with this:

  • Any QtWebKit since Qt 5.2 -> only the revived QtWebKit
  • Qt/PyQt 5.2 or newer -> 5.6 (or maybe 5.7) and newer
  • Python 3.4 or newer -> 3.5 or newer

@blyxxyz Oh, thanks for letting me know - I wasn't aware the package is only available for 64bit.

I don't think this is something which should hold me back though, I'm afraid 😆

FWIW you should still be able to use QtWebEngine with export LIBGL_ALWAYS_SOFTWARE=1 - whether the performance is still usable seems to depend on your hardware, though.

@The-Compiler

This comment has been minimized.

Show comment
Hide comment
@The-Compiler

The-Compiler Jul 3, 2017

Collaborator

@jgkamat FWIW, I will keep Python 3.4 support if and only if:

  • Gentoo still didn't stabilize 3.4 until then
  • Someone (e.g. you 😉) comes up with a Dockerfile which allows me to test this on Travis and which requires less than 50 minutes to build on Travis (ideally, less than 10 or so). See the existing files for inspiration.

I tried getting something to run with Sabayon (which comes with binary packages), and after some struggling I almost got it to work - then I noticed it builds PyQt without the webengine useflag. It might be possible to install Qt from binaries with equo (their Entropy package manager with binary packages) and then build PyQt from source against it (which should take some 20min or so) - but I'm not going to try, I have no idea how Gentoo works.

Collaborator

The-Compiler commented Jul 3, 2017

@jgkamat FWIW, I will keep Python 3.4 support if and only if:

  • Gentoo still didn't stabilize 3.4 until then
  • Someone (e.g. you 😉) comes up with a Dockerfile which allows me to test this on Travis and which requires less than 50 minutes to build on Travis (ideally, less than 10 or so). See the existing files for inspiration.

I tried getting something to run with Sabayon (which comes with binary packages), and after some struggling I almost got it to work - then I noticed it builds PyQt without the webengine useflag. It might be possible to install Qt from binaries with equo (their Entropy package manager with binary packages) and then build PyQt from source against it (which should take some 20min or so) - but I'm not going to try, I have no idea how Gentoo works.

@The-Compiler

This comment has been minimized.

Show comment
Hide comment
@The-Compiler

The-Compiler Jul 4, 2017

Collaborator

On a second thought, Python 3.5 has another big benefit: It's much better when resolving circular imports. We could probably get rid of most imports which are inside functions when switching to it...

Collaborator

The-Compiler commented Jul 4, 2017

On a second thought, Python 3.5 has another big benefit: It's much better when resolving circular imports. We could probably get rid of most imports which are inside functions when switching to it...

@The-Compiler

This comment has been minimized.

Show comment
Hide comment
@The-Compiler

The-Compiler Jul 4, 2017

Collaborator

And another update I haven't thought about yet:

tox < 2.3.1

Looking at:

Fedora 26, Ubuntu 16.04 and some others use 2.3.1, but we can drop everything that's older. (note that the 0.x versions are probably the messenger application "tox", not the Python thing)

See the tox changelog - this will help a lot with #2341.

Collaborator

The-Compiler commented Jul 4, 2017

And another update I haven't thought about yet:

tox < 2.3.1

Looking at:

Fedora 26, Ubuntu 16.04 and some others use 2.3.1, but we can drop everything that's older. (note that the 0.x versions are probably the messenger application "tox", not the Python thing)

See the tox changelog - this will help a lot with #2341.

@The-Compiler The-Compiler referenced this issue Jul 4, 2017

Closed

Travis upgrades for v1.0 #2780

4 of 7 tasks complete
@lupinix

This comment has been minimized.

Show comment
Hide comment
@lupinix

lupinix Jul 12, 2017

For Fedora this should be fine, Fedora 24 (which ships Qt 5.6) is end of life in one month. Newer Fedora releases match the new criteria.

lupinix commented Jul 12, 2017

For Fedora this should be fine, Fedora 24 (which ships Qt 5.6) is end of life in one month. Newer Fedora releases match the new criteria.

@The-Compiler The-Compiler referenced this issue Jul 12, 2017

Merged

qutebrowser: 0.10.1 -> 0.11.0 #27193

6 of 8 tasks complete
@The-Compiler

This comment has been minimized.

Show comment
Hide comment
@The-Compiler

The-Compiler Jul 21, 2017

Collaborator

Cleanups to be done after dropping support

Python 3.5

  • Drop qutebrowser.utils.typing
  • Move imports in functions to the top of modules where possible
  • ...

Legacy QtWebKit

  • Drop qtutils.is_qtwebkit_ng()
  • Drop legacy WebKit history reading
  • Drop HTML history page
  • Various tests only running with legacy QtWebKit
  • ...

Old Qt versions

  • Drop lots and lots of workarounds (grep for WORKAROUND)
  • Drop PyQt/Qt conditional code (grep for version_check, VERSION)
  • Drop TypeError in except (RuntimeError, TypeError): for deleted objects
  • Drop @pyqtSlot annotations maybe?
  • ...
Collaborator

The-Compiler commented Jul 21, 2017

Cleanups to be done after dropping support

Python 3.5

  • Drop qutebrowser.utils.typing
  • Move imports in functions to the top of modules where possible
  • ...

Legacy QtWebKit

  • Drop qtutils.is_qtwebkit_ng()
  • Drop legacy WebKit history reading
  • Drop HTML history page
  • Various tests only running with legacy QtWebKit
  • ...

Old Qt versions

  • Drop lots and lots of workarounds (grep for WORKAROUND)
  • Drop PyQt/Qt conditional code (grep for version_check, VERSION)
  • Drop TypeError in except (RuntimeError, TypeError): for deleted objects
  • Drop @pyqtSlot annotations maybe?
  • ...
@abbradar

This comment has been minimized.

Show comment
Hide comment
@abbradar

abbradar Jul 26, 2017

Contributor

Hi! From NixOS side:

  1. We already use newer Python (3.6) and Qt 5 (5.9) by default and have no problem in updating various Python packages if needed (haven't checked closely on them).
  2. (Personally) all for dropping QtWebKit. I'll see into packaging QtWebKit-NG later; for now I think we already have QtWebEngine by default on NixOS so it's okay.
Contributor

abbradar commented Jul 26, 2017

Hi! From NixOS side:

  1. We already use newer Python (3.6) and Qt 5 (5.9) by default and have no problem in updating various Python packages if needed (haven't checked closely on them).
  2. (Personally) all for dropping QtWebKit. I'll see into packaging QtWebKit-NG later; for now I think we already have QtWebEngine by default on NixOS so it's okay.

The-Compiler added a commit that referenced this issue Sep 18, 2017

The-Compiler added a commit that referenced this issue Sep 18, 2017

The-Compiler added a commit that referenced this issue Sep 18, 2017

The-Compiler added a commit that referenced this issue Sep 18, 2017

The-Compiler added a commit that referenced this issue Sep 18, 2017

The-Compiler added a commit that referenced this issue Sep 18, 2017

The-Compiler added a commit that referenced this issue Sep 18, 2017

The-Compiler added a commit that referenced this issue Sep 18, 2017

The-Compiler added a commit that referenced this issue Sep 18, 2017

The-Compiler added a commit that referenced this issue Sep 18, 2017

@The-Compiler

This comment has been minimized.

Show comment
Hide comment
@The-Compiler

The-Compiler Sep 18, 2017

Collaborator

Python < 3.5, Qt < 5.7.1, PyQt < 5.7 and legacy QtWebKit are now dropped in master, resulting in about 1000 lines less code! 🎉

Collaborator

The-Compiler commented Sep 18, 2017

Python < 3.5, Qt < 5.7.1, PyQt < 5.7 and legacy QtWebKit are now dropped in master, resulting in about 1000 lines less code! 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment