This repository has been archived by the owner. It is now read-only.

Towards PhantomJS 2.5 #14458

Open
ariya opened this Issue Aug 8, 2016 · 62 comments

Comments

Projects
None yet
@ariya
Copy link
Owner

ariya commented Aug 8, 2016

Going forward, what will happen in PhantomJS land:

Source Code

  • Use system-installed Qt 5. This is established with pull request #14386 from @Vitallium.
  • Use system-installed QtWebKit. Also part of the above pull request. Ideally of course this is using the latest QtWebKit (see annulen/webkit project from @annulen).
    Compared to vanilla QtWebKit of Qt 5.x, this latest QtWebKit enables lots of new modern web platform features, check Comparison with QtWebKit 5.6 for more details.

Packaging

  • Remove build.py. It won't work anymore with the two above changes. Consequently, building PhantomJS will be similar to building any other Qt-based projects (qmake && make).
    We may need to supply a simple configure script and a convenient make install to make packaging easier for the distribution packagers (Debian/Ubuntu/Fedora/Nix/Homebrew/NuGet/etc). See qconf for details.
  • #14459: Recreate static build packaging. A single executable with virtually no dependency is what makes PhantomJS very popular. This is also important to support installer wrapper such as npm's phantomjs-prebuilt for Node.js, phantomjs gem for Ruby, etc.

Features

We need to ensure that the Linux version can still operate in pure headless mode (without X11).

Regression: it won't be possible to disable web security.
#14461: Regression: there won't be support for printing headers & footers.

@ariya ariya self-assigned this Aug 8, 2016

ariya added a commit that referenced this issue Aug 8, 2016

Use system-installed Qt and QtWebKit (issue #14458):
* Upgrade QtBase module
* Remove QtBase
* Remove QtWebkit
* Make ready for new bleeding edge WebKit
* Use configurable properties
* Move back to old headers

@ariya ariya added this to the Release 2.5 milestone Aug 8, 2016

@ariya

This comment has been minimized.

Copy link
Owner Author

ariya commented Aug 8, 2016

@annulen Is there any news on a distribution packager's willingness to make an official package of QtWebKit TP2 (or later releases)? How about Homebrew @MikeMcQuaid?

This becomes important as it will ensure that a future distribution-packaged PhantomJS does not deviate (get "crippled") regarding its feature sets. For instance, if a distribution contains a PhantomJS 2.5 release built using the outdated Qt 5.2 and its QtWebKit module, it will potentially confuse the users since it will not support all the latest web platform features.

@Vitallium

This comment has been minimized.

Copy link
Collaborator

Vitallium commented Aug 8, 2016

Reference issue: annulen/webkit#115

@moimael

This comment has been minimized.

Copy link

moimael commented Aug 8, 2016

Any plan to port phantomjs to Qt Webengine , since now Qt WebKit will not see new features ?

@annulen

This comment has been minimized.

Copy link

annulen commented Aug 8, 2016

@ariya I think it's even more important to ensure that crippled versions are not distributed at all, so users don't get confused and go straight to official release binaries.

@annulen

This comment has been minimized.

Copy link

annulen commented Aug 8, 2016

@moimael Read the topic plz

@ariya

This comment has been minimized.

Copy link
Owner Author

ariya commented Aug 8, 2016

@moimael Check this post. In all cases, feel free to start a separate discussion (this issue is focusing on our 2.5 effort).

@MikeMcQuaid

This comment has been minimized.

Copy link
Contributor

MikeMcQuaid commented Aug 8, 2016

@annulen Is there any news on a distribution packager's willingness to make an official package of QtWebKit TP2 (or later releases)? How about Homebrew @MikeMcQuaid?

@ariya We'll package it when it's a stable version but are open to having a PR open at any point.

This becomes important as it will ensure that a future distribution-packaged PhantomJS does not deviate (get "crippled") regarding its feature sets. For instance, if a distribution contains a PhantomJS 2.5 release built using the outdated Qt 5.2 and its QtWebKit module, it will potentially confuse the users since it will not support all the latest web platform features.

From a packagers perspective the best way around this would be to check for such things in your configure/CMakeLists.txt/qmake and refuse to build if you're using an outdated version. Similarly, you could consider bumping PhantomJS' version when new Qt 5 releases come out with bumped requirements to note the minimum features that are required.

@ariya

This comment has been minimized.

Copy link
Owner Author

ariya commented Aug 8, 2016

I don't think it's always possible to prevent a modified/tweaked/crippled version of PhantomJS: take a look at Debian package https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=817277.

Not having a bundled Qt/QtWebKit certainly will simplify the packager's life. That said, it won't help much if the available system Qt/QtWebKit is still outdated.

@annulen

This comment has been minimized.

Copy link

annulen commented Aug 8, 2016

In Debian Stable you'll always have QtWebKit version, outdated by a couple of years

@annulen

This comment has been minimized.

Copy link

annulen commented Aug 8, 2016

In other distros (except rolling-release ones) it may be slightly less outdated, but not much

https://blogs.gnome.org/mcatanzaro/2016/02/01/on-webkit-security-updates/

@MikeMcQuaid

This comment has been minimized.

Copy link
Contributor

MikeMcQuaid commented Aug 8, 2016

I don't think it's always possible to prevent a modified/tweaked/crippled version of PhantomJS: take a look at Debian package https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=817277.

Sure. Some package managers (i.e. Homebrew) frown on patching, though, so if you want to require a certain version of a dependency the best way to do that is to ensure you do so in your build system.

@ariya

This comment has been minimized.

Copy link
Owner Author

ariya commented Aug 8, 2016

@annulen That package (https://packages.debian.org/sid/web/phantomjs) is intended for stretch. But you're right, in the near future it will be the next Debian stable.

@ariya

This comment has been minimized.

Copy link
Owner Author

ariya commented Aug 8, 2016

if you want to require a certain version of a dependency the best way to do that is to ensure you do so in your build system.

Solid advice! Thank you.

@annulen

This comment has been minimized.

Copy link

annulen commented Aug 8, 2016

@ariya QtWebKit requires somewhat more work to be suitable for shipping in Stretch than just to be usable by PhantomJS, work is in progress

@jbg77

This comment has been minimized.

Copy link

jbg77 commented Aug 8, 2016

Why no support for printing footer and header ?

@ariya

This comment has been minimized.

Copy link
Owner Author

ariya commented Aug 8, 2016

Why no support for printing footer and header ?

Please track #14461.

@therealsamlin

This comment has been minimized.

Copy link

therealsamlin commented Aug 11, 2016

Quick question, given that build.py is removed now how do we build at the moment?

@ariya

This comment has been minimized.

Copy link
Owner Author

ariya commented Aug 11, 2016

@samlinbris Check the description, i.e. "Consequently, building PhantomJS...".

@therealsamlin

This comment has been minimized.

Copy link

therealsamlin commented Aug 11, 2016

@ariya Yeah, I've seen that bit. As I'm relatively new to this field I don't really know how qmake works. I've downloaded qmake and made it available on my mac. However, I'm not sure what's the next step after this. Surely it's not qmake deploy/build-and-package.sh

@ariya

This comment has been minimized.

Copy link
Owner Author

ariya commented Aug 11, 2016

@samlinbris See if you can familiarize yourself with Qt and/or consult a Qt expert.

@ariya ariya added the In progress label Aug 11, 2016

@annulen

This comment has been minimized.

Copy link

annulen commented Apr 13, 2017

ok, you have --release already so you don't need -DCMAKE_BUILD_TYPE=Release

-DQT_INSTALL_PREFIX and -DQMAKE_EXECUTABLE are not used

Please try building it from scratch after removing WebKitBuild directory

@ktecho

This comment has been minimized.

Copy link

ktecho commented Apr 13, 2017

@ktecho

This comment has been minimized.

Copy link

ktecho commented Apr 13, 2017

Wow... compiling QT in Ubuntu is a mess. Just a few things to help others with some problems I found:

1- In QT 5.8 -no-gtkstyle is no longer supported: ERROR: Unknown command line option -no-gtkstyle.
2- Using -fontconfig -system-freetype don't find the libraries in Ubuntu Yakkety even when they're installed. I just removed both modifiers.
3- Compilation fails with a wayland related problem, even when I specified -skip wayland modifier:
/usr/bin/ld.gold: error: cannot find -lqwayland-generic
4- It doesn't honor -nomake tools modifier, because my compile is failing trying to compile qttools:

make[3]: Entering directory '/home/ktecho/qt-everywhere-opensource-src-5.8.0/qttools/src/qtdiag'
linking ../../bin/qtdiag
/usr/bin/ld.gold: error: cannot find -lqwayland-generic
collect2: error: ld returned 1 exit status
Makefile:83: recipe for target '../../bin/qtdiag' failed
make[3]: *** [../../bin/qtdiag] Error 1

It seems like it's a known problem since Jan/13: https://bugreports.qt.io/browse/QTBUG-28909

In my very-slow test server (same Ubuntu Yakketi), QT5.8 has compiled ok (¿?), but compiling qtwebkit is taking ages. Will report back.

@annulen

This comment has been minimized.

Copy link

annulen commented Apr 13, 2017

You can use official Qt 5.8.0 binaries and Linux binaries of QtWebKit from https://github.com/annulen/webkit/releases/tag/qtwebkit-tp5

@ktecho

This comment has been minimized.

Copy link

ktecho commented Apr 13, 2017

@annulen Thanks a lot for the help. The process is a bit harsh for someone who has never made anything with QT.

For anyone using Ubuntu Yakketi, a version not supported by 2.5beta binaries provided by Vitaly, the best option is to download @nkovacs static binary from here, that works fine: https://github.com/nkovacs/selenium-standalone-phantomjs/tree/11420aaf2174428b56086673232365c820f98a42

@JonathanWylie

This comment has been minimized.

Copy link

JonathanWylie commented Apr 25, 2017

Hi @ariya are there plans to continue with phantomjs after @Vitallium has stepped down?
Or is it really dead in the water now?
Headless chrome seems to use a lot more CPU resources, so want to stick with phantomjs for now.
I have been doing some work on a fix to phantomjs 2.1.1 but not sure if it's worth me making a pull request.

@344389541

This comment has been minimized.

Copy link

344389541 commented Jun 1, 2017

hi. Where is the 2.5.0 version of the code.

@haroon-sheikh

This comment has been minimized.

Copy link
Contributor

haroon-sheikh commented Jun 5, 2017

@344389541 It's in master.

@victorhazbun

This comment has been minimized.

Copy link

victorhazbun commented Jul 6, 2017

When are you gonna release the 2.5.0 version?

@haroon-sheikh

This comment has been minimized.

Copy link
Contributor

haroon-sheikh commented Jul 9, 2017

@victorhazbun @Vitallium is releasing the 2.5 beta v2 this weekend. :)

@haroon-sheikh

This comment has been minimized.

Copy link
Contributor

haroon-sheikh commented Jul 10, 2017

@Vitallium I know you've relased beta 2 for windows. Do you know when we can get the updated beta binaries for linux please?

@sowani

This comment has been minimized.

Copy link

sowani commented Jul 24, 2017

Can somebody please tell me how I can get source code for PhantomJS 2.5? The comments above say that it's there in the master, so is there any specific tag associated with it? I was expecting it to be v2.5 etc., but I am not sure. I want to evaluate 2.5 code along with system installed Qt. Any definitive help in this regard is much appreciated! Thanks.

@ysangkok

This comment has been minimized.

Copy link

ysangkok commented Jul 25, 2017

2.5 is not out yet. The latest is 2.5beta2. If you want the master branch, click the green "Clone or download" on https://github.com/ariya/phantomjs and choose ZIP. There is apparently no tag.

@Vitallium

This comment has been minimized.

Copy link
Collaborator

Vitallium commented Jul 25, 2017

Correct! We tag release after it has been published.

@sowani

This comment has been minimized.

Copy link

sowani commented Jul 25, 2017

@Vitallium

This comment has been minimized.

Copy link
Collaborator

Vitallium commented Jul 25, 2017

@sowani Almost. First of all you need to build new QtWebKit - https://github.com/annulen/webkit/wiki
After that, you can build PJS as usual.

@clarkk

This comment has been minimized.

Copy link

clarkk commented Nov 17, 2017

Any news about the header/footer "issue" in the coming release? Will it be possible to print headers and footers in PDF documents? :)

@annulen

This comment has been minimized.

Copy link

annulen commented Nov 17, 2017

@clarkk Feature would be welcome in QtWebKit if someone prepared a pull request to https://github.com/annulen/webkit with all necessary parts from PhantomJS' fork

@ghost ghost removed the In progress label Jan 10, 2018

@ghost ghost removed this from the Release 2.5 milestone Jan 10, 2018

@dawesi

This comment has been minimized.

Copy link

dawesi commented Jan 22, 2018

Medium.com has declared this project dead...

  1. Phantom JS is dead!
    https://medium.com/@tomastrajan/6-best-practices-pro-tips-for-angular-cli-better-developer-experience-7b328bc9db81

(not that anyone with any sense of software quality or stability uses Chrome for anything)

then again most posts on medium.com are from developers with framework fad attention spans of a can of coke... causing their company to rebuild all their apps every three months to support the latest fad and remove last years fad that is no longer maintained, and for zero net benefit. #sodumb

@tuxayo

This comment has been minimized.

Copy link

tuxayo commented Feb 18, 2018

@dawesi

Medium.com has declared this project dead

Someone on their Medium.com page has declared this project dead...
Otherwise, Facebook.com, Twitter.com, etc declare a lot of things :P

@asowani

This comment has been minimized.

Copy link

asowani commented Feb 19, 2018

@tuxayo Good to hear your comment! So does that imply that phantomjs has a new maintainer and a defined roadmap?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.