QtWebEngineProcess.exe blocks Spyder launch in WinPython-64bit-3.5.2.3Qt5

[jondo]

In WinPython-64bit-3.5.2.3Qt5, when I launch Spyder.exe, only an unresponsive frame of the Spyder window becomes visible, and a QtWebEngineProcess.exe uses much CPU:

Killing the QtWebEngineProcess.exe process makes Spyder responsive, but the Help pane stays gray.

This is on Windows 7 Professional 64bit.

[jondo]

I have tested the Qt5 versions of several WinPython releases before, and don't remember ever getting Spyder to open. I attributed this to the Qt5 version still being "under development". Maybe that's still so?

[stonebig]

hum, PyQt5.7.0 is rather stable, I don't experience this "old" PyQt5.6.0 problems.

Ideas:

• you didn't install "Microsoft Visual C++ Redistributable for Visual Studio 2015" ? (maybe with KB 2999226 before). see https://www.visualstudio.com/fr/downloads/
• you didn't wait long enough (let say 3 minutes) ,
• you have an unfriendly anti-virus.

I have no more Windows 7, so I can't help more.

[hiccup7]

I am not having any problem with WinPython-64bit-3.5.2.3Qt5 on Windows 8.1, which I find identical to Windows 7 for Python purposes.

I don't know what will go wrong if you execute spyder.exe without first setting the environment variables used by Spyder (such as HOME and PATH). Better to use spyder.bat (or create your own batch file) to set the environment variables.

I like to install WinPython under C:\Program Files. However, unless I create a custom batch file to change the HOME environment variable, WinPython locates the settings folder as a subfolder of WinPython. Windows then requires that any Python module writing to the settings folder (such as Spyder) runs with administrator privileges. When run with standard user privileges instead, such code could hang.

I noticed with the first run of Spyder in WinPython-64bit-3.5.2.3Qt5, the iPython console hangs/fails. Closing Spyder and rerunning solves the problem.

The version of Spyder included in WinPython-64bit-3.5.2.3Qt5 is a development version (not released). If @stonebig will let us know what date/tag of Spyder is in WinPython-64bit-3.5.2.3Qt5, you could ask the Spyder team for help.

[nsymms]

I can confirm this issue. It's happening to me for new installs on Windows 7 and on a Win Server 2008 R2. The main spyder window won't paint. The app appears to be operating fine otherwise, it just won't display anything.

Killing one of the QtWebEngineProcess will allow the screen to draw some of the other areas, but obviously the process you just killed won't draw anyting. So it looks like some kind of deadlock.

Edit: The install I used is WinPython-64bit-3.5.2.3Qt5

[hiccup7]

@nsymms didn't say whether he/she is using spyder.bat or spyder.exe (directly). If the problem occurs with spyder.bat, here are some things to try (in addition to what I mentioned in my last post):

1. Spyder v3.0.1 from http://www.lfd.uci.edu/~gohlke/pythonlibs/#spyder
2. WinPython-64bit-3.5.2.3 (with Qt4)

[stonebig]

I had these sort of problems, I thought it was a Qt issue but maybe it could be a Windows 7 issue (at that time I was maybe still on Windows 7). In that case, the suggestion of @hiccup7 is the simplest pertinent one.

[nsymms]

@hiccup7 I'm running from the .bat file (via the stub exe in the root folder). I'm not sure what needs to be set to run directly from the main spyder.exe.

• I updated to Spyder v3.0.1 via the whl in your link -- same result as before.
• I installed WinPython-64bit-3.4.4.5Qt5 -- everything works great.
• I installed WinPython-64bit-3.5.2.3 (with Qt4) -- everything works great.

So for now I'm going weigh my options and choose either python 3.4 with Qt5 or 3.5 with Qt4. Actually for my current projects I don't think it matters.

[hiccup7]

@nsymms - you can debug further by running root\scripts\spyder.bat. This would help diagnose whether the problem is with root\spyder.exe.

I am using a custom batch file, which sets up the environment variables and calls root\python-3.5.2.amd64\Scripts\spyder.exe
Besides using Windows 8.1, this is a second difference between my setup and yours.

[nsymms]

Same result running from the .bat file. I also installed the recommended C++ redistributable package to make sure my DLLs were up to date. No change.

[hiccup7]

Another difference for my setup is that I am using MS Visual C++ Redistributable for Visual Studio 2015, update 2. If you downloaded the latest, it is update 3. I seem to remember that @stonebig mentioned a bug with update 3, so I kept update 2.

[hiccup7]

@nsymms - Based on your other test results, I suspect that the problem is with spyder.bat. Here is my custom batch file for invoking Spyder:

@echo off
set WINPYDIR=C:\Program Files\WinPython-3.5.2.3Qt5\python-3.5.2.amd64
set WINPYVER=3.5.2.3Qt5
set HOME=%APPDATA%\WinPython\env3.5.2.3Qt5
set PATH=%WINPYDIR%\Lib\site-packages\PyQt5;%WINPYDIR%\;%WINPYDIR%\DLLs;%WINPYDIR%\Scripts;%WINPYDIR%\..\tools;%PATH%
cd %WINPYDIR%\Scripts
rem force default pyqt5 kit for Spyder
set QT_API=pyqt5
spyder.exe %*

The main reason for my customization is the HOME folder, which I prefer to be in the user data instead of the program folder. I always create a new user data folder for each WinPython environment in Windows Explorer before executing this batch file, which writes to it. Not sure what happens if you don't create the HOME folder first.

My batch file is as simple as possible so I understand any possible failures. It omits non-existent folders in the PATH. But maybe I avoided a bug in the spyder.bat in the WinPython release.

[jondo]

Re: @stonebig's first ideas:

• I already had installed the suggested KB 2999226 and the "Microsoft Visual C++ Redistributable for Visual Studio 2015 (x64) - 14.0.23918" (This is update 2.).
• I have deactivated my Antivirus and waited more than 30 minutes.

My WinPython-64bit-3.5.2.3Qt5 root folder is still in C:\Users\robert\Downloads\, reducing the risk of permission issues.

Opening a console in root\scripts\and running spyder.bat there also does not help.

Creating %APPDATA%\WinPython\env3.5.2.3Qt5 and running @hiccup7's spyder.bat (with my WINPYDIR) also does not help.

WinPython-64bit-3.5.2.3 (with Qt4) works, and I am using it for now.
WinPython-64bit-3.4.4.5Qt5 also works.

[jondo]

So the problem comes from the switch from PyQt5 5.5.1 (in WinPython-64bit-3.4.4.5Qt5) to PyQt5 5.7, where QWebKit is replaced by QtWebEngineWidgets.

I have found a code snippet to test the latter outside of Spyder:

import sys

from PyQt5.QtCore import QUrl
from PyQt5.QtWidgets import QApplication
from PyQt5.QtWebEngineWidgets import QWebEngineView

app = QApplication(sys.argv)
wv = QWebEngineView()
wv.load(QUrl("https://pypi.python.org/pypi/PyQt5"))
wv.show()
app.exec_()

I saved this to a file and launched it in the "WinPython Command Prompt".

D:\temp\QtWebEngineProcess-blocks-Spyder>python QWebEngineTest.py

This reproduced the error! I see the unresponsive frame of a "python" window, a QtWebEngineProcess.exe using CPU, and, most important, error messages:

Several times this block:

Failed to load libEGL (Das angegebene Modul wurde nicht gefunden.)
QWindowsEGLStaticContext::create: Failed to load and resolve libEGL functions
Failed to load opengl32sw.dll (Das angegebene Modul wurde nicht gefunden.)
Failed to load and resolve WGL/OpenGL functions

also

QOpenGLWidget: Failed to create context

and

composeAndFlush: makeCurrent() failed

Does this ring a bell with anyone?

[jondo]

Ok, I found the similar Spyder issue 3332 and a post on the PyQt mailing list about missing files in the PyQt5 5.7 Windows wheels. Will test.

[hiccup7]

@jondo - Your evidence suggests that differences between our graphics cards explains why you see the problem and I don't. In other words, not Windows 7 vs. Windows 8.1. Great find!

[jondo]

My graphics card is an on-board "Intel HD Graphics 3000".

I have installed qt-opensource-windows-x86-msvc2015_64-5.7.0.exe and copied libEGL.dll and libGLESv2.dll fromQt5.7.0\5.7\msvc2015_64\bin\ to WinPython-64bit-3.5.2.3Qt5\python-3.5.2.amd64\Lib\site-packages\PyQt5\Qt\bin\. This gave a black Spyder window, and the following error message with my QWebEngineTest.py:

Failed to create D3D shaders.

QOpenGLShaderProgram::uniformLocation( qt_Matrix ): shader program is not linked

QOpenGLShader::link : Failed to create D3D shaders.

Putting from OpenGL import GL in site-packages\spyder\app\start.py (as in the Spyder issue) did not help Spyder.

I have then downloaded opengl32sw-64.7z, as also described in the Spyder issue, but I see no effect when putting the opengl32sw.dll into python-3.5.2.amd64\ or site-packages\PyQt5\Qt\bin\ :-/

[hiccup7]

My graphics card is an on-board "Intel HD Graphics 4600". Looking at the drivers loaded, I see OpenCL and not OpenGL. Have you tried going to the Intel site for an update to your graphics card driver?

[stonebig]

I remember seeing this issue on Spyder, but the switch to Nvidia and Windows 10 made me forgot it.

[stonebig]

a nice résumé summary of what is to include where in WinPython would be helpfull, in case next PyQt lags by a month

[hiccup7]

Even though half of English words are from French, it is interesting how the meanings of these words in English have diverged over time. I think @stonebig intends "summary" for "résumé" above. In US English, the word "résumé" denotes the same as "curriculum vitae" or "C.V." in French and British English. It is one of the challenges of negotiating the Tower of Babel. :)

[jondo]

I have now updated my graphics card driver, but this did not help. I still see the Failed to create D3D shaders errors and the black Spyder window (both with and without opengl32sw.dll, and both with and without from OpenGL import GL in spyder\app\start.py).

@hiccup7, how do I "look at the drivers loaded", like you did?

@stonebig, I cannot give a summary yet, since I still haven't got Spyder working. But the mentioned post about missing files in the PyQt5 5.7 Windows wheels might be true. Here's another case where it helped. My above comment describes in detail which files I copied from Qt to PyQt5.

[hiccup7]

Control Panel > Device Manager > select among Display adapters > Properties in right-click menu > Driver tab > Driver Details button.

[jondo]

I don't recognize any of the "Driver Details" filenames as connected with OpenCL or OpenGL.
I am planning to test with another graphics card that I'll get soon.

[hiccup7]

There is a new Spyder version released today:
http://www.lfd.uci.edu/~gohlke/pythonlibs/#spyder
Hope it solves the problem.

[nsymms]

The new spyder version didn't help me. However, I did get somewhere. It is definitely related to graphics drivers. My question, however, is why does PyQt5 work with older versions of Python but not 3.5?

My discovery was this: All my problems (and work on this) to date have been on remotely connected Windows computers. Today I actually visited one of those remote workstations and tried the newest versions and they all did in fact work perfectly well on that machine. But they don't render anything if I am connected via remote desktop.

I'll keep fiddling around with this as I get time. Hopefully somebody else can make more progress.

[jondo]

I have seen this problem both remotely connected and locally.

Earlier appearances of the message Failed to create D3D shaders:

• on Stackoverflow, with the suggestion to use a Qt build with native OpenGL;
• Qt bug 44815, whose fix is already in Qt 5.7, though;
• and the most promising: Qt bug 53680 has the hint that updating to the latest DirectX version helped! I will try ...

[hiccup7]

My question, however, is why does PyQt5 work with older versions of Python but not 3.5?

Didn't someone report that the problem happens with PyQt5 5.7 and not 5.5.1?

[stonebig]

After PyQt-5.5.1, riverbanks shifted to a "wheel" format, and there is apparently a learning curve to get it right.

The shift from QWebKit to QtWebEngineWidgets was also apparently a big and painfull task on Windows, and about at the same time.

[jondo]

Running the DirectX End-User Runtime Web Installer has fixed the problem for me!
Both the above test script and Spyder are now running.

@nsymms, can you please test this, too?

@stonebig, to sum it up for you:

• The files libEGL.dll and libGLESv2.dll need to be taken from the Qt5.7.0\5.7\msvc2015_64\bin\ directory of qt-opensource-windows-x86-msvc2015_64-5.7.0.exe and put into WinPython-64bit-3.5.2.3Qt5\python-3.5.2.amd64\Lib\site-packages\PyQt5\Qt\bin\.
  This might be something to be included in the next WinPython release.
• DirectX has to be installed or updated. This might be something to put into the WinPython release notes.

[nsymms]

Fixed.
Using the old DirectX web-installer did nothing for me, but the spyder solution above (#3376 -- adding the DLL), made it work.

[stonebig]

done for next build