Event loop issues with IPython 0.12 and PyQt4 (QDialog.exec_ and more)

[denisri]

Using IPython 0.12 (or 0.12.1) and PyQt4 (or even Qt widgets compiled in C++ run from IPython), I get some strange behaviours of the event loop. Typically, QFileDialog.exec_() returns immediately when exec_ is called (and returns 0).
I also get the main event loop quitting (and finishing with a segfault) in some Qt event processing, but I could not reproduce this problem in a simple program.
For the dialog problem, I have found someone else having the same problem:
http://stackoverflow.com/questions/10683594/ipython-0-12-pyqt4-qdialog-qmenu-exec-does-not-block-returns-immedi
Strangely, the problem does not show up if the QFileDialog is built directly from the Ipython shell, but only when it is built from within a Qt slot (for instance a menu callback).
I have this problem on Linux (Ubuntu 10.04, using system python 2.6.5, system Qt 4.6.2, system PyQt 4.7.2, and a custom IPython install).
I do not get the same problem using IPython 0.11.
Here is a small program example displaying the problem. It can be run from a ipython shell (ipython --gui=qt):

from PyQt4 import QtCore, QtGui

class CustomWidget( QtGui.QMainWindow ):
    def __init__( self, parent=None, flags=0 ):
        QtGui.QMainWindow.__init__( self, parent, QtCore.Qt.WindowFlags( flags ) )
        menu = self.menuBar()
        ac = QtGui.QAction( 'open', self )
        m = menu.addMenu( 'File' )
        m.addAction( ac )
        ac.triggered.connect( self.openFile )

  def openFile( self, checked=False ):
      print 'openFile'
      dialog=QtGui.QFileDialog(self,'','.')
      f=dialog.exec_()
      # returns immediately, the dialog doesn't even show up (or subliminally)
      print 'file dialog result:', f

w = CustomWidget( None, QtCore.Qt.WA_DeleteOnClose )
w.show()

# here, the dialog works as expected
#dialog=QtGui.QFileDialog(w,'','.')
#f=dialog.exec_()
#print 'file dialog result from main shell:', f

[Carreau]

Did you try instanciating QApplication before ?

Note that you might need to do something like :

 app=QtGui.QApplication.instance()  # checks if QApplication already exists
 if not app:    # create QApplication if it doesnt exist
       app = QtGui.QApplication(sys.argv)

as PyQt/PySide do not clear everything at application exit.

[denisri]

Yes I did, but as in this example I was running the example from ipython --gui=qt, the QApplication is already instantiated by IPython, and the event loop is already running.

[fperez]

@jdmarch, you guys have the most in-house Qt expertise of the team; any thoughts on this one?

[denisri]

Two precisions:
I also see the same problem with the new IPython 0.13.
And it also happens on the Mac version of IPython 0.12 (also with python 2.6 and Qt 4.6).

[fperez]

Bummer, I suspect this is going to be a tough one to fix. These event loop bugs are typically very hard to track down. Please post any additional insights you may have, as I'm not optimistic about fast progress on this one.

[jdmarch]

Sorry, we have not had time to look at this.

[fperez]

No worries, I don't expect it will be easy :)

[bfroehle]

Thanks for the test script. git bisect puts the blame on 76f9a74 which as part of #815.

76f9a74750ecfe0325aa2d3213b8720b7c584936 is the first bad commit
commit 76f9a74750ecfe0325aa2d3213b8720b7c584936
Author: Christian Boos <cboos@bct-technology.com>
Date:   Thu Sep 22 09:59:36 2011 +0200

    inputhook: make PyQt4 plays nicer with pyreadline

    PyQt4's own input hook is not really appropriate when PyReadline is active
    (and possibly other readline packages running the PyOS_InputHook repeatedly).
    Arguably this should be fixed in PyQt4 itself, but in the meantime we can
    install a local hook which does the Qt event loop processing in a less
    aggressive way.

[denisri]

OK, so (without looking very thoroughly), I guess it's the "app.quit" called from a timer, which probably quits all event loops, including local event loops of modal dialogs. Then restarting with "app.exec_()" does not restore local loops where they were, but only the main loop.
Well, it's just my interpretation...
So if I'm right, it probably needs something softer than app.quit()

[bfroehle]

I would expect that the Python-implemented input hook should mostly mimic the C-level input hook, except with our tweaks regarding waiting keyboard input and reduced timeouts.

However reading through python-qt4/sip/QtCore/qcoreapplication.sip, it is clear that we are only implementing the Q_OS_WIN32 version of the input hook.

For example, compare

static int qtcore_input_hook()
{
    QCoreApplication *app = QCoreApplication::instance();

    if (app && app->thread() == QThread::currentThread())
    {
#if defined(Q_OS_WIN32)
        QTimer timer;
        QObject::connect(&timer, SIGNAL(timeout()), app, SLOT(quit()));

        do
        {
            timer.start(100);
            QCoreApplication::exec();
            timer.stop();
        }
        while (!_kbhit());

        QObject::disconnect(&timer, SIGNAL(timeout()), app, SLOT(quit()));
#else
        QSocketNotifier notifier(0, QSocketNotifier::Read, 0);
        QObject::connect(&notifier, SIGNAL(activated(int)), app, SLOT(quit()));
        QCoreApplication::exec();
        QObject::disconnect(&notifier, SIGNAL(activated(int)), app, SLOT(quit()));
#endif
    }

    return 0;
}

with https://github.com/ipython/ipython/blob/rel-0.13/IPython/lib/inputhookqt4.py#L88

[bfroehle]

I've mocked up what I think may be a fix in bfroehle/ipython@inputhook_qt4.

In Win32 I've kept the original code, but added timer.timeout.disconnect(app.quit) (which I guess probably wasn't necessary?).

This seems to address the issue in my Ubuntu 12.04 installation, but I've only done limited testing. You can read some background on the whole readline inputhook problem at http://www.riverbankcomputing.com/pipermail/pyqt/2007-July/016512.html

[denisri]

I have tried the same in the meantime, and it also fixes the problem for me (Ubuntu 12.04 also).
No I don't think the disconnects are needed.
Thanks a lot.

[bfroehle]

Yes, this sounds like a reasonable interpretation since typing in the
ipython window also closes the openFile dialog.

[bfroehle]

I'm no expert on Qt4 here, but do we need to launch an actual QCoreApplication? Or can we just create a QEventLoop and run that?

See bfroehle/ipython@inputhook_qt4_alt.

[fperez]

We're only after the event loop control, so if it's safe to do that only with an event loop, I don't think we need an app. What I don't know is if not creating an app will make things better or worse with user code that does try to create an app.

In any case, if we make changes to this logic, the examples in docs/examples/lib should be manually checked, as well as our instructions on event loop integration, because this would be a fairly significant semantic change.

[bfroehle]

For a little bit of background on why one might want to just use QEventLoop::exec, let's examine what QCoreApplication::exec does:

int QCoreApplication::exec()
{
    if (!QCoreApplicationPrivate::checkInstance("exec"))
        return -1;

    // ... [some assertions]

    threadData->quitNow = false;
    QEventLoop eventLoop;
    self->d_func()->in_exec = true;
    self->d_func()->aboutToQuitEmitted = false;
    int returnCode = eventLoop.exec();
    threadData->quitNow = false;
    if (self) {
        self->d_func()->in_exec = false;
        if (!self->d_func()->aboutToQuitEmitted)
            emit self->aboutToQuit();
        self->d_func()->aboutToQuitEmitted = true;
        sendPostedEvents(0, QEvent::DeferredDelete);
    }

    return returnCode;
}

As far as I can tell, it's a small wrapper around QEventLoop::exec which also:

• Sets some variables regarding the current status
• Emits an aboutToQuit signal right before the function returns (which is the root cause of @denisri's issue).

However I'm not Qt expert, so this approach might not be valid at all... shrug