Implement proper async cancelling

[nicolas-raoul]

It seems that calculating Git properties blocks other aspects of Nautilus.

In real-world usage, most of the time I use Nautilus to browse, read or modify files, and only relatively rarely do I need to pay attention the Git status.

So, how about letting Nautilus run at full speed, and perform expensive calculations in a different thread with a lower priority?

This question about speeding up a Nautilus Python extension says:

Since nautilus-python 0.7 are available update_file_info_full and cancel_update, which allows you to program async calls. You can check the documentation of Nautilus 0.7 for more details.

I know it is easier said than done :-)

[chrisjbillington]

When I first wrote the code I profiled it and determined that the git call is not the bottleneck, and in fact very little time is spent within the extension at all - it's nautilus drawing all those icons that is slow (I noticed things are fast if I return that no icon needs to be drawn). Whilst I'm sure huge repositories exist where the git call would be slow too, I'm yet to find one (I think I tested on the linux kernel), so I think little would be gained by making the git call asynchronous.

So I suspect the best that could be done is delaying drawing the icons, but nautilus would still spend the same amount of time drawing them. If I delayed and then returned them all to nautilus at once, nautilus would still hang for the same amount of time making it unresponsive to clicks or scrolls or whatnot. To get any gain in responsiveness I'd have to space them out in time, having nautilus draw at some number of icons per second.

I might look into this but it's not a super high priority at the moment!

Pull requests welcome :)

[nicolas-raoul]

The average git status on my repos takes 2 seconds, sometimes much more.
Yesterday, I opened the folder where I have my 150 Git repositories and for some reason that made my whole GUI unusable, unfortunately (I am not saying that it is a git_nautilus_icons bug, just mentioning that it happens ^_^). I understand it is a low priority issue, and will try to see what can be done.

[chrisjbillington]

Ah, yes, if they're individually taking 2 seconds and you have 150 of them well...
Since you're in the folder containing the git repos, and not within the repos themselves, there are probably not many icons for nautilus to draw, but there are many git status calls, so I can see it would a problem in that situation.

Making the git call asynchronous might indeed be the right solution then.

[chrisjbillington]

There is unfortunately an issue with the nautilus python extension framework which means that python threads cease to run when the extension is not otherwise active. This complicates things somewhat. As far as I can tell this is just a bug in nautilus-python, it looks like the GIL is not released after a callback has run, such that Python code from other threads cannot run if no callback is in progress in the main thread. However, nautilus-python is written in a way that makes it clear the GIL is intended to be released there, so it's an unintentional bug rather than a design flaw. The issue is known and there looks to be little interest in fixing it, and I cannot immediately see how to fix it myself.

So, to make things asynchronous, I would likely have to use a subprocess to run the git calls, plus a gobject idle to run a callback every 0.1 seconds or something to check if the git calls are done. Despite being less desirable than threads, this is definitely possible and not too tricky, and I'm pretty savvy with multiprocessing, so I will do this at some point soon when I have a few hours and the motivation.

[chrisjbillington]

@nicolas-raoul this ought to be resolved in 621b7e0, if you are interested in testing!

[chrisjbillington]

I haven't yet implemented cancelling of status calls, so if you navigate to a directory with a lot of git repos, and then to another directory, the extension will still be calculating the status for the old directory and so it will not get around to the new directory until the first one is done. But, it won't freeze Nautilus's GUI in the meantime.

I will have to restructure things a little more to get the cancelling working, as it will mean something like keeping a list of files are are waiting on each git status call that is in progress, and then when all of the files are cancelled from Nautilus, terminating the git status call. Something like that.

[nicolas-raoul]

Testing it now, thanks a lot! :-)

[hmaarrfk]

Well i came here because I learned about the Nautilus API from your example.

I am really impressed that you got it to work so well.

I just wanted to let you know that I ran into some issues with the async call
https://gitlab.gnome.org/GNOME/nautilus-python/-/issues/19