Skip to content

Commit

Permalink
matchers.helpers: Revert to using multiprocessing instead of threading
Browse files Browse the repository at this point in the history
Using threads here is (mostly) blocking our UI thread and I don't really
understand why. Until I can figure this out, pushing us back to
multiprocessing fixes our UI interactivity at a fairly minor performance
penalty.

Python's GIL means that throwing our comparison on to a thread doesn't
actually get us real multi-CPU concurrency... but it shouldn't be
causing the comparison thread to starve the main thread. It's not
actually blocking either, because some UI events *do* get through while
a comparison is running, just not many.

My current suspicion is that this is some kind of interaction between
the GIL, thread scheduling and GLib's event loop. However, that's a
guess because I don't have any other explanation.
  • Loading branch information
kaiw committed Jul 15, 2017
1 parent e4aa576 commit 07ec4fd
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions meld/matchers/helpers.py
@@ -1,7 +1,7 @@

import logging
import multiprocessing
import queue
import threading
import time

from gi.repository import GLib
Expand All @@ -12,7 +12,7 @@
log = logging.getLogger(__name__)


class MatcherWorker(threading.Thread):
class MatcherWorker(multiprocessing.Process):

matcher_class = myers.InlineMyersSequenceMatcher

Expand Down Expand Up @@ -45,12 +45,12 @@ class CachedSequenceMatcher(object):

def __init__(self):
self.cache = {}
self.tasks = queue.Queue()
self.tasks = multiprocessing.JoinableQueue()
# Limiting the result queue here has the effect of giving us
# much better interactivity. Without this limit, the
# result-checker tends to get starved and all highlights get
# delayed until we're almost completely finished.
self.results = queue.Queue(5)
self.results = multiprocessing.JoinableQueue(5)
self.thread = MatcherWorker(self.tasks, self.results)
self.task_id = 1
self.queued_matches = {}
Expand Down

0 comments on commit 07ec4fd

Please sign in to comment.