newgil backport

[rcohen]

BPO	7753
Nosy	@malemburg, @nascheme, @jcea, @pitrou, @briancurtin
Files	newgil-2.7.diff: Backport of newgil to trunk gil-2.7.txt

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2010-01-27.14:32:28.548>
created_at = <Date 2010-01-22.01:40:55.355>
labels = ['interpreter-core', 'performance']
title = 'newgil backport'
updated_at = <Date 2011-06-09.06:13:10.178>
user = 'https://bugs.python.org/rcohen'

bugs.python.org fields:

activity = <Date 2011-06-09.06:13:10.178>
actor = 'jmehnle'
assignee = 'none'
closed = True
closed_date = <Date 2010-01-27.14:32:28.548>
closer = 'lemburg'
components = ['Interpreter Core']
creation = <Date 2010-01-22.01:40:55.355>
creator = 'rcohen'
dependencies = []
files = ['15967', '15974']
hgrepos = []
issue_num = 7753
keywords = ['patch', 'needs review']
message_count = 32.0
messages = ['98122', '98134', '98139', '98140', '98143', '98150', '98156', '98158', '98164', '98165', '98166', '98167', '98168', '98191', '98192', '98388', '98393', '98411', '98412', '98417', '98418', '98423', '98429', '98432', '98433', '98468', '98480', '98481', '98482', '98540', '98542', '105992']
nosy_count = 13.0
nosy_names = ['lemburg', 'nascheme', 'jcea', 'pitrou', 'schmir', 'kevinwatters', 'Omnifarious', 'jnoller', 'twhitema', 'Ringding', 'brian.curtin', 'rcohen', 'jmehnle']
pr_nums = []
priority = 'high'
resolution = 'rejected'
stage = 'patch review'
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue7753'
versions = ['Python 2.7']

[rcohen]

This is a backport of the newgil work to the 2.7 branch (otherwise known as trunk.) Patch is against r77678.

Created by diffing r76193 (last in the newgil branch)
against r76189 and applying the result. Generally applied cleanly and it looked as though only 1 reject needed to be massaged in. Passes all tests except for test_command, but the unpatched source also fails that test on my machine.

Perf numbers for linux on Intel(R) Core(TM)2 Quad CPU Q9300 @ 2.50GHz
Using benchmark from here:
http://www.mail-archive.com/python-dev@python.org/msg43407.html
-j0
2.7 : 14.93s, 15.71s, 15.03s
newgil: 15.32s, 15.45s, 15.78s

-j4
2.7 : 22.44s, 22.30s, 22.19s
newgil: 18.75s, 18.80s, 18.79s

[nascheme]

Looks like the 2.7 patch has a spurious change to /Lib/unittest/runner.py.

[malemburg]

A few things:

• The ceval_gil.h file is missing from the patch.

• Please check whether you can move these variable declarations inside the main loop function (as statics):

+/* This single variable consolidates all requests to break out of the fast path

• in the eval loop. /
  +static volatile int eval_breaker = 0;
  +/ Request for droppping the GIL /
  +static volatile int gil_drop_request = 0;
  +/ Request for running pending calls /
  +static volatile int pendingcalls_to_do = 0;
  +/ Request for looking at the async_exc field of the current thread state */
  +static volatile int pending_async_exc = 0;

• Instead of deprecating the set/getcheckinterval APIs, convert the arguments passed to these to the switch interval. The new APIs set/getswitchinteral are then not needed.

• Please add the fallback solutions from the time module in case gettimeofday() is not available. You cannot assume that "all modern POSIX systems" implement that API - it was introduced in POSIX 2001 and Python 2.x still supports OSes that were released prior to that year.

• Note that gettimeofday() et al. are rather slow APIs on most systems. It would be better to directly use a simple performance timer since we don't really need any timing accuracy in managing the GIL when run with threads. Have a look at the clock_gettime() API for ways of accessing such timers. There may also be other ways of accessing such times via pthreads.

• I haven't looked at the timing details of the implementation, but please make sure that it works even if the clock interval is a few ms (while gettimeofday() provides us resolution, the system clock may still only tick in ms intervals - another reason to use CPU performance timers instead).

Thanks.

[malemburg]

See Tools/pybench/systimes.py for more comments on clocks, ticks and update frequency.

[pitrou]

• Instead of deprecating the set/getcheckinterval APIs, convert the
  arguments passed to these to the switch interval. The new APIs
  set/getswitchinteral are then not needed.

This is certainly not a good idea. The semantics are different, which is the reason why I created a separate API in the first place instead of recycling the old one.

Actually, there should be hardly any reason to want to change the interval with the new GIL implementation. It was required with the old GIL due to the fact that counting opcodes is a very poor way of estimating wall clock durations.

Note that gettimeofday() et al. are rather slow APIs on most systems

In my measurements under Linux it seemed to be as fast as clock_gettime() actually. But I don't know about other systems. I didn't want to complicate the patch (with several different paths) more than seemed necessary.

[nascheme]

Here is an updated set of patches that includes the ceval_gil.h file as well as the documentation changes.

[malemburg]

Antoine Pitrou wrote:

Antoine Pitrou <pitrou@free.fr> added the comment:

> * Instead of deprecating the set/getcheckinterval APIs, convert the
> arguments passed to these to the switch interval. The new APIs
> set/getswitchinteral are then not needed.

This is certainly not a good idea. The semantics are different, which is the reason why I created a separate API in the first place instead of recycling the old one.

Actually, there should be hardly any reason to want to change the interval with the new GIL implementation. It was required with the old GIL due to the fact that counting opcodes is a very poor way of estimating wall clock durations.

Applications currently use those APIs for two reasons:

• they use lots of threads and need to make sure that they
  all run more often than they do with the default settings e.g.
  to make them respond faster to external inputs: in such a
  case, they use a low check interval number

• they don't use any threads and thus don't need to check
  for other threads at all or only rarely: in such a case they
  use a large check interval number

While the semantics are different, the result is similar and
the actual numbers used are usually determined by experiment
or rough estimate - noone expects the APIs to provide any kind
of exact timing and it's not needed either.

Turning the existing APIs into no-ops is certainly not a good
idea, since that will change application performance for both
use cases.

Esp. the second use case (single threaded process) is a rather
common one in Python - using multiple processes is often the more
scalable approach for Python applications, than relying on
multiple threads.

> Note that gettimeofday() et al. are rather slow APIs on most systems

In my measurements under Linux it seemed to be as fast as clock_gettime() actually. But I don't know about other systems. I didn't want to complicate the patch (with several different paths) more than seemed necessary.

Right, but clock_gettime() provides a better resolution.

Both APIs are relatively slow, so if you call them often, it's
better to go with a different mechanism such as one built on
CPU performance counters.

[pitrou]

While the semantics are different, the result is similar and
the actual numbers used are usually determined by experiment
or rough estimate - noone expects the APIs to provide any kind
of exact timing and it's not needed either.

There's no reasonable formula for computing an absolute switching
interval from the current "check interval" number. Besides, as I said,
there's no demonstrated reason to change the interval with the new GIL.

• they don't use any threads and thus don't need to check
  for other threads at all or only rarely: in such a case they
  use a large check interval number

Well, if you don't use any threads you don't have to change the setting
in the first place :-). You might want to do it in order to
"micro-optimize" your app (i.e. save 1-2% on CPU-bound code), but this
is unnecessary with the new GIL, since the interval is ignored when
there's only one thread running.

Turning the existing APIs into no-ops is certainly not a good
idea, since that will change application performance for both
use cases.

The new GIL will change performance far more anyway. Trying to simulate
the old behaviour is doomed to failure. If we want to keep the current
performance characteristics, we'd better not backport the new GIL (which
I'm fine with by the way).

[malemburg]

Antoine Pitrou wrote:

Antoine Pitrou <pitrou@free.fr> added the comment:

> While the semantics are different, the result is similar and
> the actual numbers used are usually determined by experiment
> or rough estimate - noone expects the APIs to provide any kind
> of exact timing and it's not needed either.

There's no reasonable formula for computing an absolute switching
interval from the current "check interval" number.

Just use some conversion formula, e.g. switching interval in micro
seconds = constant * byte code check interval. We can then determine
a constant to match todays CPU performance.

Given the inexact nature of the check interval that's a very
reasonable formula.

Besides, as I said,
there's no demonstrated reason to change the interval with the new GIL.

Didn't I just point out two use cases ?

Perhaps I'm missing some feature of the new GIL. Is there some
documentation for it somewhere ?

> * they don't use any threads and thus don't need to check
> for other threads at all or only rarely: in such a case they
> use a large check interval number

Well, if you don't use any threads you don't have to change the setting
in the first place :-). You might want to do it in order to
"micro-optimize" your app (i.e. save 1-2% on CPU-bound code), but this
is unnecessary with the new GIL, since the interval is ignored when
there's only one thread running.

Hmm, how do you determine that only one thread is running ?

What if an extension uses threads that are not under Python
control, e.g. when embedding the JVM or when using a CORBA
ORB ?

> Turning the existing APIs into no-ops is certainly not a good
> idea, since that will change application performance for both
> use cases.

The new GIL will change performance far more anyway. Trying to simulate
the old behaviour is doomed to failure. If we want to keep the current
performance characteristics, we'd better not backport the new GIL (which
I'm fine with by the way).

I don't follow you.

If an application currently sets the check
interval to a low value, Python should switch to other threads
more often, ie. the switch interval needs to be low.

If an application currently sets the check interval to a high
value, Python should switch to other threads less often, ie.
the switch interval needs to be higher.

Both cases can be mapped to the switch interval, so why not
do it ?

[rcohen]

Thanks Neil for fixing up the patch.

As for Marc-Andre's comments, I'm happy to backport any further
changes which happen on the py3k branch. I'd like to keep this as a
strict backport, only changing things as necessary to get it to work
with the older codebase.

Ross

[pitrou]

Just use some conversion formula, e.g. switching interval in micro
seconds = constant * byte code check interval. We can then determine
a constant to match todays CPU performance.

Well, there are two problems:

• opcode execution cost varies so wildly that trying to pick up an
  average would be meaningless (it can vary by many order of magnitudes
  between e.g. a POP_TOP and a CALL_FUNCTION calling a C extension method)
• the interval doesn't have the same effect in the new GIL as it has in
  the old GIL; in the old GIL, it merely gives the OS a chance to switch
  threads if it wants to (there are many missed opportunities, by a
  proportion which will be system-dependent); in the new GIL, thread
  switching is enforced which means it is not necessary to stop so often.

So we'd have a formula like:

new GIL interval = old GIL interval * opcode duration / proportion of
missed switching opportunities in the old GIL

where two of the three factors on the right side are totally
unpredictable :)

Perhaps I'm missing some feature of the new GIL. Is there some
documentation for it somewhere ?

Almost nothing besides what is found in ceval_gil.h and in the following
thread:
http://mail.python.org/pipermail/python-dev/2009-October/093321.html
Dave Beazley did a high-level presentation about it:
http://www.dabeaz.com/python/NewGIL.pdf

> Well, if you don't use any threads you don't have to change the setting
> in the first place :-). You might want to do it in order to
> "micro-optimize" your app (i.e. save 1-2% on CPU-bound code), but this
> is unnecessary with the new GIL, since the interval is ignored when
> there's only one thread running.

Hmm, how do you determine that only one thread is running ?

Actually, I don't determine it, it's a side effect of how the GIL is
implemented. If no thread is waiting for the GIL, the main thread isn't
"asked" to drop it.

What if an extension uses threads that are not under Python
control, e.g. when embedding the JVM or when using a CORBA
ORB ?

These don't wait for the GIL, hopefully :-)

[malemburg]

On second thought, you're right: while sys.setcheckinterval() could
be made to work with the switching interval, sys.getcheckinterval()
could not be made to return anything useful.

Still, please have a look at the other issues mentioned. They
apply to new GIL in general, regardless of whether implemented
in 3.x or 2.7.

It would also be interesting to see the affect on other threaded
applications, not just the "benchmark" mentioned on the python-dev
thread.

Since there aren't all that many Python 3.x applications in the wild,
it's probably a good idea to check in the patch and then run a few
Python 2.x apps with and without it.

How does pybench react to the patch ? It's single-threaded at the
moment, so a good test of how a single-threaded application would
behave with the patch.

Thanks,

Marc-Andre Lemburg
eGenix.com

---

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::

eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
http://www.egenix.com/company/contact/

[malemburg]

Antoine Pitrou wrote:

> Perhaps I'm missing some feature of the new GIL. Is there some
> documentation for it somewhere ?

Almost nothing besides what is found in ceval_gil.h and in the following
thread:
http://mail.python.org/pipermail/python-dev/2009-October/093321.html
Dave Beazley did a high-level presentation about it:
http://www.dabeaz.com/python/NewGIL.pdf

Thanks. Dave's slides help a lot. The logic certainly sounds like
a major improvement !

>> Well, if you don't use any threads you don't have to change the setting
>> in the first place :-). You might want to do it in order to
>> "micro-optimize" your app (i.e. save 1-2% on CPU-bound code), but this
>> is unnecessary with the new GIL, since the interval is ignored when
>> there's only one thread running.
>
> Hmm, how do you determine that only one thread is running ?

Actually, I don't determine it, it's a side effect of how the GIL is
implemented. If no thread is waiting for the GIL, the main thread isn't
"asked" to drop it.

> What if an extension uses threads that are not under Python
> control, e.g. when embedding the JVM or when using a CORBA
> ORB ?

These don't wait for the GIL, hopefully :-)

Well, they might call back into Python, but I guess that's not
supported anyway, not even in 2.x, since such a thread wouldn't
have a properly setup Python thread state.