changelog.txt

Stackless-Python News
+++++++++++++++++++++

This file documents user visible changes to the Stackless extension of C-Python.
For other changes see Misc/NEWS.

What's New in Stackless 3.X.X?
==============================

*Release date: 20XX-XX-XX*

- https://github.com/stackless-dev/stackless/issues/175
  Cleanup of the Stackless C-API: Including stackless_api.h in an extension
  module no longer defines internal stuff. Additionally I moved test code
  from module "stackless" into the new module "_teststackless".

- https://github.com/stackless-dev/stackless/issues/177
  Fix a crasher in prickelpit.c slp_cannot_execute().
  Stackless versions released after March 2017 can crash, if a program executes
  an unpickled tasklet, that lost its C-state. Now Stackless always raises
  RuntimeError.

- https://github.com/stackless-dev/stackless/issues/152
  C-Python bpo30860 consolidates stateful C globals under a single struct.
  Stackless also moves its stateful C globals to the appropriate structures
  (runtime-state or interpreter-state or thread-state). This change modifies
  some internal APIs and structures.

- https://github.com/stackless-dev/stackless/issues/149
  C-Python bpo-25612 (#1773) moves the exception state information from frame
  object to coroutine (generator/thread) object where it belongs. As a
  consequence Stackless moves the exception state information from thread
  to tasklet. This changes the pickle format of frame, tasklet and generator
  objects compared to previous major versions of Stackless.

- https://github.com/stackless-dev/stackless/issues/166
  Fix C-API functions PyEval_EvalFrameEx() and PyEval_EvalFrame().
  They are now compatible with C-Python.

- https://github.com/stackless-dev/stackless/issues/168
  Make Stackless compatible with old Cython extension modules compiled
  for regular C-Python.

- https://github.com/stackless-dev/stackless/issues/167
  Replace 'printf(...)' calls by PySys_WriteStderr(...). They are used to emit
  an error message, if there is a pending error while entering Stackless
  subsystem.

- https://github.com/stackless-dev/stackless/issues/154
  Fix a minor bug in STACKLESS_PROMOTE_WRAPPER and STACKLESS_PROMOTE_METHOD.

- https://github.com/stackless-dev/stackless/issues/150
  Thread ids are now unsigned. See bpo-6532. This change of the Python C-API
  causes analog changes in the Stackless C-API. In
  PyTasklet_BindThread(PyTaskletObject *task, unsigned long thread_id)
  the type of "thread_id" changed from "long" to "unsigned long". And
  the return value of PyStackless_GetCurrentId(void) changed from "long" to
  "unsigned long".
  The prototype of PyStackless_GetCurrentId(void) was missing from
  stackless_api.h. It is now defined.

- https://github.com/stackless-dev/stackless/issues/146
  Always initialize _stackless.bonb type. Prevents a NULL-pointer access,
  if a thread terminates with a MemoryError exception.

- https://github.com/stackless-dev/stackless/issues/145
  Fix the upper bound of tests in test/test_doctest.py.

- https://github.com/stackless-dev/stackless/issues/144
  Replace copied code in pricklepit.c dictview_new.

- https://github.com/stackless-dev/stackless/issues/143
  Fix non-portable usage of bit fields.

- https://github.com/stackless-dev/stackless/issues/141
  Call Py_DECREF(frame) with the right recursion_depth.

- https://github.com/stackless-dev/stackless/issues/137
  C-API: All functions of the form Py{Object, Function, CFunction}_*Call* now
  use STACKLESS_PROMOTE_xxx internally and may be called with
  slp_try_stackless != 0. See the comment in core/stackless_impl.h.

- https://github.com/stackless-dev/stackless/issues/133
  WARNING: C-extension modules written for Stackless Python must be recompiled
  for Stackless Python 3.6, because definition of METH_STACKLESS changed.

- https://github.com/stackless-dev/stackless/issues/140
  Fix bugs in prickelpit.c unwrap_frame_arg() and slp_find_execname().

- https://github.com/stackless-dev/stackless/issues/138
  Correctly propose Stackless calls for C-functions.

- https://github.com/stackless-dev/stackless/issues/136
  Replace 'xrange' with 'range' in *.py. Python 3 has no xrange.

- https://github.com/stackless-dev/stackless/issues/135
  Fix small memory leaks for types with Stackless extensions.

- https://bitbucket.org/stackless-dev/stackless/issues/130
  Pickling exhausted generators no longer crashes.

- https://bitbucket.org/stackless-dev/stackless/issues/129
  C-API: Calling PyTasklet_New( NULL, ...) no longer crashes.

- https://bitbucket.org/stackless-dev/stackless/issues/128
  Fix pickling of the module 'stackless'.

- https://bitbucket.org/stackless-dev/stackless/issues/127
  Disable the Stackless specific code for pickling 'iterator' and
  'callable_iterator' objects. C-Python 3.3 already pickles them. Stackless-3.5
  can't read pickles of 'iterator' or 'callable_iterator' objects created with
  Stackless 3.4.2 or older versions.
  Fix pickling of 'method-wrapper' objects.
  Fix copy.copy() for 'method', 'dict_keys', 'dict_values' and 'dict_items'
  objects.

- https://bitbucket.org/stackless-dev/stackless/issues/126
  Load the module stackless early in all C-Python tests. This ensures a defined
  behaviour of the tests even, if the execution order gets randomised.

- https://bitbucket.org/stackless-dev/stackless/issues/125
  This document (changelog.txt) is included in the documentation as
  "What’s New in Stackless-Python ..."

- https://bitbucket.org/stackless-dev/stackless/issues/123
  Fix a rare assertion violation during interpreter shutdown with active
  daemon threads.

- https://bitbucket.org/stackless-dev/stackless/issues/122
  Disable the C-stack cache, if Py_REF_DEBUG is defined. The cache causes false
  block leak errors from the test suite. The performance penalty is irrelevant in
  debug builds.

- https://bitbucket.org/stackless-dev/stackless/issues/121
  Fix a reference leak. If Python clears a thread state and a destructor or a
  weakref-callback runs Python code, Python used to leak a reference to a
  C-stack object.

- https://bitbucket.org/stackless-dev/stackless/issues/120
  Stackless now correctly grabs the head_mutex, when it iterates over the list
  of thread states.

- https://bitbucket.org/stackless-dev/stackless/issues/119
  Fix a rare bug in the stack unwinding mechanism, that caused a SystemError
  exception or an assertion violation, if a __del__()-method or a weakref
  callback runs during stack unwinding.

- https://bitbucket.org/stackless-dev/stackless/issues/115
  Fix an unlikely crash caused by an context manager, which silences an
  exception.

- https://bitbucket.org/stackless-dev/stackless/issues/117
  Fix various reference leaks:
  - Leak of a reference to None in generator.throw()
  - Leak of a reference to the thread-id of every thread returned by
    stackless.threads
  - Leak of a reference to None (usually) in the C-implementations of
    channel.send_throw() and channel.send_exception()
  - Leak of a reference to the caught exception (type and value) and traceback,
    if Stackless invokes an user defined error-handler.
  - Leak to an object, if a Python profile or trace function fails.
  - Various leaks, if stackless._wrap.frame.__setstate__() fails, because its
    state argument is invalid.
  - Leak of references to Python stack frames or Stackless C-frames, if
    a tasklet didn't run to its end or various other conditions. This part
    of the fix changes the Stackless reference counting for frames to follow
    the general rules for Python objects.
  - Leak of a reference to a frame in stackless.tasklet.__setstate__()

  Additionally, this change brings the handling of caught exceptions more in
  line with C-Python.

- https://bitbucket.org/stackless-dev/stackless/issues/112
  Adapt Stackless to Python 3.5.
  - PyGenObject got two additional fields
  - Skip the CPython test case test.test_pickle.*.test_local_lookup_error
  - Enable the pickling of coroutine objects

- https://bitbucket.org/stackless-dev/stackless/issues/111
  Restore the Python ABI function PyGen_New(). Previously Stackless named this
  function PyGenerator_New() and used a macro the redefine PyGen_New as
  PyGenerator_New. Stackless is now again binary compatible with Python-
  extensions using PyGen_New().

- https://bitbucket.org/stackless-dev/stackless/issues/107
  Improve unpickling of traceback objects. Stackless now reconstructs the
  frame.f_back linkage in frames directly referenced by traceback objects.
  Stackless no longer pretends to be able to pickle arbitrary frame objects.

- https://bitbucket.org/stackless-dev/stackless/issues/110
  Remove the already non functional remains of psyco support.

- https://bitbucket.org/stackless-dev/stackless/issues/109
  The Stackless python*.dll no longer exports private symbols. Symbols
  required by Stackless-aware 3rd party extension modules should still be
  available.

- https://bitbucket.org/stackless-dev/stackless/issues/108
  Use PyVarObject_HEAD_INIT to initialise type objects. See PEP 3123.

- https://bitbucket.org/stackless-dev/stackless/issues/106
  Raise RuntimeError, if you call tasklet.setup() on a tasklet, that is
  already alive.

- https://bitbucket.org/stackless-dev/stackless/issues/105
  Fix an occasional NULL-pointer access during interpreter shutdown.

- https://bitbucket.org/stackless-dev/stackless/issues/104
  Initialise the global variable slp_initial_tstate early before it is used.

- https://bitbucket.org/stackless-dev/stackless/issues/103
  Improve the interpreter shutdown, if deeply nested
  tasklets are present. Previously you some assertions could fail.

- https://bitbucket.org/stackless-dev/stackless/issues/101
  Enhance Tools/gdb/libpython.py to support debugging Stackless Python.
  You can now debug Stackless Python with gdb. Unfortunately gdb still
  does not know about inactive tasklets and their frames.

- https://bitbucket.org/stackless-dev/stackless/issues/99
  On UNIX like systems you can use the command
  $ make teststackless
  to run the Stackless unit tests. Previously the command required some non
  POSIX commands.

- https://bitbucket.org/stackless-dev/stackless/issues/97
  Fix the stack switching for optimized builds for all Unix-like architectures
  except amd64, where it was already OK. This change removes the "static"
  declaration from the function slp_switch(). This forces the compiler
  to adhere to the ABI specification. On i386 and amd64 it is now save to
  configure stackless with --enable-stacklessfewerregisters, except if you
  compile for darwin.

- https://bitbucket.org/stackless-dev/stackless/issues/94
  Calls to __init__(self, ...) are now stackless, if
  soft-switching is enabled.
  Note: pickling and unpickling an object while its __init__() method runs,
  may cause unexpected interactions between the object reconstruction and the
  remaining part of __init__().

- https://bitbucket.org/stackless-dev/stackless/issues/98
  Prevent an unlikely NULL-pointer access in safe_pickle.c pickle_callback().
  The bug is very old and might affect multiprocessing.

- https://bitbucket.org/stackless-dev/stackless/issues/96
  Impose a very high limit on the recursion depth of cPickle.
  Previously an infinite recursion could eat up all you memory
  and finally crash Stackless. No the recursion stops after using
  about 170MB (32bit) / 300MB (64bit) of memory.

- https://bitbucket.org/stackless-dev/stackless/issues/93
  Unify tasklet.kill(), tasklet.throw() and tasklet.raise_exception().
  They now behave almost identically. This affects the error handling in some
  corner cases:
  + Some crash bugs have been fixed. Stackless Python now raises RuntimeError
    instead of crashing.
  + It is not possible to raise an exception in a dead (== not alive) tasklet.
    In general, if you try it, the method call fails with an appropriate
    RuntimeError. However there are exceptions:
  + For Stackless versions before 3.3.7
    tasklet.raise_exception(exc_class, *args) raises exc_class(*args) instead
    of RuntimeError.
  + If you call tasklet.kill() or tasklet.throw(TaskletExit) or
    tasklet.raise_exception(TaskletExit) and if the tasklet already ran to its
    end (tasklet.frame is None), the call succeeds.
  + If you call tasklet.kill() on a tasklet, that didn't run to its end
    (tasklet.frame is not None), but has no thread (tasklet.thread_id == -1),
    Stackless drops the frame and proceeds as if the tasklet ran to its end.
  Most users won't notice the difference.

- https://bitbucket.org/stackless-dev/stackless/issues/89
- https://bitbucket.org/stackless-dev/stackless/issues/81
  Fix (crash) bugs during thread and interpreter shutdown.
  If a thread dies, Stackless now really kills tasklets with C-state. During
  interpreter shutdown, Stackless also kills daemon threads, if they switch
  tasklets. This change avoids crashes during a later shutdown stage.

- https://bitbucket.org/stackless-dev/stackless/issues/92
  If you run Stackless with the option '-v' (or set the environment variable
  PYTHONVERBOSE), Stackless prints a warning message, if it deallocates a
  tasklet, that has C-state.
  Additionally, the methods tasklet.bind() and tasklet.bind_thread() now check
  correctly, if the tasklet has C-state.

- https://bitbucket.org/stackless-dev/stackless/issues/91
  Stackless now resets the recursion depth, if you re-bind
  a tasklet to another callable.

- https://bitbucket.org/stackless-dev/stackless/issues/90
  Stackless now raises a RuntimeError, if you try to unbind (tlet.bind(None))
  a main-tasklet.

- https://bitbucket.org/stackless-dev/stackless/issues/78
  Documentation update: if you iterate over a channel, the
  sender must send StopIteration manually. See
  http://stackless.readthedocs.io/en/latest/library/stackless/channels.html#channel.__iter__

- https://bitbucket.org/stackless-dev/stackless/issue/83
  Fix demo/tracing.py: don't use flextype features. Unfortunately
  it is no longer possible to change the __repr__ method of class tasklet.

- https://bitbucket.org/stackless-dev/stackless/issue/85
  Fix a rare problem in the watchdog logic.

- https://bitbucket.org/stackless-dev/stackless/issue/80
  Fix an assertion failure during shutdown. The assertion was added by
  the first attempt to fix issue #60. Thanks to Masamitsu Murase
  for reporting the bug and for providing the fix.

- https://bitbucket.org/stackless-dev/stackless/issue/79
  Add __all__ to module stackless. Add entries to the module's __dict__
  for the computed properties (i.e. current, main, ...). This helps IDEs
  (PyDev) to recognise the expression "stackless.current" as a defined name.
  Fixed tasklet.raise_exception and channel.send_exception to raise a correct
  TypeError, if called without arguments.

- https://bitbucket.org/stackless-dev/stackless/issue/77
  Fix a NULL pointer exception, if a call to stackless.tasklet() fails
  (i.e. called with invalid arguments).

- https://bitbucket.org/stackless-dev/stackless/issue/60
  Fix a crash

- https://bitbucket.org/stackless-dev/stackless/issue/71
  Fix a regression in Lib/pickle.py. Pickling functions
  didn't work any longer.

- Fix for test_watchdog_on_tasklet_hard due to re !!!. The caching behaviour
  of re.py was changed in debug mode, which influenced the number of opcodes
  spent in our test.

- Fix compiler warnings found on MAC.

- https://bitbucket.org/stackless-dev/stackless/issue/61
  Fix a crash after unpickling hard switched tasklet. It is now
  always possible to inspect the local variables (frame.f_locals)
  of an unpickled frame object.


What's New in Stackless 3.3.5?
==============================

*Release date: 2014-03-11*

- Fix channel and tasklet initialization.  Initialization is now done
  using __init__ methods.
  Tasklet.__new__() now just binds the tasklet to the calling thread and
  ignores extra arguments.
  The tasklet.__init__() now works like tasklet.bind.
  Channel.__new__() now ignores extra arguments.
  These changes make subclassing taskelets and channels less confusing.

- https://bitbucket.org/stackless-dev/stackless/commits/26a53cfeeeffeaacb55
  Make stackless.run() more flexible.  It can now be called by any tasklet.
  Without arguments, stackless.run() now nests, so that the innermost run()
  is awoken when the run-queue empties.  With timeout arguments, the first
  call to stackless.run() wins, so that it is possible to override
  watchdog behaviour on the outside of the app.  subsequent timeout calls
  to stackless.run() then behave as though they were called without
  arguments.

- https://bitbucket.org/stackless-dev/stackless/commits/2c252b72ddaf628fb1
  Remove the "Flextype" internal type extension apparatus.  Tasklets
  and channels are now regular builtin objects.  C api functions like
  PyChannel_Send() no longer magically invoke python methods on subtypes.
  "stackless" module is now composed of builting "_stackless" and the
  regular "stackless.py" modules.  The latter of the two provides the
  "attribute-like" module access.

- https://bitbucket.org/stackless-dev/stackless/commits/949c518c67c75a1
  Fix binary layout of PyTypeObject and PyHeapTypeObject for stackless,
  restoring binary compatibility with extensions built with regular
  Python.  In particular, extensions that themselves extend PyTypeObject,
  such as PySize and PyQT, now work with stackless.

- https://bitbucket.org/stackless-dev/stackless/commits/b373fc28f004c
  Various fixes to deadlock detection and error porpagation.

- https://bitbucket.org/stackless-dev/stackless/issue/54
  Fix a crash


- Issue #50:
  Don't raise StopIteration in a deadlock situation.
  When a channel is closed, only raise StopIteration out of
  channel.__next__.  Send/receive raise ValueError.

- Issue #51:
  The tasklet attrubtes "is_main" and "is_current" now are computed
  relative to the tasklet's thread, not the calling thread.

- http://www.stackless.com/ticket/24: Enable unbinding of tasklets.
  The method tasklet.bind(None) now unbinds a tasklet.

- Add the stackless.atomic() context manager to enter an atomic state.

- Make manual inter-thread scheduling of tasklets more robust.  Fix
  various edge and corner cases pertaining to scheduling tasklets on
  a different thread.

- Reverse the "immediate" flag for tasklet.throw().  it is now
  tasklet.throw(..., pending=False).

- Don't silently ignore TaskletExit on the main tasklet.

- Diable pre-emptive switching in switch-trapped mode.

- When threads are killed, we try killing all their tasklets. They then
  become thread-unbound, so that the Python(r) thread state can be safely
  released. A tasklet without a thread reports its thread_id as -1.

- https://bitbucket.org/stackless-dev/stackless/issue/26:
  Add tasklet.bind_thread() method, to re-bind a soft-switchable
  tasklet to another thread.

- https://bitbucket.org/stackless-dev/stackless/issue/28:
  Enhance tasklet.bind() so that it can bind arguments as well without
  making the tasklet runnable.

- https://bitbucket.org/stackless-dev/stackless/issue/29:
  Add stackless.switch(), to do manual switching of tasklets without
  placing the source in the runnable queue.  I.e. the target is run and
  the source is paused.

- https://bitbucket.org/stackless-dev/stackless/issue/38
  Bug fix: An exception in the schedule callback caused an
  assertion failure in taskletobject.c, line 51

- https://bitbucket.org/stackless-dev/stackless/issue/40
  Bug fix: If a thread state had an exception and tracing was enabled,
  the exception was not preserved on a soft switch.

- https://bitbucket.org/stackless-dev/stackless/issue/41
  Bug fix: Traced tasklet is unpicklable

- https://bitbucket.org/stackless-dev/stackless/issue/42
  Bug fix: During the execution of the schedule callback the internal state
  of the interpreter was sometimes inconsistent. This caused tasklet related
  query methods and computed attributes to return incorrect results.

- https://bitbucket.org/stackless-dev/stackless/issue/43
  Enhancement: new attributes tasklet.trace_function and
  tasklet.profile_function

- https://bitbucket.org/stackless-dev/stackless/issue/44
  Bug fix: some computed tasklet attributes (i.e. tasklet.alive)
  returned incorrect values when they were accessed from an thread the
  tasklet didn't belong to.

- https://bitbucket.org/stackless-dev/stackless/issue/45
  Enhancement: It is now possible to query installed schedule
  and channel callback functions.

- https://bitbucket.org/stackless-dev/stackless/issue/46
  Bug fix: invoking tasklet.run() on a tasklet from a already
  terminated thread no longer crashes the interpreter.

- https://bitbucket.org/stackless-dev/stackless/issue/52
  Use distinct product codes for the Stackless MSI installer.
  Increment the value of the product code of the regular CPython release
  by 8 if it is Stackless.


- Fix unlinking of tasks in kill_tasks_with_stacks. The earlier code
  was a bit weird and would cause miscompilation on X64.

- http://www.stackless.com/ticket/23: Fix compile error with STACKLESS_OFF
  defined

- http://www.stackless.com/ticket/21: Fix crash on exit.

- http://www.stackless.com/ticket/22: soft switching for context managers.

- #17667: Fix Windows build using "build_pgo.bat -2".

- http://www.stackless.com/ticket/18: unexpected increase of nesting level in
  __call__.

- http://www.stackless.com/ticket/20: fix infinite recursion in __call__

- http://www.stackless.com/ticket/17: Fix a few clang warnings.

- http://www.stackless.com/ticket/14: Prevent overly optimization with
  gcc 4.7.x.

- Add the feature of a tasklet error handler, global to the module.

- Add the tasklet.throw() method, which can raise an exception on a tasklet
  with traceback, just like channel.send_throw.  It also has the facility to
  either activate the target immediately (default) or just make it runnable.

- Add the switch-trap flag. Add unittests for channel action and stackless.run
  and stackless.schedule.

- Fix tasklet.raise_exception.  It had invalid flags and didn't work.

- Fix default argument handling of channel.send_throw

- Add channel.send_throw(exc[,val[,tb]]) to raise a new exception or propagate
  an old one across a channel.

- Make sure that the tasklet's "atomic" flag inhibits thread switching, so that
  it can be used when synchronizing tasklets from different threads.

- Apply Anselm Kruis' patch to get amd64 linux building again without the 'bp'
  register problem.

- When a tasklet is killed, remove it from being "active" in the cstate.
  Otherwise, kill_task_with_cstatcks will spin endlessly trying to kill it.
  We have to fix kill_tasks_with_cstacks more properly later.

- Integrate changes to tasklet GC as run by CCP for over a year.  Tasklets are
  now properly garbage collected unless they have a C stack.

- Add a new api, PyStackless_GetCurrentId(), and stackless.getcurrentid(), to
  help with bookkeeping that has to do with callstacks.

- Fix errors where a PyCFrame could confuse code, both when pickling and when
  emitting warnings.  Now use proper internal Stackless methods to get at the
  frame.

- Remove tasklet become and capture methods/support. This functionality is
  considered in retrospect to be a mistake, and has been on track for removal
  for several years.  Goodbye cool functionality!

- A first draft of Stackless C API documentation. This has rewritten
  descriptions for most of the functions, but some still have the API include
  file comments.

- Committing Jeff Senn's non-Windows compilation changes as mailed to the
  Stackless mailing list
  (http://www.stackless.com/pipermail/stackless/2010-July/004766.html).

- the _lsprof.c module is now stackless aware.  This allows the same
  cProfile.Profile() instance to be used by multiple tasklets, for whole
  program profiling. A patch with this new _lsprof.c has also been submitted
  to bug.python.org. What remains now is the ability to set tracing/profiling
  for all tasklets globally.


Here is a break. Nobody wrote a change log. If you are interested
in software archaeology look at http://svn.python.org/view/stackless.


The very beginning of Stackless
===============================

20020524	ct	There appears to be a problem with stack switching
			and tkinter. When I disable it, things work. This is
			most probably a stack misuse with C locals used as
			globals. This gives a problem when squirreled away...
			Should we possibly disable stack manips completely
			by default and have the user switch it on?

20020524	ct	There have been massive changes in thee meantime, and
			I've missed logging them in the changelog.
			This will become better now.
			The old ideas of reusing old stackless stuff are gone.
			There is now always "hardware" switching of tasklets.
			Also, Limbo like channels have been introduced, which
			give us a very simple multitasking.

200203 to	ct	Thinking, thinking and reading. Alef, Limbo, Occam.
200205		Got hired by IronPort for working on Stackless and
			their application based upon it.

20020122	ct	There are some influences of olde Stackless. Without repeating
			the full mess, a number of ideas are very useful
			Especially, the frame dispatcher idea will be revived.
			It turns out to make sense:
			Try to let the innermost interpreter resolve some recusrions.
			Find an equivalent of old Py_UnwindToken. Probably the
			protocol of the new generators can be extended.
			Now I *do* think to modify frames. They get an additional
			field for their C stack part.
			There will definately be a revival of f_execute.
			This fits perfectly: For pure frames, this is eval_frame.
			For frames which need to restore their stack, it will be a helper
			function.
			I can see the light :-)

20020121	ct	Initial setup, first Stackless version (without functionality yet)