Stackless-Python

Author:Richard Tew (richard.m.tew@gmail.com)
Release:3.4.9
Date:September 05, 2018

Stackless-Python is an enhanced version of the Python® programming language[1]. It allows programmers to reap the benefits of thread-based programming without the performance and complexity problems associated with conventional threads. The microthreads that Stackless adds to the Python® programming language are a cheap and lightweight convenience, which if used properly, can not only serve as a way to structure an application or framework, but by doing so improve program structure and facilitate more readable code.

If you are reading this text as part of a version of a Python® interpreter you have installed, then you have installed Stackless-Python rather than standard Python®[2].

What you need to know

Stackless-Python provides a minimal framework and it is not accompanied by any support functionality, that would otherwise address the common needs that may arise when building a more targeted framework around it.

Blocking operations

When operations are invoked that block the Python® interpreter, the user needs to be aware that this inherently blocks all running tasklets. Until the tasklet that engaged that operation is complete, the Python® interpreter and therefore scheduler is blocked on that operation and in that tasklet. Operations that block the interpreter are often related to synchronous IO (file reading and writing, socket operations, interprocess communication and more), although time.sleep() should also be kept in mind. The user is advised to choose asynchronous versions of IO functionality.

Some third-party modules are available that replace standard library functionality with Stackless-compatible versions. The advantage of this approach is that other modules which use that standard functionality, also work with Stackless when the replacement is installed. The Stackless socket module is the most commonly used replacement module.

Exceptions

Certain exceptions that may occur within tasklets, are expected to reach all the way up the call stack to the scheduler. This means that naively using the except statement may result in hard to track down problems.

A description of the problem with bare except statements can be read in the documentation for the TaskletExit exception.

Debugging

The Stackless scheduling mechanism changes the way the Python® debugging hooks work so that debugging hooks are set per-tasklet rather than per-thread. However, very few debuggers, certainly none of those in the standard library take this into account. As a result of this, debugging is unlikely to work with special handling being worked into your use of Stackless.

A description of this problem can be read in the Stackless debugging documentation.

External resources

There are a range of resources available outside of this document. However most of them are nowadays quite dated:

History

Continuations are a feature that require the programming language they are part of, to be implemented in a way conducive to their presence. In order to add them to the Python® programming language, Christian Tismer modified it extensively. The primary way in which it was modified, was to make it Stackless. And so, his fork of standard Python® was named Stackless-Python.

Now, storing data on the stack locks execution on an operating system thread to the current executing functionality, until that functionality completes and the stack usage is released piece by piece. In order to add continuations to standard Python®, that data needed to be stored on the heap instead, therefore decoupling the executing functionality from the stack. With the extensive changes this required in place, Christian released Stackless Python.

Maintaining the fork of a programming language is a lot of work, and when the programming language changes in ways that are incompatible with the changes in the fork, then that work is sigificantly increased. Over time it became obvious that the amount of changes to standard Python® were too much weight to carry, and Christian contemplated a rewrite of Stackless. It became obvious that a simpler approach could be taken, where Stackless was no longer stackless and no longer had continuations.

Following the rewrite, a framework was designed and added inspired by coming from CSP and the Limbo programming language. From this point on, Stackless was in a state where it contained the minimum functionality to give the benefits it aimed to provide, with the minimum amount of work required to keep it maintained.

A few years later in 2004, while sprinting on Stackless in Berlin, Christian and Armin Rigo came up with a way to take the core functionality of Stackless and build an extension module that provided it. This was the creation of greenlets, which are very likely a more popular tool than Stackless itself today. The greenlet source code in practice can be used as the base for green threading functionality not just in the Python® programming language, but in other programming languages and projects.

With Stackless-Python a solid product, Christian’s focus moved onto other projects, PyPy among them. One of his interests in PyPy was a proper implementation of the Stackless functionality, where it could be integrated as a natural part of any Python® interpreter built.

For a while, Stackless-Python languished, with no new versions to match the releases of standard Python® itself. Then in 2006, CCP sent Kristján Valur Jonsson and Richard Tew to PyCon where they sprinted with the aid of Christian Tismer. The result was an up to date release of Stackless-Python. From this point in time, maintaining and releasing Stackless-Python has been undertaken by Richard and Kristján. A few years later Anselm Kruis joined the team.

[1]“Python” and the Python logos are trademarks or registered trademarks of the Python Software Foundation, used by Stackless-Python with permission from the Foundation. See http://www.python.org/psf/trademarks/ for details.
[2]With the term “standard Python®” we refer to the reference implementation of the Python® programming language that is released by the Python Software Foundation on http://www.python.org.