A test runner for free threaded Python
Starting with the 3.13 release, CPython has support for a build of Python called free threading where the global interpreter lock (GIL) is disabled. Free-threaded execution allows for full utilization of the available processing power by running threads in parallel on available CPU cores. While not all software will benefit from this automatically, programs designed with threading in mind will run faster on multi-core hardware.
Before today I'd never actually exploded free-threaded Python. I decided to take a dive and see if I could build a simple test runner that utilises free-threading.
The main advantage here is that code that's initially loaded at import time is only ever executed once. Whereas with something like pytest-xdist every single worker has to pay that upfront cost.
The major downside is that obviously you might end up shooting yourself in the foot because different threads of execution might modify global state, which can be common in tests without good discipline.
It turns out this was relatively easy to achieve, I was quickly able to whip something together that behaves similarly to pytest-xdist but with free threading.
Caching
As a little bonus challenge I decided to try and add some caching support. Typically to do this in Python you'd have to reach for some kind of build tool like bezel or Pants. Other ecosystems (like Go) achieve this using information from the compiler.
In Python it's pretty tricky to know how a change in one location might affect the rest of your codebase. It's also tricky to work out a dependency graph for each file.
ruff has some experimental tooling in the form of
uv run ruff analyze graph
to create a map of Python file dependencies within your project. The output looks something like:
{
"src/calc/__init__.py": [],
"src/calc/arith.py": [],
"src/calc/stats.py": [
"src/calc/arith.py"
],
"src/calc/text.py": [],
"src/calc/todos.py": [],
"tests/fixtures/numbers.py": [],
"tests/fixtures/todos.py": [
"src/calc/todos.py"
],
"tests/test_arith.py": [
"src/calc/arith.py"
],
"tests/test_catalog.py": [
"src/calc/todos.py",
"tests/fixtures/todos.py"
],
"tests/test_stats.py": [
"src/calc/stats.py",
"tests/fixtures/numbers.py"
],
"tests/test_text.py": [
"src/calc/text.py"
],
"tests/test_todos.py": [
"src/calc/todos.py",
"tests/fixtures/todos.py"
]
}
Using that info is enough to naively determine* which tests to execute best on changes to our codebase since the last execution.
*The whole thing works pretty well, albeit with some pretty major limitations.
