Lead Image © Lucy Baldwin, 123RF.com

Mastering the Way of Parallel Python

Snake Oil

Article from ADMIN 91/2026

By Federico Lucifredi

For all its success, Python's reach into parallelism in all its forms has been limited to a great many, yet confusingly suboptimal, strategies. But that is changing.

C and its evil brother C++ are the languages of choice for systems programming, because they compile by mapping directly to CPU instructions, introducing no additional overhead for functionality not in use in your program – and no runtime performance taxes. The cost of this lean performance is a somewhat more difficult environment in which to debug code when compared with modern interpreted languages, manual memory management, and the occasional pointer arithmetic purgatory.

These costs were historically worth the performance advantage for production use (something that is now changing rapidly with the increasing use of accelerator offloading to GPUs); however, the costs nevertheless were and remain a serious hindrance when fast prototyping is required. As algorithmic godfather Donald Knuth told us in the 1970s, premature optimization is the root of all evil [1]. You should prototype in a more forgiving environment – one that helps you shape the solution and frees your creativity before perfecting its final form for machine performance.

In the 1990s, Perl opened Pandora's box of expressive, interpreted languages – albeit with a wonderfully expressive and yet equally treacherous syntax. Python polished this path with a perhaps more rigid approach, yet one that groups of programmers can more comfortably share, thanks to its consistency and superior user experience. Allowing some notable exceptions for specialized tools, like designing scientific research analytics in MatLab or Mathematica or perhaps prototyping visual effects in the Processing programming language, the go-to language for prototyping general-purpose code today is unquestionably Python, with its reach constantly expanding as its libraries [2] expand to ever new reaches.