Major Python update 3.9: What’s new
Python 3.9, released October 5th, brings forward significant changes to both the features of the language and to how the language is developed. Python has mushroomed in popularity the last few years, and its use has exploded in rapidly evolving areas such as data science and machine learning. The project is working hard to keep pace with all of the new demands.
By Serdar Yegulalp, Senior Writer, InfoWorld
Here is a rundown of all of the big new features in Python 3.9.
Python switches to a yearly release cycle
Up until this point, Python has been developed and released on an eighteen-month cadence. PEP 602 proposed that the Python development team adopt an annual release cycle, and that proposal has been accepted.
An annual release cycle means fewer features per release, but it also means faster feedback on feature testing, fewer potentially breaking changes for each release, and thus more incentive for users and Linux distribution managers to upgrade Python more often. It also means new features proposed late in the development cycle won’t take as long to be rolled into a new release.
The new timeline means Python 3.9 will ship in October 2020. Python 3.10 officially started pre-alpha development on May 19, 2020, will enter the alpha development phase when Python 3.9 ships, and will ship in October 2021. Future Python releases will follow the same pattern.
Python becomes faster by default
Every revision of Python enjoys performance improvements over the previous version. Python 3.9 rolls in two big improvements that boost performance without requiring any changes to existing code.
The first improvement involves more use of the vectorcall protocol introduced in Python 3.8. vectorcall makes many common function calls faster by minimizing or eliminating temporary objects created for the call. In Python 3.9, several Python built-ins — range, tuple, set, frozenset, list, dict — use vectorcall internally to speed up execution.
The second big performance enhancer is more efficient parsing of Python source code. The new parser for the CPython runtime wasn’t designed to address performance issues, but rather to deal with internal inconsistencies in the original parser. However, an important fringe benefit is faster parsing, especially for large volumes of code.
More Python string and dictionary functions
Python makes it easy to manipulate common data types, and Python 3.9 extends this ease with new features for strings and dictionaries. For strings, there are new methods to remove prefixes and suffixes, operations that have long required a lot of manual work to pull off. For dictionaries, there are now union operators, one to merge two dictionaries into a new dictionary and one to update the contents of one dictionary with another dictionary.
Decorators lose some restrictions
Decorators let you wrap Python functions to alter their behaviors programmatically. Previously, decorators could only consist of the @ symbol, a name (e.g. func) or a dotted name (func.method) and optionally a single call (func.method(arg1, arg2)). With Python 3.9, decorators can now consist of any valid expression.
One long-standing way to get around this restriction was to create a function or lambda expression that would stand in for a more complex expression when used as a decorator. Now any expression will do, provided it yields something that can function as a decorator.
New Python type operations
Over the last few versions, Python has expanded support for type hinting. This is mainly for the sake of linters and code checkers; types aren’t enforced at runtime in CPython, and there are no plans to make Python a statically typed language. But type hinting is a powerful tool to ensure consistency in large codebases, so Python code can still benefit from having type hints.
Two new features for type hinting and type annotations made their way into Python 3.9. In one, type hints for the contents of collections — e.g., lists and dictionaries — are now available in Python natively. This means you can for instance describe a list as list[int] — a list of integers — without needing the typing library to do it.
The second addition to Python’s typing mechanisms is flexible function and variable annotations. This allows the use of the Annotated type to describe a type using metadata that can be examined ahead of time (with linting tools) or at runtime. For instance, Annotated[int, ctype(“char”)] could be used to describe an integer that should be considered as a char type in C. By default, Python would do nothing with such an annotation, but it could be used by code linters.
Improvements to Python internals
Cleaning up, refining, and modernizing Python’s internals is an ongoing initiative for Python’s developers, and Python 3.9 has a couple of changes in that vein.
The first is a redesign of the way modules interact with the import machinery. Python extension modules, written in C, may now use a new loading mechanism that makes them behave more like regular Python modules when imported. Several modules in Python’s standard library newly support this behavior: _abc, audioop, _bz2, _codecs, _contextvars, _crypt, _functools, _json, _locale, operator, resource, time, _weakref. The new loading mechanism not only allows extension modules to be handled more flexibly by Python, but also enables new capabilities such as advanced hooking behaviors.
The second cleanup initiative is a stable internal ABI for CPython, one guaranteed to last for the lifetime of Python 3. Historically, each major revision of Python has been ABI-incompatible with previous versions, requiring extension modules to be recompiled for every new version. From now on, any extension modules that use the stable ABI will work across Python versions. With Python 3.9, the following modules in the standard library use the stable ABI: audioop, ast, grp, _hashlib, pwd, _posixsubprocess, random, select, struct, termios, zlib.
Other changes in Python 3.9
