🎡 cibuildwheel 2.0

The cibuildwheel package has just had a major release with some fantastic features. Python 2.7 and 3.5 support has been removed (and PyPy3.6), allowing us to update to the latest manylinux and auditwheel versions, and support the newly unified manylinux PyPy3.7 images. We now allow users to select pypa/build as a build frontend. We now have a custom option to enable pre-release Pythons (3.10 currently) for testing before they are ABI stable (please don’t release wheels until that happens). Maybe most exciting, cibuildwheel now supports configuration in pyproject.toml, allowing you to be even further isolated from dependence on your CI system; you can easily produce Linux and Windows wheels locally (macOS still installs to system locations). And, since my last post and introduction post, cibuildwheel is now part of the PyPA!

Config mode

The most exciting new feature is the new config mode. This allows you to specify anything you would have added as environment variables in your pyproject.toml file as static configuration. For example, almost every package activates cibuildwheel’s fantastic testing system, which installs the newly build wheel in a fresh environment and runs whatever command you give. Before, this meant that each CI you used had to have an environment variable specification like this for each CI provider:

env:
  CIBW_TEST_EXTRAS = "test"
  CIBW_TEST_COMMAND = "pytest {project}/tests"

Now, you can add the following to your pyproject.toml:

[tool.cibuildwheel]
test-extras = "test"
test-command = "pytest {project}/tests"

And, if you run this locally, you don’t have to add those environment variables manually!

CIBW_BUILD="cp39*" pipx run cibuildwheel --platform linux

You can override any setting with an environment variable, and you can specify a different file name with --config-file if you want to.

So what should you specify in TOML, and what should remain environment variables? I would recommend all static settings be in your pyproject.toml, such that a default run of cibuildwheel does something sensible. Configuration (including platform-specific configuration), skipping things you do not support, etc. Variables that depend on which matrix item you are in, such as CIBW_BUILD, should remain environment variables. Things that are specific to the machine you are on (like Linux archs) likely should remain command line / environment variables too. MacOS architectures probably could be specified in TOML, however, since that simply requires a recent Xcode. For example:

[tool.cibuildwheel.macos]
archs = ["auto", "universal2"] # or whatever you support
test-skip = ["*universal2:arm64"]

The docs now contain tabs with both configuration methods for each setting, along with improved examples. cmake is a great example of a rather complex configuration setup. There is a separate documentation branch for the 1.x series if you need to access the information about working around Python 2 quirks.

Build selection

You can now select build frontends; the classic “pip” frontend, or pypa/build.

[tool.cibuildwheel]
build-frontend = "build"

Please try it out and report any problems you come across. Eventually, we might change the default to build. You should already be building your SDists with pipx run build --sdist.

Python support

We now support pre-release Pythons that are not yet recommended for wheel upload. To enable this, set CIBW_PRERELEASE_PYTHONS or pass --prerelease-pythons on the command line. It is not a good idea to “burn” this into your pyproject.toml, as it’s only for testing. When Python enters RC phase, you can start uploading extensions to PyPI, so the next version of Python (3.10 currently) will leave the prerelease phase.

PyPy unification

We integrated PyPy3.7 into the official manylinux images, so besides the manylinux2010-based image that was previously available, you can now build with the official manylinux2010, manylinux2014 and manylinux_2_24 images (regardless of if you use cibuildwheel or not). We also are using PyPy 3.7.5 now, which fixes several potential issues, and builds 64-bit wheels (only) on Windows. This also means we support PyPy on ARM and 32-bit Linux. If you support PyPy, it is highly recommended you upgrade to cibuildwheel 2.0.

Also, if you are using the manylinux images directly, you now can use python3.Y shortcuts, and pipx is now installed; this means you can even use manylinux to run tools like tox and nox! For example:

docker run --rm -itv $PWD:/src -w /src quay.io/pypa/manylinux_2_24_x86_64:latest pipx run nox

cibuildwheel uses this in its noxfile to run pip-tools compile on all versions of Python, actually.

Everything else

We finally can use recent manylinux images (since manylinux dropped Python 2.7 and Python 3.5 some time ago) allow us to use auditwheel4, which produces slightly different wheel names - they include the classic tag as well as the new GLIBC based tag.

While it was backported to the 1.x series, support for Apple Silicon Python 3.8 wheels was added to the existing Python 3.9 support. The GitHub Action now redirects stderr to stdout, fixing the out-of-order logs sometimes produces. Developer tasks now use nox, simplifying our maintainers and making it easier to contribute.

Wrapup

All told, there has never been a better time to build wheels for Python projects. The new configuration mode makes it easy to run Linux and Windows locally, and simplifies deployment on multiple CI providers. Python 3.10 support, PyPA/build, PyPy unification, and removal of old Pythons all make cibuildwheel more capable and powerful for building modern wheels.

We have plans for the future, too; better support for Limited API (stable ABI) projects (for testing mostly, they are easy to build today - just limit your build to one Python version). We hope to provide examples for extracting and building from an SDist. We hope to continue to improve the documentation, both ours,

comments powered by Disqus