I’ve spent the last few years trying to make it easy for anyone to extend Python with compiled languages. I’ve worked on pybind11, a powerful C++ library that allows users to write advanced Python extensions using just C++11, used by some of the largest projects, SciPy, PyTorch, Google, LLVM, and tens of thousands of other libraries, down to very small extensions. I also work on cibuildwheel, which makes building binaries (called wheels) on continuous integration (CI) simple. It is again powerful enough to used by huge projects, like Scikit-learn, matplotlib, mypy; and is simple enough to be used by hundreds of other packages. Recently it was accepted into the Python Packaging Authority (PyPA). There is one missing piece, though, to complete this picture of compiled extensions that easy to use for small projects, and powerful enough for large projects: the build system. I believe the solution to that is scikit-build, and I’d like to work on it over the next three years, and I need your help.
Scikit-build is a tool for integrating a package with a CMake build system into
Python. You can utilize the vast collection of packages and projects using
CMake already, and you have access to modern building features, like
multithreaded builds, library discovery, superb compiler and IDE support, and
all sorts of extended tooling. Modern CMake is quite pleasant to write compared
to times past; I have written a book and training course on
it. We ship up-to-date
ninja wheels for all binary platforms.
I’m writing a proposal for an NSF CSSI Elements project containing three parts. The first part will cover core development on Scikit-build to address the current shortcomings and to prepare it for a post-distutils (Python 3.12+) world. The second part would cover assisting libraries with a science use case in either transitioning to scikit-build (ideally from an existing CMake build system with Python bindings, but I can help mentor developers in writing bindings (ideally pybind11), setting up CI, and writing CMake code as well (see my book or workshop on Modern CMake, and I’m happy to help old scikit-build projects transition to better practices). As part of this, I would be building up the examples and documentation, leading into the third part of the proposal: A series of training events and training material, including plans for something alongside SciPy.
Do you have an interesting use case? I’d love to hear about it. Feel free to
reach out to me on twitter
@henryschreiner3 or by email,
princeton.edu. I’m looking projects that would be interested in trying out
the new scikit-build during years two and three. Tangibly, I’m looking for
and a 1-3 sentence description of how we will
collaborate over the course of the three year project - I can help you try out
scikit-build over the next three years! After the break, I will give a few more
details about the problem and outline my specific plans in a bit more detail.
You can also see an outline at
or at the end of this post. Deadline is Dec 8, 2021, so letters need to be completed
by the 7th or sooner!