ROOT on Conda Forge

Linux and macOS packages for Python 2.7, 3.6, and 3.7

For High Energy Physics, the go-to framework for big data analysis has been CERN’s ROOT framework. ROOT is a massive C++ library that even predates the STL in some areas. It is1 also a JIT C++ interpreter called Cling, probably the best in the business. If you have heard of the Xeus C++ Kernel for Jupyter, that is built on top of Cling. ROOT has everything a HEP physicist could want: math, plotting, histograms, tuple and tree structures, a very powerful file format for IO, machine learning, Python bindings, and more. It also does things like dictionary generation and arbitrary class serialization (other large frameworks like Qt have similar generation tools).

You may already be guessing one of the most common problems for ROOT. It is huge and difficult to install – if you build from source, that’s a several hour task on a single core. It has gotten much better in the last 5 years, and there are several places you can find ROOT, but there are still areas where it is challenging. This is especially true for Python; ROOT is linked to just one version of Python, and the one you get with pre-built ROOT can often be the wrong one. And, if you use the Anaconda Python distribution, which is the most popular scientific distribution of Python and massively successful for ML frameworks, the general rule even for people who build ROOT themselves has been: don’t. But now, you can get a fully featured ROOT binary package for macOS or Linux, Python 2.7, 3.6, or 3.6, from Conda-Forge, the most popular Anacanda community channel!

[Read More]

ROOT Install Options

New Conda Forge package of ROOT for Unix and more options

For particle physicists, ROOT is one of the most important toolkits around. It is a huge suite of tools that predates the C++ standard library, and has almost anything a particle physicist could want. It has driven developments in other areas too. ROOT’s current C++ interpreter, CLING, is the most powerful C++ interpreter available and is used by the Xeus project for Jupyter. The Python work has helped PyPy, with CPPYY also coming from ROOT. However, due to the size, complexity, and age of some parts of ROOT, it can be a bit challenging to install; and it is even more challenging when you want it to talk to Python. I would like to point to the brand-new Conda-Forge ROOT package for Linux and macOS, and point out a few other options for macOS installs. Note for Windows users: Due to the fact that ROOT expects the type long to match the system pointer size, 64-bit Windows cannot be supported for quite some time. While you can use it in 32 bit form, this is generally impossible to connect to Python, which usually will be a 64-bit build.

[Read More]

Histogram Speeds in Python

Let’s compare several ways of making Histograms. I’m going to assume you would like to end up with a nice OO histogram interface, so all the 2D methods will fill a Physt histogram. We will be using a 2 x 1,000,000 element array and filling a 2D histogram, or 10,000,000 elemends in a 1D histogram. Binnings are regular.

1D 10,000,000 item histogram

Example KNL MBP X24
Numpy: histogram 704 ms 147 ms 114 ms
Numpy: bincount 432 ms 110 ms 117 ms
fast-histogram 337 ms 45.9 ms 45.7 ms
Numba 312 ms 58.8 ms 60.7 ms

2D 1,000,000 item histogram

Example KNL MBP X24
Physt 1.21 s 293 ms 246 ms
Numpy: histogram2d 456 ms 114 ms 88.3 ms
Numpy: 247 ms 62.7 ms 49.7 ms
Numpy: bincount 81.7 ms 23.3 ms 20.3 ms
fast-histogram 53.7 ms 10.4 ms 7.31 ms
fast-hist threaded 0.5 (6) 62.5 ms 9.78 ms (6) 15.4 ms
fast-hist threaded (m) 62.3 ms 4.89 ms 3.71 ms
Numba 41.8 ms 10.2 ms 9.73 ms
Numba threaded (6) 49.2 ms 4.23 ms (6) 4.12 ms
Cython 112 ms 12.2 ms 11.2 ms
Cython threaded (6) 128 ms 5.68 ms (8) 4.89 ms
PyBind11 sequential 93.9 ms 9.20 ms 17.8 ms
PyBind11 OpenMP atomic 4.06 ms 6.87 ms 1.91 ms
PyBind11 C++11 atomic (32) 10.7 ms 7.08 ms (48) 2.65 ms
PyBind11 C++11 merge (32) 23.0 ms 6.03 ms (48) 4.79 ms
PyBind11 OpenMP merge 8.74 ms 5.04 ms 1.79 ms
[Read More]