Histogram Speeds in Python

Let’s compare several ways of making Histograms. I’m going to assume you would like to end up with a nice OO histogram interface, so all the 2D methods will fill a Physt histogram. We will be using a 2 x 1,000,000 element array and filling a 2D histogram, or 10,000,000 elemends in a 1D histogram. Binnings are regular.

1D 10,000,000 item histogram

Example KNL MBP X24
Numpy: histogram 704 ms 147 ms 114 ms
Numpy: bincount 432 ms 110 ms 117 ms
fast-histogram 337 ms 45.9 ms 45.7 ms
Numba 312 ms 58.8 ms 60.7 ms

2D 1,000,000 item histogram

Example KNL MBP X24
Physt 1.21 s 293 ms 246 ms
Numpy: histogram2d 456 ms 114 ms 88.3 ms
Numpy: add.at 247 ms 62.7 ms 49.7 ms
Numpy: bincount 81.7 ms 23.3 ms 20.3 ms
fast-histogram 53.7 ms 10.4 ms 7.31 ms
fast-hist threaded 0.5 (6) 62.5 ms 9.78 ms (6) 15.4 ms
fast-hist threaded (m) 62.3 ms 4.89 ms 3.71 ms
Numba 41.8 ms 10.2 ms 9.73 ms
Numba threaded (6) 49.2 ms 4.23 ms (6) 4.12 ms
Cython 112 ms 12.2 ms 11.2 ms
Cython threaded (6) 128 ms 5.68 ms (8) 4.89 ms
PyBind11 sequential 93.9 ms 9.20 ms 17.8 ms
PyBind11 OpenMP atomic 4.06 ms 6.87 ms 1.91 ms
PyBind11 C++11 atomic (32) 10.7 ms 7.08 ms (48) 2.65 ms
PyBind11 C++11 merge (32) 23.0 ms 6.03 ms (48) 4.79 ms
PyBind11 OpenMP merge 8.74 ms 5.04 ms 1.79 ms
[Read More]

A simple introduction to asyncio

This is a simple explanation of the asyncio module and new supporting language features in Python 3.5. Even though the new keywords async and await are new language constructs, they are mostly1 useless without an event loop, and that is supplied in the standard library as asyncio. Also, you need awaitable functions, which are only supplied by asyncio (or in the growing set of async libraries, like asyncssh, quamash etc.).

[Read More]

A little example of how asyncio works

This is a simple example to show how Asyncio works without using Asyncio itself, instead using a basic and poorly written event loop. This is only meant to give a flavor of what Asyncio does behind the curtains. I’m avoiding most details of the library design, like callbacks, just to keep this simple. Since this is written as an illustration, rather than real code, I’m going to dispense with trying to keep it 2.7 compatible.

[Read More]

Slots in Python

Slots seem to be poorly documented. What they do is simple, but whether they are used is tricky. This is a little mini-post on slots.

[Read More]