Histogram Speeds in Python

Let’s compare several ways of making Histograms. I’m going to assume you would like to end up with a nice OO histogram interface, so all the 2D methods will fill a Physt histogram. We will be using a 2 x 1,000,000 element array and filling a 2D histogram, or 10,000,000 elemends in a 1D histogram. Binnings are regular.

1D 10,000,000 item histogram

Example KNL MBP X24
Numpy: histogram 704 ms 147 ms 114 ms
Numpy: bincount 432 ms 110 ms 117 ms
fast-histogram 337 ms 45.9 ms 45.7 ms
Numba 312 ms 58.8 ms 60.7 ms

2D 1,000,000 item histogram

Example KNL MBP X24
Physt 1.21 s 293 ms 246 ms
Numpy: histogram2d 456 ms 114 ms 88.3 ms
Numpy: add.at 247 ms 62.7 ms 49.7 ms
Numpy: bincount 81.7 ms 23.3 ms 20.3 ms
fast-histogram 53.7 ms 10.4 ms 7.31 ms
fast-hist threaded 0.5 (6) 62.5 ms 9.78 ms (6) 15.4 ms
fast-hist threaded (m) 62.3 ms 4.89 ms 3.71 ms
Numba 41.8 ms 10.2 ms 9.73 ms
Numba threaded (6) 49.2 ms 4.23 ms (6) 4.12 ms
Cython 112 ms 12.2 ms 11.2 ms
Cython threaded (6) 128 ms 5.68 ms (8) 4.89 ms
PyBind11 sequential 93.9 ms 9.20 ms 17.8 ms
PyBind11 OpenMP atomic 4.06 ms 6.87 ms 1.91 ms
PyBind11 C++11 atomic (32) 10.7 ms 7.08 ms (48) 2.65 ms
PyBind11 C++11 merge (32) 23.0 ms 6.03 ms (48) 4.79 ms
PyBind11 OpenMP merge 8.74 ms 5.04 ms 1.79 ms
[Read More]

Python 3 upgrade

About ten years ago, Guido Van Rossum, the Python author and Benevolent Dictator for Life (BDFL), along with the Python community, decided to make several concurrent backward incompatible changes to Python 2.5 and release a new version, Python 3.0. The main changes were:

  • Using unicode strings as default, with the old string type becoming a full featured binary type
  • Changing several builtins, for example
    • The print statement became a function, allowing more consistent syntax and the use of the word print as a name
    • The confusing input removed, and raw_input now renamed to input
    • Simpler exec
    • Division is now split between / float division and // truncating division
  • Improved exception tracing, with chaining
  • Improved function call syntax with annotations and keyword only arguments, replacing little used tuple parameter unpacking
  • More class constructor features, such as nicer metaclass syntax, keyword arguments, __prepare__
  • Renamed standard libraries, to be more consistent
  • Removal of a lot of depreciated features, including old-style classes
  • Removal of a lot of depreciated syntax that had become learner stumbling blocks
  • Adding nonlocal variables
  • Extended tuple unpacking, like first, *rest = makes_a_tuple()
  • Removing the proliferation of .pyc files, instead using __pycache__ directories
  • Automatic selection of C-based standard library modules over pure Python ones if available
  • Unified the int and long types into one unlimited length integer type
[Read More]

A simple introduction to asyncio

This is a simple explanation of the asyncio module and new supporting language features in Python 3.5. Even though the new keywords async and await are new language constructs, they are mostly1 useless without an event loop, and that is supplied in the standard library as asyncio. Also, you need awaitable functions, which are only supplied by asyncio (or in the growing set of async libraries, like asyncssh, quamash etc.).

[Read More]