Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Allow a Python exception to be raised without throwing for improved performance #1853
While probably not a big deal in general code, this patch might result in significant speedups in critical code paths, such as raising an
After this patch and a change where
To be extra sure about the impact, I also tried
Important thing to note is that for conversion to
This is my first real PR to this project and while I tried to follow code style and added a test case, I most probably missed something important (even though I don't think this particular change should break anyone's use case) :) Let me know if you need more information or more testing / benchmarks. Happy to address your feedback. Thank you!
While probably not a big deal in general code, this might result in significant speedups in critical code paths, such as raising an IndexError in __getitem__() implementations to stop iterating -- for example conversion of small arrays or math vector classes to lists (or np.array, for example). Measuring with Magnum's Python bindings and the timeit module, the numbers were as follows (Release build, GCC 9, Py3.7): Vector3(1.0, 2.0, 3.0) 0.54144 µs list(Vector3(1.0, 2.0, 3.0)) 5.25274 µs Vector3() # doesn't throw 0.97913 µs Vector3() # throws 5.78696 µs raise IndexError() # throws 0.15339 µs After this patch and a change where Vector3's __getitem__ calls PyErr_SetString() and returns an empty py::object instead of throwing py::index_error, the numbers went down considerably: Vector3(1.0, 2.0, 3.0) 0.55200 µs list(Vector3(1.0, 2.0, 3.0)) 1.60032 µs Vector3() # doesn't throw 0.68562 µs Vector3() # throws 0.84481 µs raise IndexError() # throws 0.17830 µs To be extra sure about the impact, I also tried PyErr_SetString() together with throwing py::error_already_set as that's also less code being executed, but that led only to comparably minor improvement (~4 µs down from 5.8). Important thing to note is that for conversion to np.array and friends, implementing a buffer protocol is far more performant solution (under a microsecond compared to around 6 µs when relying on this patch and almost 20 µs when throwing py::index_error). So this change is mainly aimed at list conversions and general iteration.
This PR makes two tests fail, both expecting the
I'd argue this is fine (the exception still points directly to the problem) and the expected exceptions in those two tests should be updated. Furthermore this might mean that the
Prior to this commit throwing error_already_set was expensive due to the eager construction of the error string (which required traversing the Python stack). See pybind#1853 for more context and an alternative take on the issue. Note that error_already_set no longer inherits from std::runtime_error because the latter has no default constructor.