[SciPy-Dev] GIL and gaussian filter

[Matthias Kümmerer]

Hi,

I am using scipy.ndimage.filters.gaussian_filter a lot. Often, I have to blur a
range of arrays. This could be done perfectly in parallel, however the
function does not release the GIL thus I cannot use threading while
multiprocessing has quite a big overhead. When I understood the code correct,
there are no python calls in NI_Correlate1D, which is what ultimately performs
the filter.

Thus I wanted to ask whether it is possible to release the GIL inside the
correlate function and make parallel filtering easier. I will be happy to help
with this, if you can give me some hints where I have to search.

Best,
Matthias

[Sturla Molden]

Post by Matthias Kümmerer
Thus I wanted to ask whether it is possible to release the GIL inside the
correlate function and make parallel filtering easier. I will be happy to help
with this, if you can give me some hints where I have to search.

Possible, but not recommended. The way gaussian_filter is written, you
would have to release and re-aquire the GIL for every axis in the ndimage.
This is an expensive operation. It is better ti keep everything in C or
Cython, so you can release the GIL only once.

Two other comments:

- It is better to do the multi-threading inside the gaussian_filter
function.

- It is possible to express a Gaussian filter as a recursive IIR filter,
which is much faster than convolution. Also it does not truncate the filter
kernel and the speed is not dependent on the kernel size.

Sturla