Skip to content

implement dpnp.nanmedian #2191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Dec 5, 2024
Merged

implement dpnp.nanmedian #2191

merged 9 commits into from
Dec 5, 2024

Conversation

vtavana
Copy link
Collaborator

@vtavana vtavana commented Nov 21, 2024

In this PR, dpnp.nanmedian is implemented.

  • Have you provided a meaningful PR description?
  • Have you added a test, reproducer or referred to issue with a reproducer?
  • Have you tested your changes locally for CPU and GPU devices?
  • Have you made sure that new changes do not introduce compiler warnings?
  • Have you checked performance impact of proposed changes?
  • If this PR is a work in progress, are you filing the PR as a draft?

@vtavana vtavana self-assigned this Nov 21, 2024
Copy link
Contributor

github-actions bot commented Nov 21, 2024

View rendered docs @ https://intelpython.github.io/dpnp/index.html

@vtavana vtavana marked this pull request as ready for review November 22, 2024 13:24
@vtavana
Copy link
Collaborator Author

vtavana commented Dec 4, 2024

Timing for median function: dpnp shows a better performance compared to numpy, in spite of the fact that dpnp uses sorting in calculating median which is not efficient.

Intel(R) oneAPI Unified Runtime over Level-Zero, Intel(R) Data Center GPU Max 1100 12.60.7 [1.6.31294+9]
Intel(R) OpenCL, Intel(R) Xeon(R) Platinum 8480+ OpenCL 3.0 (Build 0) [2024.18.9.0.28_160000]
import dpnp, numpy
a = numpy.ones((16*8192, 32))
axis = 1
%timeit numpy.median(a, axis=axis)

a_cpu = dpnp.array(a, device="cpu")
%timeit dpnp.median(a_cpu, axis=axis); a_cpu.sycl_queue.wait();

a_gpu = dpnp.array(a, device="gpu")
%timeit dpnp.median(a_gpu, axis=axis); a_gpu.sycl_queue.wait();
median Function NumPy CPU-Xeon GPU-PVC
(2048, 2048), axis=None 17.4 ms ± 47.5 μs 16.9 ms ± 2.12 ms 2.47 ms ± 6.8 μs
(2048, 2048), axis=0 75.1 ms ± 164 μs 20.7 ms ± 1.21 ms 2.19 ms ± 4.16 μs
(2048, 2048), axis=1 17.2 ms ± 87.6 μs 15.3 ms ± 7.45 ms 1.78 ms ± 3.49 μs
(16*8192, 32), axis=None 17.3 ms ± 49.4 μs 16.5 ms ± 357 μs 2.5 ms ± 23.6 μs
(16*8192, 32), axis=0 71.5 ms ± 190 μs 18.2 ms ± 1.89 ms 2.55 ms ± 290 μs
(16*8192, 32), axis=1 19.5 ms ± 148 μs 10.7 ms ± 2.68 ms 2.34 ms ± 5.2 μs

@vtavana
Copy link
Collaborator Author

vtavana commented Dec 4, 2024

Timing for nanmedian function: When axis is not None and there a large number of axes, dpnp is slower than NumPy. This is expected because calculating nanmedian requires looping over each axis and computing the median while ignoring NaNs in that specific axis.

Intel(R) oneAPI Unified Runtime over Level-Zero, Intel(R) Data Center GPU Max 1100 12.60.7 [1.6.31294+9]
Intel(R) OpenCL, Intel(R) Xeon(R) Platinum 8480+ OpenCL 3.0 (Build 0) [2024.18.9.0.28_160000]
import dpnp, numpy
a = numpy.ones((16*8192, 32))
axis = 1
# Randomly set some elements to NaN:
w = numpy.random.random((len(a.shape), 200)) * numpy.array(a.shape)[:, None]
w = w.astype(numpy.intp)
a[tuple(w)] = numpy.nan
%timeit numpy.nanmedian(a, axis=axis)

a_cpu = dpnp.array(a, device="cpu")
%timeit dpnp.nanmedian(a_cpu, axis=axis); a_cpu.sycl_queue.wait();

a_gpu = dpnp.array(a, device="gpu")
%timeit dpnp.nanmedian(a_gpu, axis=axis); a_gpu.sycl_queue.wait();
nanmedian Function NumPy CPU-Xeon GPU-PVC
(2048, 2048), axis=None 19.3 ms ± 191 μs 27.1 ms ± 7.1 ms 4.38 ms ± 5.39 μs
(2048, 2048), axis=0 74.7 ms ± 66.7 μs 3.24 s ± 63.7 ms 1.23 s ± 5.14 ms
(2048, 2048), axis=1 34.1 ms ± 56.1 μs 3.21 s ± 58.1 ms 1.25 s ± 9.01 ms
(16*8192, 32), axis=None 18.7 ms ± 41.1 μs 25.5 ms ± 4.5 ms 4.41 ms ± 16.1
(16*8192, 32), axis=0 51.8 ms ± 83 μs 72.5 ms ± 4.56 ms 20.1 ms ± 215 μs
(16*8192, 32), axis=1 89.2 ms ± 265 μs 3min 31s ± 1.53 s 1min 19s ± 1.28 s

Copy link
Contributor

@antonwolfy antonwolfy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @vtavana, LGTM!

@vtavana vtavana merged commit e0c9cf1 into master Dec 5, 2024
46 of 48 checks passed
@vtavana vtavana deleted the impl-nanmedian branch December 5, 2024 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants