Add different dtype input test in histogram #3618

okuta · 2020-07-15T19:16:11Z

cub.device_histogram assumes following dtype combination.

same float dptye x and bin_edge
x is integer dtype, binedge is double dtype

leofang · 2020-07-15T19:49:04Z

cub.device_histogram assumes following dtype combination.

same float dptye x and bin_edge

x is integer dtype, binedge is double dtype

It's not CUB's requirement, but actually NumPy's:
https://github.com/numpy/numpy/blob/2f3b82638a24204a13c9b16a3a36a59f199df41d/numpy/lib/histograms.py#L442-L443
So I simply used SFINAE to specialize kernels that satisfy this requirement:

cupy/cupy/cuda/cupy_cub.cu

Lines 585 to 589 in 0af4d97

    
           struct _cub_histogram_range { 
        
               template <typename sampleT, 
        
                         typename binT = typename If<std::is_integral<sampleT>::value, double, sampleT>::Type> 
        
               void operator()(void* workspace, size_t& workspace_size, void* input, void* output, 
        
                   int n_bins, void* bins, size_t n_samples, cudaStream_t s) const

leofang · 2020-07-15T19:51:03Z

cupy/statistics/histogram.py

+                else:
+                    x = x.astype(bin_type, copy=False)


We don't need this. It's usually cheaper to change the bins than the input array, as the latter is often much larger.

OK, I will fix!

leofang · 2020-07-15T19:58:43Z

tests/cupy_tests/statics_tests/test_histogram.py

+        bins = testing.shaped_arange((10,), xp, dtype_a)
+
+        if xp is numpy:
+            return xp.histogram(x, bins)[0]


Can we create another test, say test_histogram_with_bins2, to test xp.histogram(x, bins)[1]?

leofang · 2020-07-15T19:59:57Z

cupy/statistics/histogram.py

+                    bin_type = numpy.float
+                else:
+                    x = x.astype(bin_type, copy=False)
+                acc_bin_edge = bin_edges.astype(bin_type, copy=True)


Do I understand correctly you wanna hide the cost of the second bin shift by creating a copy?

leofang · 2020-07-15T20:00:36Z

cupy/statistics/histogram.py

+                assert isinstance(bin_edges, cupy.ndarray)
+                bin_type = numpy.result_type(bin_edges.dtype, x.dtype)
+                if numpy.issubdtype(x.dtype, numpy.integer):
+                    bin_type = numpy.float


This is cleaner 👍 But, when _get_bin_edges() generates bin_edges, if bins is int this type requirement is already satisfied, so it's why we first test for if isinstance(bins, cupy.ndarray):. Would be nice to keep the if branch.

Certainly, it seems to be faster.
The point is that bins needs type conversion even when numpy.array or list is an input.
How about if not is instance(bins, int):?

Yeah let's prepare for #3542.

tests/cupy_tests/statics_tests/test_histogram.py

takagi · 2020-07-16T07:38:06Z

pfnCI, test this please.

pfn-ci-bot · 2020-07-16T07:38:10Z

Successfully created a job for commit 4ea497f:

Dashboard for commit 4ea497f

chainer-ci · 2020-07-16T07:59:00Z

Jenkins CI test (for commit 4ea497f, target branch master) failed with status FAILURE.

takagi · 2020-07-18T08:30:07Z

The CI also gets wrong with float32?

>>> import cupy
>>> x = cupy.testing.shaped_arange((10,), cupy, 'f')
>>> bins = cupy.arange(1, 10.1, 0.9)
>>> bins[-1] = 10.1
>>> y = cupy.empty(10, 'l')
>>> cupy.cuda.cub.device_histogram(x, bins, y)
array([1, 1, 0, 0, 0, 0, 0, 8, 0, 0])
>>> bins = bins.astype('f')
>>> cupy.cuda.cub.device_histogram(x, bins, y)
array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1])

okuta · 2020-07-19T07:21:49Z

The CI also gets wrong with float32?

We have to use double type bins, so bins = bins.astype('d') is correct. @takagi

leofang · 2020-07-19T23:57:17Z

In @takagi's snippet, x is fp32, so bins should also be fp32.

kmaehashi · 2020-08-04T07:22:14Z

Blocked by #3617

takagi · 2020-08-25T06:29:16Z

As #3617 is merged to the master now, would you resolve the conflict?

okuta · 2020-08-26T10:45:32Z

I fixed conflicts. So, this PR only adds testcase

takagi · 2020-08-28T08:59:42Z

pfnCI, test this please.

chainer-ci · 2020-08-28T09:00:14Z

Jenkins CI test (for commit 229e52e, target branch master) failed with status FAILURE.

okuta · 2020-08-31T02:18:41Z

pfnCI, test this please.

chainer-ci · 2020-08-31T02:19:14Z

Jenkins CI test (for commit 229e52e, target branch master) failed with status FAILURE.

okuta · 2020-08-31T07:47:54Z

pfnCI, test this please.

chainer-ci · 2020-08-31T08:06:50Z

Jenkins CI test (for commit 229e52e, target branch master) failed with status FAILURE.

takagi · 2020-08-31T15:29:11Z

Waiting for CI problem fixed.

takagi · 2020-09-08T04:38:17Z

pfnCI, test this please.

chainer-ci · 2020-09-08T05:01:03Z

Jenkins CI test (for commit 229e52e, target branch master) failed with status FAILURE.

Co-authored-by: Leo Fang <leofang@bnl.gov>

okuta · 2020-09-16T07:53:10Z

pfnCI, test this please.

chainer-ci · 2020-09-16T08:49:19Z

Jenkins CI test (for commit 306ed6f, target branch master) succeeded!

takagi · 2020-09-17T13:07:10Z

LGTM!

Add different dtype input test in histogram

okuta changed the title ~~Fix diff dtype histogram~~ Fix different dtype input case in histogram Jul 15, 2020

leofang mentioned this pull request Jul 15, 2020

Fix float16 ndarray input in histogram with CUB #3617

Merged

leofang suggested changes Jul 15, 2020

View reviewed changes

kmaehashi assigned takagi Jul 16, 2020

kmaehashi added cat:bug Bugs st:blocked-by-another-pr Blocked by another pull-request labels Aug 4, 2020

takagi removed the st:blocked-by-another-pr Blocked by another pull-request label Aug 25, 2020

okuta force-pushed the fix-diff-dtype-histogram branch from 4ea497f to 229e52e Compare August 26, 2020 04:45

okuta changed the title ~~Fix different dtype input case in histogram~~ Add different dtype input test in histogram Aug 26, 2020

takagi added the to-be-backported Pull-requests to be backported to stable branch label Sep 15, 2020

Fix different dtype input case in histogram

a813579

okuta and others added 5 commits September 16, 2020 16:50

Update tests/cupy_tests/statics_tests/test_histogram.py

0ac2c4d

Co-authored-by: Leo Fang <leofang@bnl.gov>

Fix

6d47664

Fix and add test

40dd5d4

Fix

79bdbc3

Fix module name

306ed6f

okuta force-pushed the fix-diff-dtype-histogram branch from 229e52e to 306ed6f Compare September 16, 2020 07:53

takagi added this to the v9.0.0a1 milestone Sep 17, 2020

takagi merged commit 2351f05 into cupy:master Sep 17, 2020

takagi added a commit to takagi/cupy that referenced this pull request Sep 17, 2020

Merge pull request cupy#3618 from okuta/fix-diff-dtype-histogram

84851c9

Add different dtype input test in histogram

takagi mentioned this pull request Sep 17, 2020

[backport] Add different dtype input test in histogram #4006

Closed

takagi added cat:test Test code / CI and removed cat:bug Bugs to-be-backported Pull-requests to be backported to stable branch labels Sep 18, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add different dtype input test in histogram #3618

Add different dtype input test in histogram #3618

okuta commented Jul 15, 2020

leofang commented Jul 15, 2020

leofang Jul 15, 2020

okuta Jul 16, 2020

leofang Jul 15, 2020

okuta Jul 20, 2020

leofang Jul 15, 2020

okuta Jul 20, 2020

leofang Jul 15, 2020

okuta Jul 16, 2020

leofang Jul 16, 2020

takagi commented Jul 16, 2020

pfn-ci-bot commented Jul 16, 2020

chainer-ci commented Jul 16, 2020

takagi commented Jul 18, 2020 •

edited

Loading

okuta commented Jul 19, 2020

leofang commented Jul 19, 2020 •

edited

Loading

kmaehashi commented Aug 4, 2020

takagi commented Aug 25, 2020

okuta commented Aug 26, 2020

takagi commented Aug 28, 2020

chainer-ci commented Aug 28, 2020

okuta commented Aug 31, 2020

chainer-ci commented Aug 31, 2020

okuta commented Aug 31, 2020

chainer-ci commented Aug 31, 2020

takagi commented Aug 31, 2020

takagi commented Sep 8, 2020

chainer-ci commented Sep 8, 2020

okuta commented Sep 16, 2020

chainer-ci commented Sep 16, 2020

takagi commented Sep 17, 2020

Add different dtype input test in histogram #3618

Add different dtype input test in histogram #3618

Conversation

okuta commented Jul 15, 2020

leofang commented Jul 15, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

takagi commented Jul 16, 2020

pfn-ci-bot commented Jul 16, 2020

chainer-ci commented Jul 16, 2020

takagi commented Jul 18, 2020 • edited Loading

okuta commented Jul 19, 2020

leofang commented Jul 19, 2020 • edited Loading

kmaehashi commented Aug 4, 2020

takagi commented Aug 25, 2020

okuta commented Aug 26, 2020

takagi commented Aug 28, 2020

chainer-ci commented Aug 28, 2020

okuta commented Aug 31, 2020

chainer-ci commented Aug 31, 2020

okuta commented Aug 31, 2020

chainer-ci commented Aug 31, 2020

takagi commented Aug 31, 2020

takagi commented Sep 8, 2020

chainer-ci commented Sep 8, 2020

okuta commented Sep 16, 2020

chainer-ci commented Sep 16, 2020

takagi commented Sep 17, 2020

takagi commented Jul 18, 2020 •

edited

Loading

leofang commented Jul 19, 2020 •

edited

Loading