Performance improvement in kernel launches by using lighter NumPy APIs #258

ybsh · 2019-11-07T07:01:55Z

I have replaced two heavy numpy function calls with lighter operations.
This reduced the train_mnist.py runtime by 20% (with the configuration shown here, noprof-nosync).

I have checked that clpy/tests/example_tests/test_gemm.py has passed.

jenkins-maekawa

Test (commit 00a575a) passed in titanv.

jenkins-maekawa

Test (commit 00a575a) passed in vega.

ybsh · 2019-11-07T08:04:08Z

@LWisteria I'd appreciate your review.

ybsh · 2019-11-08T02:10:13Z

The issue discussing the change: #257

LWisteria

@ybsh I reviewed. Change PR's title. It describes nothing now.

clpy/backend/function.pyx

jenkins-maekawa

Test (commit 99d45fe) failed in titanv.

============================= test session starts ==============================
platform linux -- Python 3.6.5, pytest-4.3.0, py-1.8.0, pluggy-0.13.0
rootdir: /home/clpy-jenkins-slave/workspace/clpy_testing_titan, inifile: setup.cfg
collected 0 items / 23 errors

==================================== ERRORS ====================================
_________ ERROR collecting tests/clpy_tests/core_tests/test_carray.py __________
test_carray.py:3: in <module>
    import clpy
../../../clpy/__init__.py:17: in <module>
    from clpy import core  # NOQA
../../../clpy/core/__init__.py:1: in <module>
    from clpy.core import core  # NOQA
clpy/backend/function.pxd:4: in init clpy.core.core
    ???
../../../clpy/backend/__init__.py:3: in <module>
    from clpy.backend import compiler  # NOQA
clpy/backend/function.pxd:4: in init clpy.backend.compiler
    ???
clpy/backend/function.pyx:38: in init clpy.backend.function
    ???
E   AttributeError: module 'clpy' has no attribute 'core'
__________ ERROR collecting tests/clpy_tests/core_tests/test_core.py ___________
test_core.py:3: in <module>
    import clpy
../../../clpy/__init__.py:17: in <module>
    from clpy import core  # NOQA
../../../clpy/core/__init__.py:1: in <module>
    from clpy.core import core  # NOQA
clpy/core/carray.pxi:14: in init clpy.core.core
    ???
E   AttributeError: type object 'clpy.core.core.Indexer' has no attribute '__reduce_cython__'
__ ERROR collecting tests/clpy_tests/core_tests/test_cupy_aliased_ndarray.py ___
test_cupy_aliased_ndarray.py:6: in <module>
    import clpy  # NOQA
../../../clpy/__init__.py:17: in <module>
    from clpy import core  # NOQA
../../../clpy/core/__init__.py:1: in <module>
    from clpy.core import core  # NOQA
clpy/core/carray.pxi:14: in init clpy.core.core
    ???
E   AttributeError: type object 'clpy.core.core.Indexer' has no attribute '__reduce_cython__'
_______ ERROR collecting tests/clpy_tests/core_tests/test_elementwise.py _______
test_elementwise.py:6: in <module>
    import clpy
../../../clpy/__init__.py:17: in <module>
    from clpy import core  # NOQA
../../../clpy/core/__init__.py:1: in <module>
    from clpy.core import core  # NOQA
clpy/core/carray.pxi:14: in init clpy.core.core

... and more 1221 lines

jenkins-maekawa

Test (commit 99d45fe) failed in vega.

============================= test session starts ==============================
platform linux -- Python 3.6.5, pytest-4.3.0, py-1.8.0, pluggy-0.13.0
rootdir: /home/clpy-jenkins-slave/workspace/clpy_testing_vega, inifile: setup.cfg
collected 0 items / 23 errors

==================================== ERRORS ====================================
_________ ERROR collecting tests/clpy_tests/core_tests/test_carray.py __________
test_carray.py:3: in <module>
    import clpy
../../../clpy/__init__.py:17: in <module>
    from clpy import core  # NOQA
../../../clpy/core/__init__.py:1: in <module>
    from clpy.core import core  # NOQA
clpy/backend/function.pxd:4: in init clpy.core.core
    ???
../../../clpy/backend/__init__.py:3: in <module>
    from clpy.backend import compiler  # NOQA
clpy/backend/function.pxd:4: in init clpy.backend.compiler
    ???
clpy/backend/function.pyx:38: in init clpy.backend.function
    ???
E   AttributeError: module 'clpy' has no attribute 'core'
__________ ERROR collecting tests/clpy_tests/core_tests/test_core.py ___________
test_core.py:3: in <module>
    import clpy
../../../clpy/__init__.py:17: in <module>
    from clpy import core  # NOQA
../../../clpy/core/__init__.py:1: in <module>
    from clpy.core import core  # NOQA
clpy/core/carray.pxi:14: in init clpy.core.core
    ???
E   AttributeError: type object 'clpy.core.core.Indexer' has no attribute '__reduce_cython__'
__ ERROR collecting tests/clpy_tests/core_tests/test_cupy_aliased_ndarray.py ___
test_cupy_aliased_ndarray.py:6: in <module>
    import clpy  # NOQA
../../../clpy/__init__.py:17: in <module>
    from clpy import core  # NOQA
../../../clpy/core/__init__.py:1: in <module>
    from clpy.core import core  # NOQA
clpy/core/carray.pxi:14: in init clpy.core.core
    ???
E   AttributeError: type object 'clpy.core.core.Indexer' has no attribute '__reduce_cython__'
_______ ERROR collecting tests/clpy_tests/core_tests/test_elementwise.py _______
test_elementwise.py:6: in <module>
    import clpy
../../../clpy/__init__.py:17: in <module>
    from clpy import core  # NOQA
../../../clpy/core/__init__.py:1: in <module>
    from clpy.core import core  # NOQA
clpy/core/carray.pxi:14: in init clpy.core.core

... and more 1221 lines

jenkins-maekawa

Test (commit 2e2f173) passed in titanv.

jenkins-maekawa

Test (commit 2e2f173) passed in vega.

ybsh · 2019-11-13T09:01:19Z

@LWisteria I appreciate your review.

jenkins-maekawa

Test (commit 53c1d5f) passed in vega.

jenkins-maekawa

Test (commit 53c1d5f) passed in titanv.

LWisteria

LGTM

LWisteria · 2019-11-14T06:48:19Z

@ybsh Can you report how the performance is improveed by this PR? I will merge after the report.

ybsh · 2019-11-14T08:58:40Z

I ran again train_mnist.py on

CuPy
ClPy before the change ( cc72c54)
ClPy after the change (82ad22b)

The configuration is the same as shown in this report.

The execution times (in seconds) are as follows.

cupy	clpy-before	clpy-after
51.21	78.05	63.73

These figures look slower overall compared to the report and I have not found the definitive cause with a quick investigation. Still, this is enough to show the modified ClPy runs faster.
This issue records what kind of overhead has been removed and how it was found.

ybsh · 2019-11-14T08:59:19Z

@LWisteria Please check my new report.

LWisteria · 2019-11-14T09:02:03Z

@ybsh Thank you for your report! This PR makes the difference from CuPy less than a half. Nice work!

ybsh · 2019-11-14T09:06:41Z

Thank you for merging.
And thank you for your advice!
@LWisteria @nsakabe-fixstars @vorj @ykitta-fixstars

Yutaro Bessho added 2 commits November 6, 2019 19:27

Get pointers of scalar kernel arguments with a different interface

7290654

Eliminate isscype, use set operation instead

cf73ef1

ybsh requested a review from LWisteria November 7, 2019 07:02

ybsh assigned LWisteria Nov 7, 2019

jenkins-maekawa approved these changes Nov 7, 2019

View reviewed changes

ybsh mentioned this pull request Nov 7, 2019

Possible performance improvement with fewer NumPy core function calls #257

Closed

LWisteria requested changes Nov 8, 2019

View reviewed changes

clpy/backend/function.pyx Outdated Show resolved Hide resolved

LWisteria assigned ybsh and unassigned LWisteria Nov 8, 2019

ybsh changed the title ~~257 reduce numpy calls~~ Performance improvement in kernel launches by using lighter NumPy APIs Nov 13, 2019

Avoid double declaration

b3c272c

jenkins-maekawa suggested changes Nov 13, 2019

View reviewed changes

Yutaro Bessho added 2 commits November 13, 2019 16:55

Declare the scalar type string in core.pxd

be17122

Clean some commented out code

f217343

jenkins-maekawa approved these changes Nov 13, 2019

View reviewed changes

Append a blank line

82ad22b

ybsh assigned LWisteria and unassigned ybsh Nov 13, 2019

jenkins-maekawa approved these changes Nov 13, 2019

View reviewed changes

LWisteria approved these changes Nov 14, 2019

View reviewed changes

LWisteria assigned ybsh and unassigned LWisteria Nov 14, 2019

ybsh assigned LWisteria and unassigned ybsh Nov 14, 2019

LWisteria merged commit 3a565e0 into clpy Nov 14, 2019

LWisteria deleted the 257-reduce_numpy_calls branch November 14, 2019 09:00

ybsh mentioned this pull request Dec 6, 2019

Why ClPy is slower than CuPy even if on the same machine? #153

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Performance improvement in kernel launches by using lighter NumPy APIs #258

Performance improvement in kernel launches by using lighter NumPy APIs #258

ybsh commented Nov 7, 2019

jenkins-maekawa left a comment

jenkins-maekawa left a comment

ybsh commented Nov 7, 2019

ybsh commented Nov 8, 2019

LWisteria left a comment

jenkins-maekawa left a comment

jenkins-maekawa left a comment

jenkins-maekawa left a comment

jenkins-maekawa left a comment

ybsh commented Nov 13, 2019

jenkins-maekawa left a comment

jenkins-maekawa left a comment

LWisteria left a comment

LWisteria commented Nov 14, 2019

ybsh commented Nov 14, 2019 •

edited

Loading

ybsh commented Nov 14, 2019

LWisteria commented Nov 14, 2019 •

edited

Loading

ybsh commented Nov 14, 2019

Performance improvement in kernel launches by using lighter NumPy APIs #258

Performance improvement in kernel launches by using lighter NumPy APIs #258

Conversation

ybsh commented Nov 7, 2019

jenkins-maekawa left a comment

Choose a reason for hiding this comment

jenkins-maekawa left a comment

Choose a reason for hiding this comment

ybsh commented Nov 7, 2019

ybsh commented Nov 8, 2019

LWisteria left a comment

Choose a reason for hiding this comment

jenkins-maekawa left a comment

Choose a reason for hiding this comment

jenkins-maekawa left a comment

Choose a reason for hiding this comment

jenkins-maekawa left a comment

Choose a reason for hiding this comment

jenkins-maekawa left a comment

Choose a reason for hiding this comment

ybsh commented Nov 13, 2019

jenkins-maekawa left a comment

Choose a reason for hiding this comment

jenkins-maekawa left a comment

Choose a reason for hiding this comment

LWisteria left a comment

Choose a reason for hiding this comment

LWisteria commented Nov 14, 2019

ybsh commented Nov 14, 2019 • edited Loading

ybsh commented Nov 14, 2019

LWisteria commented Nov 14, 2019 • edited Loading

ybsh commented Nov 14, 2019

ybsh commented Nov 14, 2019 •

edited

Loading

LWisteria commented Nov 14, 2019 •

edited

Loading