Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support numpy bridge (enabled by default in dygraph mode) #20983

Merged
merged 15 commits into from
Nov 27, 2019

Conversation

songyouwei
Copy link
Contributor

@songyouwei songyouwei commented Nov 4, 2019

NumPy Bridge is used for zero-copy creation of a Tensor with a NumPy array.
zero_copy is enabled by default when place is a CPUPlace and is in dygraph mode.

The created Tensor and NumPy array will share their underlying memory locations (only when working with CPUPlace), and change the numpy array will also change the corresponding Tensor.

Example:

import numpy as np
import paddle.fluid as fluid

data_np = np.array([[2, 3, 1]]).astype('float32')
with fluid.dygraph.guard(fluid.CPUPlace()):
    var = fluid.dygraph.to_variable(data_np, zero_copy=True)
    data_np[0][0] = 4
    var[0][0].numpy()[0] == 4  # True

@songyouwei songyouwei changed the title Support numpy bridge dygraph Support numpy bridge (zero_copy) Nov 4, 2019
@songyouwei songyouwei changed the title Support numpy bridge (zero_copy) Support numpy bridge (zero_copy=True) Nov 4, 2019
@songyouwei songyouwei changed the title Support numpy bridge (zero_copy=True) Support numpy bridge (enabled by default in dygraph mode) Nov 5, 2019
@songyouwei songyouwei closed this Nov 18, 2019
@songyouwei songyouwei reopened this Nov 18, 2019
@phlrain phlrain self-requested a review November 22, 2019 09:03
phlrain
phlrain previously approved these changes Nov 22, 2019
heavengate
heavengate previously approved these changes Nov 22, 2019
wanghaoshuang
wanghaoshuang previously approved these changes Nov 22, 2019
paddle/fluid/framework/tensor.h Outdated Show resolved Hide resolved
paddle/fluid/pybind/pybind.cc Show resolved Hide resolved
paddle/fluid/pybind/tensor_py.h Outdated Show resolved Hide resolved
paddle/fluid/pybind/tensor_py.h Outdated Show resolved Hide resolved
paddle/fluid/pybind/tensor_py.h Show resolved Hide resolved
paddle/fluid/pybind/tensor_py.h Outdated Show resolved Hide resolved
paddle/fluid/pybind/tensor_py.h Outdated Show resolved Hide resolved
paddle/fluid/pybind/tensor_py.h Outdated Show resolved Hide resolved
python/paddle/fluid/framework.py Outdated Show resolved Hide resolved
python/paddle/fluid/dygraph/base.py Show resolved Hide resolved
Copy link
Collaborator

@sneaxiy sneaxiy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Copy link

@saxon-zh saxon-zh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@phlrain phlrain merged commit d5ff79e into PaddlePaddle:develop Nov 27, 2019
@songyouwei songyouwei deleted the support-numpy-bridge-dygraph branch November 27, 2019 08:26
seiriosPlus pushed a commit to seiriosPlus/Paddle that referenced this pull request Dec 9, 2019
…le#20983)

* add numpy bridge

* fix template compile

* add unittest, add default
test=develop

* fix unittest
test=develop

* fix unittest
test=develop

* zero_copy=True for to_variable,
test=develop

* bug fix
test=develop

* disable deprecated NumPy API
test=develop

* use better design of NumpyAllocator
test=develop

* fix Py_None check
test=develop

* reset c++ tracer when jump out dygraph guard
test=develop

* refine PADDLE_ENFORCE_xx format
test=develop

* bug fix of tracer switch
test=develop

* update decref
test=develop
seiriosPlus pushed a commit to seiriosPlus/Paddle that referenced this pull request Dec 9, 2019
…le#20983)

* add numpy bridge

* fix template compile

* add unittest, add default
test=develop

* fix unittest
test=develop

* fix unittest
test=develop

* zero_copy=True for to_variable,
test=develop

* bug fix
test=develop

* disable deprecated NumPy API
test=develop

* use better design of NumpyAllocator
test=develop

* fix Py_None check
test=develop

* reset c++ tracer when jump out dygraph guard
test=develop

* refine PADDLE_ENFORCE_xx format
test=develop

* bug fix of tracer switch
test=develop

* update decref
test=develop
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants