GPU PythonFunction operator #1655
Conversation
Signed-off-by: Rafal <Banas.Rafal97@gmail.com>
|
!build |
|
CI MESSAGE: [1071819]: BUILD STARTED |
|
CI MESSAGE: [1071819]: BUILD PASSED |
| @@ -27,6 +27,13 @@ | |||
| import nvidia.dali.libpython_function_plugin | |||
|
|
|||
|
|
|||
| cupy = None | |||
There was a problem hiding this comment.
This is not needed. Bellow global is creating cupy variable.
There was a problem hiding this comment.
When i remove this line, it ends up with:
File "/home/rafal/mydali/build/venv/lib/python3.6/site-packages/nvidia/dali/ops.py", line 33, in _setup_cupy
if cupy is None:
NameError: name 'cupy' is not defined
Signed-off-by: Rafal <Banas.Rafal97@gmail.com>
banasraf
changed the title
Signed-off-by: Rafal <Banas.Rafal97@gmail.com>
|
!build |
|
CI MESSAGE: [1073791]: BUILD STARTED |
|
CI MESSAGE: [1073791]: BUILD PASSED |
| .DocStr("Executes a python function. \n" | ||
| "The operator can be used to execute custom python code within the DALI pipeline. " | ||
| "The called function will get tensors' data as numpy arrays for CPU operators" | ||
| " or as cupy arrays for GPU operators and should return results in the same format." |
There was a problem hiding this comment.
| " or as cupy arrays for GPU operators and should return results in the same format." | |
| " or as CuPy arrays for GPU operators and should return results in the same format." |
maybe?
There was a problem hiding this comment.
Will fix in the next PR (GPU Torch operator)
| class PythonFunctionPipeline(Pipeline): | ||
| def __init__(self, function, device, num_outputs=1): | ||
| super(PythonFunctionPipeline, self).__init__(BATCH_SIZE, NUM_WORKERS, DEVICE_ID, | ||
| seed=SEED, |
There was a problem hiding this comment.
Note: You don't need to set the seed if you just want it to be a random number
| @@ -5,6 +5,7 @@ target_dir=./dali/test/python | |||
|
|
|||
| test_body() { | |||
| nosetests --verbose -m '(?:^|[\b_\./-])[Tt]est.*cupy' test_dltensor_operator.py | |||
| nosetests --verbose test_gpu_python_function_operator.py | |||
There was a problem hiding this comment.
Note: If you name your file as test_operator_*.py it is picked automatically by oython self test (not sure if that is what you want)
There was a problem hiding this comment.
I don't want to, because i cannot import cupy in python self test
Why we need this PR?
Pick one, remove the rest
What happened in this PR?
Fill relevant points, put NA otherwise. Replace anything inside []
It was implemented as wrapper around DLTensorPythonFunctionOperator
Operator added in ops.py
ops.py
New tests added
Updated the operator docs
JIRA TASK: [DALI-1189]