Add tests for operator cast. Revert to plain batched cast kernel until the optimized one is fixed. #3927
Conversation
|
CI MESSAGE: [4912461]: BUILD STARTED |
|
CI MESSAGE: [4912461]: BUILD FAILED |
| print("At sample", i) | ||
| I = np.array(inp[i]) | ||
| O = np.array(out[i]) | ||
| R = ref[i] | ||
| print(I) | ||
| print(R) | ||
| print(O) | ||
| mask = np.logical_not(np.isclose(O, R, eps)) | ||
| print("Differences at", mask) | ||
| print(I[mask]) | ||
| print(R[mask]) | ||
| print(O[mask]) | ||
| print(np.count_nonzero(mask), "wrong values out of", mask.size) | ||
| assert np.array_equal(out[i], ref[i]) |
There was a problem hiding this comment.
Whis would create some verbose printing in tests (every print is a new line). If we really need these prints, how about using print(f"...")?
There was a problem hiding this comment.
This is printed only if a failure occurs.
There was a problem hiding this comment.
I see. So still, how about making the code prettier by print(f"...")?
There was a problem hiding this comment.
It doesn't make much difference as long as we actually want arguments to be separated by spaces...
| else: | ||
| return x.astype(dtype) | ||
|
|
||
| def random_shape(rng, ndim:int, max_size:int): |
There was a problem hiding this comment.
I believe standard formatting of type hints is like:
def foo(myarg: int)|
|
||
| @nottest | ||
| def _test_operator_cast(ndim, batch_size, in_dtype, out_dtype, device): | ||
| src = lambda: generate(rng, ndim, batch_size, in_dtype, out_dtype) |
There was a problem hiding this comment.
Why do we need this? Shouldn't the following be sufficient?
inp = fn.external_source(generate(...))This lambda assignment looks suspicious
There was a problem hiding this comment.
I need to pass a function, not the results.
There was a problem hiding this comment.
How about making it more pythonic, by making a generate actually a generator and passing a generator object to external_source?
def generate(...):
yield out
fn.external_source(generate(...))There was a problem hiding this comment.
This is counterproductive - requires more work from me (writing a generator, specifying how to handle cycling) and from the operator (it will have to handle do a lot of inspection and will ultimately end up wrapping that generator in a yet another lambda).
There was a problem hiding this comment.
BTW, can you have an infinite generator? Because that's what I want here. Wrapping it in a generator that returns one item and cycling it doesn't seem any more pythonic than a lambda.
I guess one thing that would work instead of a lambda, is partial.
There was a problem hiding this comment.
can you have an infinite generator?
Do you mean something like that?
def gen():
while True:
yield batch
As to the pythonness I think the only objection is not generator over lambda but giving a name to a lambda.
https://peps.python.org/pep-0008/#programming-recommendations
Always use a def statement instead of an assignment statement that binds a lambda expression directly to an identifier:
# Correct:
def f(x): return 2*x
# Wrong:
f = lambda x: 2*x
| else: | ||
| x |
There was a problem hiding this comment.
Does this else clause do anything? If not, is it needed at all?
…e BinSearch is broken). Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
Signed-off-by: Michał Zientkiewicz <mzient@gmail.com>
…e there. Signed-off-by: Michał Zientkiewicz <mzient@gmail.com>
|
CI MESSAGE: [4919803]: BUILD STARTED |
|
CI MESSAGE: [4919803]: BUILD PASSED |
…l the optimized one is fixed. (#3927) * Add tests for operator cast. Revert to plain batched cast kernel (the BinSearch is broken). Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
…l the optimized one is fixed. (NVIDIA#3927) * Add tests for operator cast. Revert to plain batched cast kernel (the BinSearch is broken). Signed-off-by: Michal Zientkiewicz <michalz@nvidia.com>
Signed-off-by: Michal Zientkiewicz michalz@nvidia.com
Category:
Add tests for operator
castBug-fix (workaround) - use old cast kernel.
Description:
Additional information:
Affected modules and functionalities:
Key points relevant for the review:
Checklist
Tests
Documentation
DALI team only
Requirements
REQ IDs: N/A
JIRA TASK: N/A