ND4J: Add convenience method for where_np op

[AlexDBlack]

We have it, but users have to interact with the op directly, allocate output shapes
https://github.com/deeplearning4j/deeplearning4j/blob/451dd76b50355358dc176f2b704e98c43423c5b8/libnd4j/include/ops/declarable/generic/boolean/where_np.cpp
https://github.com/deeplearning4j/deeplearning4j/blob/451dd76b50355358dc176f2b704e98c43423c5b8/nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/linalg/api/ops/impl/controlflow/WhereNumpy.java

So, something like Nd4j.where(...)

For anyone reading this - wokraround is using this, plus Nd4j.getExecutioner().calculateOutputShapes(op):
https://github.com/deeplearning4j/dl4j-examples/blob/049b072a8a082012e5f813269884e8ed524a484d/nd4j-examples/src/main/java/org/nd4j/examples/CustomOpsExamples.java

[tzolov]

The second snippets points to a Nd4j.getExecutioner().exec(op) not Nd4j.getExecutioner().calculateOutputShapes(op) example?

From the WhereNumpy test code it is not clear how to return the element indexes (which is the np.where behavior) and not the values? Do i need to preinitialize the put array with the indexes?
Are there some more elaborate WhereNumpy examples?

[tzolov]

For example, is it possible to use WhereNumpy for implementing the following np.where(...) like logic:

private INDArray getIndexWhere(INDArray input, Predicate<Double> predicate) {
   // expects only vector or scalar !
   List<Integer> indexes = new ArrayList<>();
   for (int i = 0; i < input.size(0); i++) {
      if (predicate.test(input.getDouble(i))) 
         indexes.add(i);
   }

   return CollectionUtils.isEmpty(indexes) ?
      Nd4j.empty(DataBuffer.Type.FLOAT) : return Nd4j.create(indexes);
}

[AlexDBlack]

OK, so looking at this again:

One usage of where_np op takes 3 input variables, condition, x, y
and 1 output variable
Condition is a 0 or 1 valued array of the same shape as x/y

For the single input variable case (i.e., only "condition" array specified): it seems we return a 2d array of coordinates instead of a tuple of 1d coordinates

deeplearning4j/libnd4j/include/ops/declarable/generic/boolean/where_np.cpp

Lines 86 to 113 in 451dd76

	// in this case we return 2D matrix, which basically contains coordinates fo true
	REQUIRE_TRUE(block.width() == 1, 0, "Where op takes either 1 or 3 operands, But got %d operands instead", block.width());
	int width = condition->rankOf();
	std::vector<int> dims = ShapeUtils<T>::convertAxisToTadTarget(width, {0});
	NDArrayList<T> list(0, true);
	int cnt = 0;
	Nd4jLong idx[MAX_RANK];
	for (int e = 0; e < condition->lengthOf(); e++) {
	shape::ind2subC(condition->rankOf(), condition->shapeOf(), e, idx);
	auto offset = shape::getOffset(0, condition->shapeOf(), condition->stridesOf(), idx, condition->rankOf());
	T v = condition->buffer()[offset];
	if (v != (T) 0.0f) {
	auto array = new NDArray<T>('c', {1, condition->rankOf()});
	for (int f = 0; f < condition->rankOf(); f++)
	array->putIndexedScalar(f, (T) idx[f]);
	list.write(cnt++, array);
	}
	}
	auto result = list.stack();
	OVERWRITE_RESULT(result);

So, to answer your question: essentially, yes, but (same as numpy) input is an array post applicaiton of the predicate, not predicate as an argument.

That said, it seems the where_np shape function is wrong:

deeplearning4j/libnd4j/include/ops/declarable/generic/boolean/where_np.cpp

Lines 121 to 145 in 451dd76

	DECLARE_SHAPE_FN(where_np) {
	if (block.width() == 3) {
	auto inShape = inputShape->at(1);
	Nd4jLong *newshape;
	COPY_SHAPE(inShape, newshape);
	return SHAPELIST(newshape);
	} else {
	// FIXME: we can't estimate result here in this case
	auto inShape = inputShape->at(0);
	Nd4jLong *newshape;
	ALLOCATE(newshape, block.getWorkspace(), shape::shapeInfoLength(2), Nd4jLong);
	newshape[0] = 2;
	newshape[1] = 10;
	newshape[2] = 10;
	newshape[3] = 1;
	newshape[4] = 1;
	newshape[5] = 0;
	newshape[6] = 1;
	newshape[7] = 99;
	return SHAPELIST(newshape);
	}

A workaround for now is simply calculating the number of matches yourself - for example:

int count = Nd4j.getExecutioner().exec(new MatchCondition(cg.params(), Conditions.equals(0))).z().getInt(0);

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.