DOC: Confusing advanced indexing when mixing tuple, integer, and range

[rocketcollider]

Issue with current documentation:

When addressing a 3+ dim array with a tuple in its 3rd or later dims, the returned shape depends on the order of previous ranges/integers:

>>> k = np.ones((4,5,6),)
>>> k[:,0,(1,2)].shape
(4, 2) #expected
>>> k[0,:,(1,2)].shape
(2, 5) #unexpected! actually transpose of right answer
>>> k[0,:,:][:,(1,2)].shape
(5, 2) #workaround
>>> k[:1,:,(1,2)].shape
(1, 5, 2) #workaround 

This might be an actual bug. It seems the actually unexpected case should be canonically correct: zero getting broadcast to (0,0). The documentation is unclear as to how this situation should behave, the example in that case uses unclear variable names:

result[i_1, ..., i_M] == x[ind_1[i_1, ..., i_M], ind_2[i_1, ..., i_M],
                       ..., ind_N[i_1, ..., i_M]]

from here:
https://numpy.org/doc/stable/user/basics.indexing.html#integer-array-indexing

Idea or request for content:

The priority between ranges and tuples seems to be undefined and I don't know what it should be. If the described behavior is expected, the documentation should describe behavior when mixing with ranges better.

[jakevdp]

Hi – agreed that this can be confusing, but it is working as expected. This situation is covered by the docs in this section: https://numpy.org/devdocs/user/basics.indexing.html#combining-advanced-and-basic-indexing

Two cases of index combination need to be distinguished:

1. The advanced indices are separated by a slice, Ellipsis or newaxis. For example x[arr1, :, arr2].

2. The advanced indices are all next to each other. For example x[..., arr1, arr2, :] but not x[arr1, :, 1] since 1 is an advanced index in this regard.

In the first case, the dimensions resulting from the advanced indexing operation come first in the result array, and the subspace dimensions after that. In the second case, the dimensions from the advanced indexing operations are inserted into the result array at the same spot as they were in the initial array

It goes on to show some specific examples of this.

The reason for moving the dimensions to the front is that in the general case the output shape is the result of broadcasting the shapes of the advanced indices, and the semantics of this broadcasting are ambiguous if the resulting dimensions are not made to be adjacent.

[rocketcollider]

I would still suggest highlighting potential pitfalls where integer-array-indexing is introduced instead of a different block. The documentation is quite dense (which is good), so I don't expect more information on a topic once there is a different headline.

The most minimal intervention I can think of that would improve the doc in this regard is changing the order of "Integer array indexing" and "Boolean array indexing". That way, the pitfalls are described immediately after the "Integer array indexing".

However, the quoted example:

result[i_1, ..., i_M] == x[ind_1[i_1, ..., i_M], ind_2[i_1, ..., i_M],
                   ..., ind_N[i_1, ..., i_M]]

is still very confusing. It's not clear to me what's going on here. At the very least it requires a line similar to

result = x[ind_1, ind_2, ..., ind_N]

but then ind_1 aso. would need to be 1-dimensional? I stay confused.

Additionally, I would change the counterfactual example in the second case, i.e.:

2. The advanced indices are all next to each other. For example x[..., arr1, arr2, :] but not x[1, :, arr1] since 1 is an advanced indexing array after broadcasting to arr1 in this regard.

As a general note: This problem was hard to identify, hard to search for, hard to understand and required an un-intuitive workaround. More discussion/warnings of this happening, more details on why it's happening and potential workarounds would be appreciated.

P.S.: I am actually curious about the reasons for this behavior and will read up on it now, but in the debugging-phase I have little attention to spare and don't yet know which source to read in detail.

[muhammed-soysl]

Hello, as a first-time contributor, I would like to try and improve this part of the documentation.