Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Support list of slices with Array.__getitem__ #125

Closed
bburan opened this Issue · 1 comment

2 participants

bburan Anthony Scopatz
bburan

Numpy supports both lists and tuples of slice objects::

x = np.random.random((100, 100))

In [34]: x[[slice(None, None, 20), slice(0, 10)]]
Out[34]: 
array([[ 0.0635731 ,  0.72360578,  0.28578261,  0.34618145,  0.70897503,
         0.20977744,  0.23431481,  0.1390509 ,  0.11412244,  0.43835589],
       [ 0.69659108,  0.16387496,  0.12865324,  0.76225587,  0.96499252,
         0.22729787,  0.15729999,  0.5211293 ,  0.47869863,  0.387693  ],
       [ 0.59313861,  0.62169338,  0.45813824,  0.94456447,  0.10127425,
         0.20181705,  0.44692005,  0.85943362,  0.89373644,  0.42671621],
       [ 0.23176836,  0.28132046,  0.37657883,  0.44450819,  0.02216597,
         0.20233609,  0.74471432,  0.4777428 ,  0.85918876,  0.26790031],
       [ 0.50411654,  0.35891554,  0.79704542,  0.5915836 ,  0.91206998,
         0.66992248,  0.1642178 ,  0.66129781,  0.45990812,  0.93659354]])

In [36]: x[(slice(None, None, 20), slice(0, 10))]
Out[36]: 
array([[ 0.0635731 ,  0.72360578,  0.28578261,  0.34618145,  0.70897503,
         0.20977744,  0.23431481,  0.1390509 ,  0.11412244,  0.43835589],
       [ 0.69659108,  0.16387496,  0.12865324,  0.76225587,  0.96499252,
         0.22729787,  0.15729999,  0.5211293 ,  0.47869863,  0.387693  ],
       [ 0.59313861,  0.62169338,  0.45813824,  0.94456447,  0.10127425,
         0.20181705,  0.44692005,  0.85943362,  0.89373644,  0.42671621],
       [ 0.23176836,  0.28132046,  0.37657883,  0.44450819,  0.02216597,
         0.20233609,  0.74471432,  0.4777428 ,  0.85918876,  0.26790031],
       [ 0.50411654,  0.35891554,  0.79704542,  0.5915836 ,  0.91206998,
         0.66992248,  0.1642178 ,  0.66129781,  0.45990812,  0.93659354]])

However, PyTables Arrays support only tuples of slice objects. It would be nice if we could use either interchangeably since Numpy also supports it::

In [29]: fh = tables.openFile('test.hd5', 'w')

In [30]: fh.createArray('/', 'random', np.random.random((100, 100)))
Out[30]: 
/random (Array(100, 100)) ''
  atom := Float64Atom(shape=(), dflt=0.0)
  maindim := 0
  flavor := 'numpy'
  byteorder := 'little'
  chunkshape := None

In [31]: fh.root.random[(slice(None, None, 20), slice(0, 10))]
Out[31]: 
array([[  8.40506729e-01,   2.73939136e-01,   3.27099832e-01,
          7.65041609e-01,   3.18342119e-01,   3.39324537e-01,
          6.11720827e-01,   4.57440635e-01,   6.43570572e-01,
          7.33830939e-01],
       [  3.84772759e-01,   7.62781489e-01,   8.07766732e-01,
          8.15595532e-02,   8.10896890e-01,   5.04088648e-01,
          2.52887856e-01,   7.22761598e-02,   5.58040611e-04,
          9.87909639e-01],
       [  1.45608962e-01,   7.96068841e-01,   4.88318137e-02,
          1.66848079e-01,   5.87356693e-03,   6.71066113e-01,
          3.71198992e-01,   4.15988693e-01,   9.31524133e-01,
          9.56320735e-01],
       [  7.68039998e-01,   4.82000435e-01,   6.70072453e-03,
          8.13839642e-01,   1.90253452e-01,   4.79538070e-01,
          6.40673242e-01,   3.35719325e-01,   3.50606201e-01,
          8.41937685e-01],
       [  1.66944794e-01,   6.26779736e-01,   2.74711630e-01,
          6.75971810e-02,   7.43803477e-01,   5.23274023e-01,
          3.68279743e-01,   1.62176886e-01,   5.65540083e-01,
          8.24560468e-01]])

In [32]: fh.root.random[[slice(None, None, 20), slice(0, 10)]]
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
c:\users\brad\desktop\<ipython-input-32-15b9896b6b44> in <module>()
----> 1 fh.root.random[[slice(None, None, 20), slice(0, 10)]]

C:\Python27\lib\site-packages\tables\array.pyc in __getitem__(self, key)
    690             except TypeError:
    691                 # Finally, try with a fancy selection

--> 692                 selection, reorder, shape = self._fancySelection(key)
    693                 arr = self._readSelection(selection, reorder, shape)
    694 

C:\Python27\lib\site-packages\tables\array.pyc in _fancySelection(self, args)
    613                     else:
    614                         list_seen = True
--> 615                 nexp = numpy.asarray(exp, dtype="i8")
    616                 # Convert negative values

    617                 nexp = numpy.where(nexp < 0, length+nexp, nexp)

C:\Python27\lib\site-packages\numpy\core\numeric.pyc in asarray(a, dtype, order)
    233 
    234     """
--> 235     return array(a, dtype, copy=False, order=order)
    236 
    237 def asanyarray(a, dtype=None, order=None):

TypeError: long() argument must be a string or a number, not 'slice'
Anthony Scopatz
Owner

Numpy slicing with tuples and lists is not, in fact, the same...though for many cases they are equivalent. The prime example of this is for accessing multiple fields from a structured array: http://docs.scipy.org/doc/numpy/user/basics.rec.html#accessing-multiple-fields-at-once This is more similar to the slicing out of tables.Table objects.

In theory I would be ok with being able to pass in any iterable of slices to tables.Array objects, but I think that Tables themselves need some more thought. Also there may be other usecases I am missing. (For instance indexing by boolean arrays also has a special meaning in numpy, which I don't think is duplicated in PyTables.)

If you have a more comprehensive set of features, and possibly a pull request, we would look into this more....

Anthony Scopatz scopatz closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.