Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

fixed row/col orientation for 2D arrays #1834

Merged
merged 2 commits into from

2 participants

@John-Colvin

Pytables uses the opposite convention to pandas for row/col. Using the numpy transpose .T corrects for this without copying the data and keeping the orientation of 1D arrays intact.

@John-Colvin John-Colvin fixed row/col orientation for 2D arrays
Pytables createArray function uses the opposite convention to pandas
for row/col. Using the numpy transpose .T corrects for this without
copying the data and keeping the orientation of 1D arrays intact.
80b5082
@John-Colvin

Is solution for #1824

@John-Colvin

my apologies, it appears I was a little premature with this pull request as the files it writes can't be read again. Should I close this request or edit it in-place as/when the problem is resolved?

@John-Colvin John-Colvin added tranposed attribute, used in _read_array
the transposed attribute keeps track of whether the data has been 
transposed, allowing it to be correctly read by _read_array.
Some minor rearrangement of _read_array was necessary to reduce
duplicate code.
4ff42db
@John-Colvin

earlier problems are fixed by adding a new _v_attr signalling the transpose has been applied. It's not that pretty, but it's logical and appears to work.

@wesm
Owner

Thanks. I think the _v_attr hack also will make it so legacy files can be read

@John-Colvin

Yes it should do, because it defaults to false if it's not found.

@wesm wesm referenced this pull request from a commit
@wesm wesm BUG: fixes for test suite re: #1834 8d110f2
@wesm wesm merged commit 4ff42db into from
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Sep 3, 2012
  1. @John-Colvin

    fixed row/col orientation for 2D arrays

    John-Colvin authored
    Pytables createArray function uses the opposite convention to pandas
    for row/col. Using the numpy transpose .T corrects for this without
    copying the data and keeping the orientation of 1D arrays intact.
  2. @John-Colvin

    added tranposed attribute, used in _read_array

    John-Colvin authored
    the transposed attribute keeps track of whether the data has been 
    transposed, allowing it to be correctly read by _read_array.
    Some minor rearrangement of _read_array was necessary to reduce
    duplicate code.
This page is out of date. Refresh to see the latest.
Showing with 20 additions and 8 deletions.
  1. +20 −8 pandas/io/pytables.py
View
28 pandas/io/pytables.py
@@ -686,7 +686,10 @@ def _read_index_node(self, node):
def _write_array(self, group, key, value):
if key in group:
self.handle.removeNode(group, key)
-
+
+ #Transform needed to interface with pytables row/col notation
+ value = value.T
+
if self.filters is not None:
atom = None
try:
@@ -701,6 +704,7 @@ def _write_array(self, group, key, value):
value.shape,
filters=self.filters)
ca[:] = value
+ getattr(group, key)._v_attrs.transposed = True
return
if value.dtype.type == np.object_:
@@ -719,6 +723,8 @@ def _write_array(self, group, key, value):
getattr(group, key)._v_attrs.shape = value.shape
else:
self.handle.createArray(group, key, value)
+
+ getattr(group, key)._v_attrs.transposed = True
def _write_table(self, group, items=None, index=None, columns=None,
values=None, append=False, compression=None):
@@ -958,23 +964,29 @@ def _read_array(group, key):
import tables
node = getattr(group, key)
data = node[:]
+ attrs = node._v_attrs
+
+ transposed = getattr(attrs, 'transposed', False)
if isinstance(node, tables.VLArray):
- return data[0]
+ ret = data[0]
else:
- attrs = node._v_attrs
-
dtype = getattr(attrs, 'value_type', None)
shape = getattr(attrs, 'shape', None)
if shape is not None:
# length 0 axis
- return np.empty(shape, dtype=dtype)
+ ret = np.empty(shape, dtype=dtype)
if dtype == 'datetime64':
- return np.array(data, dtype='M8[ns]')
- return data
-
+ ret = np.array(data, dtype='M8[ns]')
+ ret = data
+
+ if transposed == True:
+ return ret.T
+ else:
+ return ret
+
def _unconvert_index(data, kind):
if kind == 'datetime64':
index = DatetimeIndex(data)
Something went wrong with that request. Please try again.