Backwards incompatibility in io/pytables with existing data prior to 0.8 causes data corruption

[jreback]

trying to append to a table that was created before 0.8 (e.g. before the kind of datetime64 existed) causes corrupted data (essentially the index is converted to datetime64, but since its in nanoseconds, existing datetimes are not evaluated correctly). easiest to convert existing data (read in and just write it out), so this patch is simply an exception which is raised if incompatble kinds are detected.

---
 pandas/io/pytables.py |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/pandas/io/pytables.py b/pandas/io/pytables.py
index 0f386c7..e3b84fb 100755
--- a/pandas/io/pytables.py
+++ b/pandas/io/pytables.py
@@ -825,6 +825,12 @@ class HDFStore(object):
             # the table must already exist
             table = getattr(group, 'table', None)

+            # check for backwards incompatibility
+            if append:
+                existing_kind = table._v_attrs.index_kind
+                if existing_kind != index_kind:
+                    raise Exception("incompatible kind in index [%s - %s]" % (existing_kind,index_kind))
+
         # add kinds
         table._v_attrs.index_kind = index_kind
         table._v_attrs.columns_kind = cols_kind
-- 
1.7.2.5

[wesm]

Thanks. Added this and a unit test