Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scipy.io.savemat silently discards bad dictionary keys #10200

Open
orena1 opened this issue May 21, 2019 · 3 comments
Open

scipy.io.savemat silently discards bad dictionary keys #10200

orena1 opened this issue May 21, 2019 · 3 comments
Labels
defect A clear bug or issue that prevents SciPy from being installed or used as expected scipy.io

Comments

@orena1
Copy link
Contributor

orena1 commented May 21, 2019

I have a large (not complicated) dictionary that when I try to save it with sio.savemat I get 400 B file

Reproducing code example:

test.zip

import pickle
import scipy.io as sio
import os
data = pickle.load(open("test.p","r"))
sio.savemat('data',data)
os.path.getsize('data.mat')

Scipy/Numpy/Python version information:

('1.1.0', '1.14.2', sys.version_info(major=2, minor=7, micro=15, releaselevel='final', serial=0))
@matthew-brett
Copy link
Contributor

To give more detail - the test pickle has a dictionary with values that are also dictionaries. These dictionaries have integers as keys. The scipy.io routines discard these, silently, because Matlab structs don't allow integers as fields:

>> struct(1, 'one')
Error using struct
Field names must be non-empty character vectors.

As a result, the thing that gets saved is:

In [23]: d2 = sio.loadmat('data.mat')

In [24]: d2
Out[24]: 
{'__globals__': [],
 '__header__': 'MATLAB 5.0 MAT-file Platform: posix, Created on: Wed May 22 10:57:44 2019',
 '__version__': '1.0',
 'v0': array([[None]], dtype=object),
 'v1': array([[None]], dtype=object),
 'v_post0': array([[None]], dtype=object),
 'v_post1': array([[None]], dtype=object)}

@pv pv added defect A clear bug or issue that prevents SciPy from being installed or used as expected scipy.io labels Aug 4, 2019
@pv pv changed the title scipy.io.savemat save 400 B file scipy.io.savemat silently discards bad dictionary keys Aug 4, 2019
@erheron
Copy link
Contributor

erheron commented Mar 15, 2020

@matthew-brett any ideas for possible fixes here?
I'd be interested in tackling this, but no thoughts for now :|

@matthew-brett
Copy link
Contributor

Nothing immediately springs to mind. Is there any container in Matlab that can deal with integer keys? I see that containers.Map will allow keys that aren't valid variable names, but I don't think it will allow integers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
defect A clear bug or issue that prevents SciPy from being installed or used as expected scipy.io
Projects
None yet
Development

No branches or pull requests

4 participants