scipy.io.loadmat failure in 1.7.0

[jcharlong]

My issue is about a failure to load MATLAB files with scipy 1.7.0. I use scipy.io.loadmat (with squeeze_me=True) to load MATLAB files & it works just fine in 1.6.3.

I traced the issue down to a change in scipy.io.matlab.mio5.MatFile5Reader's get_variables method. I noticed when diffing the 1.7.0 branch with the 1.6.3 branch that line 313 was changed from asstr(hdr.name) (from numpy.compat import asstr) to hdr.name.decode('latin1'). Looks like this change was introduced in the following commit.

It appears as though asstr handles the case when hdr.name is None, which occurs with some of the MATLAB files I am working with.

Reverting to using asstr resolves the issue & so does changing the line to hdr.name.decode('latin1') if hdr.name is not None else 'None' (asstr(None) == 'None')). Not sure which would be more appropriate, or if hdr.name even should ever be None.

Error message:

Error
Traceback (most recent call last):
(redacted a few unrelated traceback lines)
  File "/usr/lib/python3.8/site-packages/scipy/io/matlab/mio.py", line 226, in loadmat
    matfile_dict = MR.get_variables(variable_names)
  File "/usr/lib/python3.8/site-packages/scipy/io/matlab/mio5.py", line 313, in get_variables
    name = hdr.name.decode('latin1')
AttributeError: 'NoneType' object has no attribute 'decode'

Scipy/Numpy/Python version information:

1.7.0 1.20.3 sys.version_info(major=3, minor=8, micro=5, releaselevel='final', serial=0)

[ev-br]

Could you provide a self-contained example? Both python code and a (minimal) matlab file, which together reproduce the problem?

[jcharlong]

I can reproduce locally by simply running

import scipy.io

data = scipy.io.loadmat('matlab.mat')

Where matlab.mat is the problem file.

I tried but can't seem to generate a new MATLAB file that can produce this effect. Even simply loading & re-saving a problem file (via MATLAB) appears to resolve the issue. Not sure what the original MATLAB version/environment/settings were when the file was created.

The issue is cropping up at work where we use some MATLAB files as inputs to some unit tests. Sadly because of this I cannot supply the original files as they contain proprietary data.

[ev-br]

I think adding a if hdr.name is not None else 'None' guard would be OK if it fixes a backcompat corner case issue even if there is no unit test we could add.

Would you be willing to send a PR @jcharlong?

Am tentatively marking it for a 1.7.1 milestone -- something was bound to start it, I suppose.

[jcharlong]

Yes, I can take a stab at submitting a PR. Likely won't be able to get to it until this upcoming weekend at the earliest though.