matplotlib savefig limited to size of the path

[hiyamgh]

I have been trying to save a figure using matplotlib. The size of the path I specify to plt.savefig was too long. Specifically:

# I am using the following
plt.savefig(os.path.join(output_folder, 'forecasts_scatter_%s.png' % model_name))

The output of os.path.join(output_folder, 'forecasts_scatter_%s.png' % model_name) is:

../output_ubr_shallow/all_columns_minus_weather_minus_lags_minus_civilians_balance/smogn_0.95_Manhattan_rel_mat1/grid_search/train_test_forecasts_scatterplot_bisector/forecasts_scatter_lasso.png

But I was getting the error:

Traceback (most recent call last):
  File "C:/Users/96171/Desktop/ministry_of_public_health/CodeUbrCorrected/bmwmlmcw.py", line 56, in <module>
    lm.cross_validation(model, hyperparameters[model_name], model_name)
  File "C:\Users\96171\Desktop\ministry_of_public_health\CodeUbrCorrected\cross_validation_smogn.py", line 554, in cross_validation
    self.cross_validation_grid(model_used, hyperparams, model_name)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn\utils\testing.py", line 348, in wrapper
    return fn(*args, **kwargs)
  File "C:\Users\96171\Desktop\ministry_of_public_health\CodeUbrCorrected\cross_validation_smogn.py", line 1444, in cross_validation_grid
    'predicted')
  File "C:\Users\96171\Desktop\ministry_of_public_health\CodeUbrCorrected\cross_validation_smogn.py", line 1772, in plot_actual_vs_predicted_scatter_bisector
    plt.savefig(output_folder + 'forecasts_scatter_%s' % model_name)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\pyplot.py", line 689, in savefig
    res = fig.savefig(*args, **kwargs)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\figure.py", line 2094, in savefig
    self.canvas.print_figure(fname, **kwargs)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\backend_bases.py", line 2075, in print_figure
    **kwargs)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\backends\backend_agg.py", line 521, in print_png
    cbook.open_file_cm(filename_or_obj, "wb") as fh:
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\cbook\__init__.py", line 407, in open_file_cm
    fh, opened = to_filehandle(path_or_file, mode, True, encoding)
  File "C:\Users\96171\AppData\Local\Programs\Python\Python36\lib\site-packages\matplotlib\cbook\__init__.py", line 392, in to_filehandle
    fh = open(fname, flag, encoding=encoding)
FileNotFoundError: [Errno 2] No such file or directory: '../output_ubr_shallow/all_columns_minus_weather_minus_lags_minus_civilians_balance/smogn_0.95_Manhattan_rel_mat1/grid_search/train_test_forecasts_scatterplot_bisector/forecasts_scatter_lasso.png'

I replaced this with:

# made the figure name shorter: 'scatter_%s.png' instead of 'forecasts_scatter_%s.png'
plt.savefig(os.path.join(output_folder, 'scatter_%s.png' % model_name))

and it worked.

[QuLogic]

Does open('../output_ubr_shallow/all_columns_minus_weather_minus_lags_minus_civilians_balance/smogn_0.95_Manhattan_rel_mat1/grid_search/train_test_forecasts_scatterplot_bisector/forecasts_scatter_lasso.png', 'wb') work in Python console?

If not, then this is a Python/Windows issue and I'm not sure there's much we can do about it.

[skypaver]

That's might be the reason.
These characters cannot be included in files' name.

[dstansby]

I'm going to close this since it doesn't look like it is a Matplotlib issue - if you can reply to @QuLogic's comment above, or provide more info feel free to re-open this issue or open a new one!

[apohali]

I have a similar problem. The length of the name of file or the parent directory needs to be short. Otherwise I get FileNotFoundError: [Errno 2] No such file or directory. I have tried saving in a different directory using the same filename and it worked fine. So the problem is not use of characters that cannot be used in filename.

[jklymak]

@apohali Please open a new issue with steps needed to reproduce the problem. However please check that a normal python file-open works. If not, we can't help.

[qzdeng]

I got the same issue. After testing, the problem seems to be that the overall character amount in fname must be < 256

[rcomer]

@qzdeng did you also try a normal file-open similar to #17313 (comment)?

[qzdeng]

@qzdeng did you also try a normal file-open similar to #17313 (comment)?

True, it is a Windows caused problem