plot_weather.py fails sometimes

[jamessynge]

The latest weather plot on PAN006 is from yesterday morning, with more recent attempts failing with errors like this:

Creating weather plotter for 20180701UT
  Determining sunrise, sunset, and twilight times
  Retrieving data from Mongo database
  Retrieved 1040 entries between 2018-06-30T20:05:56.790000 and 2018-07-01T20:04:35.895000
  Setting up plot for time range: 2018-06-30T20:05:06.235139 to 2018-07-01T20:05:06.235139
  Will generate last hour plot for time range: 2018-07-01T19:05:06.235139 to 2018-07-01T20:10:06.235139
Plot Ambient Temperature vs. Time
Plot Temperature Difference vs. Time
Plot Wind Speed vs. Time
Plot Rain Frequency vs. Time
Plot Safe/Unsafe vs. Time
Plot PWM Value vs. Time
Saving Figure: /var/panoptes/weather_plots/today.png
Traceback (most recent call last):
  File "/var/panoptes/POCS/scripts/plot_weather.py", line 743, in <module>
    wp.make_plot(args.plot_file)
  File "/var/panoptes/POCS/scripts/plot_weather.py", line 117, in make_plot
    self.save_plot(plot_filename=output_file)
  File "/var/panoptes/POCS/scripts/plot_weather.py", line 704, in save_plot
    self.fig.savefig(plot_filename, dpi=self.dpi, bbox_inches='tight', pad_inches=0.10)
  File "/home/panoptes/anaconda3/envs/panoptes-env/lib/python3.6/site-packages/matplotlib/figure.py", line 1834, in savefig
    self.canvas.print_figure(fname, **kwargs)
  File "/home/panoptes/anaconda3/envs/panoptes-env/lib/python3.6/site-packages/matplotlib/backend_bases.py", line 2267, in print_figure
    **kwargs)
  File "/home/panoptes/anaconda3/envs/panoptes-env/lib/python3.6/site-packages/matplotlib/backends/backend_agg.py", line 507, in print_png
    FigureCanvasAgg.draw(self)
  File "/home/panoptes/anaconda3/envs/panoptes-env/lib/python3.6/site-packages/matplotlib/backends/backend_agg.py", line 422, in draw
    self.renderer = self.get_renderer(cleared=True)
  File "/home/panoptes/anaconda3/envs/panoptes-env/lib/python3.6/site-packages/matplotlib/backends/backend_agg.py", line 444, in get_renderer
    self.renderer = RendererAgg(w, h, self.figure.dpi)
  File "/home/panoptes/anaconda3/envs/panoptes-env/lib/python3.6/site-packages/matplotlib/backends/backend_agg.py", line 92, in __init__
    self._renderer = _RendererAgg(int(width), int(height), dpi, debug=False)
ValueError: Image size of 737170x915 pixels is too large. It must be less than 2^16 in each direction.

@wtgee Could you make suggestions as to how to proceed? matplotlib is not my top skill, but I'm guessing 700K pixels wide is a real no-no.

[wtgee]

I've seen that error in the past when trying to remove the bbox_inches='tight' in favor of a self.fig.tight_layout(). But nothing has changed here recently so I don't think it would have to do with that.

I'm wondering if the range of values is somehow wrong. The date looks okay for one day. PAN001 consistently has 1041 or 1040 entries in that range.

Has matplotlib or numpy been updated recently? (Or I guess since this has been working)

[jamessynge]

I've not updated matplotlib or numpy recently. I'm inclined to think there is a bug in our code rather than in the deps. I'm guessing that some extra logging is in order.

…

On Sun, Jul 1, 2018 at 6:47 PM Wilfred Tyler Gee ***@***.***> wrote: I've seen that error in the past when trying to remove the bbox_inches='tight' in favor of a self.fig.tight_layout(). But nothing has changed here recently so I don't think it would have to do with that. I'm wondering if the range of values is somehow wrong. The date looks okay for one day. PAN001 consistently has 1041 or 1040 entries in that range. Has matplotlib or numpy been updated recently? (Or I guess since this has been working) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#528 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABYmU9i_Y4sOLBru5FTfW1_9Qn34swErks5uCVFqgaJpZM4U-dCX> .

[wtgee]

Could easily be in our code but it's run without problems for a long time on PAN001. Has it ever worked reliably on your system?

[jamessynge]

No, I don't think it has ever been reliable on PAN006.

…

On Sun, Jul 1, 2018 at 8:35 PM Wilfred Tyler Gee ***@***.***> wrote: Could easily be in our code but it's run without problems for a long time on PAN001. Has it ever worked reliably on your system? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#528 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABYmU_-C2TBhsyolO4Si4c9B-imeBDMjks5uCWrqgaJpZM4U-dCX> .

[wtgee]

You can try just removing the bbox_inches='tight' line (and/or the dpi) to see if that will at least make it work.

[wtgee]

This was fixed via a recommendation here: matplotlib/matplotlib#883.

Namely, @jamessynge added bbox_extra_artists as in the following:

        self.fig.savefig(plot_filename, dpi=self.dpi,
                         bbox_inches="tight", bbox_extra_artists=[],
                         pad_inches=0.10)

Should create a PR for this.

[wtgee]

I will test this on PAN001, where the plot was already working, and make sure it is consistent. Then let's get the one-word PR in. :)