Remove PyPlot from ModiaMath installation

[MartinOtter]

There are the following problems:

• Pkg.add("ModiaMath") fails with the JuliaCIBot of the Julia package registry (METADATA) because errors occur when "PyPlot" is installed by JuliaCIBot, see Tag ModiaMath.jl v0.2.1 #16719. Manual intervention of the Julia team is needed to still accept a release (because the CL of Julia fails and the Travis CL of the ModiaMath github repository has to be inspected). This is bad, because such a manual procedure is then also needed for Modia and Modia3D (because they depend on ModiaMath).

• The Travis CL of ModiaMath required some "tweaking" in the .travis.yml script to explicitely install PyPlot.

• It is also not nice, that potential users must first install PyPlot correctly, before they can run a simulation example (even, if they just would like to try ModiaMath out, without taking the large burden to install PyPlot).

• Most of the time of the Travis CL (about 10-15 min. from 20 min) is spent to install PyPlot. Getting feedback of a failed Travis CL would be much quicker if PyPlot would not be installed. This holds probably for every reasonable plot package (e.g. Makie), because they have a huge functionality and installing all of this takes time.

The issues above could be fixed if the result plotting would by default have the plot package "NoPlot" (so the command "ModiaMath.plot(..)" would just give an info message that no plot package has yet been selected for ModiaMath) but no error would occur. Once the user has installed PyPlot, ModiaMath.plot(..) would plot with PyPlot (in the future optionally also with Makie; currently, I cannot use Makie, because it does no install on Windows; opened a ticket).

The implementation could be performed in a similar way as in Plots by using the @require macro from package Requires.jl that makes functionality only available if the defined package was loaded outside of the package.

Help and suggestions are welcome (@HildingElmqvist, @tshort, @ChrisRackauckas).

P.S. The goal would be to release a new version of ModiaMath with the feature above, before further versions of Modia and Modia3D are registered in METADATA of Julia.

[ChrisRackauckas]

No, don't use Requires for plotting. Just use the recipe systems for Plots.jl or Makie.jl.

[crlaugh]

@ChrisRackauckas: With your background in the Julia ecosystem, would you recommend moving to Plots.jl or Makie.jl? I know that DifferentialEquations.jl uses Plots, while people seem to be talking about Makie a lot these days. It would be helpful if you could lend a perspective on the choice between the two.

[MartinOtter]

No, don't use Requires for plotting. Just use the recipe systems for Plots.jl or Makie.jl.

The problem is that installation of Makie.jl on Windows fails, so I cannot use it currently.

The problem with Plots.jl is that it has no concept of "windows". Some of the backends open a plot always in a new window, and some always plot in the same window without the possibility to plot in a new window (e.g. Plots option "reuse=false" is not supported for GR). PyPlot seems to be the only backend of Plots that has reasonable line plots and allows to plot in existing windows and in new windows (has a concept of window handles). Even if in Plots other backends would support reuse=true and reuse=false it seems not possible to define that this plot goes in window 3 or goes in the window with handle h as in PyPlot.

I also tried (in Julia 0.6) to use PyPlot via Plots (ignoring the problem with the window number). However, the plot area is clipped in some cases (if using PyPlots via Plots) so that a line close to the maximum tick is no longer visible, whereas when using PyPlot directly this error is not present.

To summarize, I really would like to use something that is installed in a more stable way as PyPlot, but this does not help if the working mode, that is completely standard in Modelica simulators, seems to be not supported by Plots. Of course, it could also be that I overlooked the feature that I need.

[MartinOtter]

No, don't use Requires for plotting. Just use the recipe systems for Plots.jl or Makie.jl.

I made new experiments with Plots with Julia 1.0 with backend PyPlot. The ?Plots.plot gives the result:

  The main plot command. Use plot to create a new plot object, and plot! to add to an existing one:

      plot(args...; kw...)                  # creates a new plot window, and sets it to be the current
      plot!(args...; kw...)                 # adds to the `current`
      plot!(plotobj, args...; kw...)        # adds to the plot `plotobj`

This is possible:

Plots.pyplot()
x=collect(1:10); y=sin.(x)
h1 = plot(x,y)
h2 = plot(x,2*y)

Contrary to the documentation, plot generates a new window the first time and then the following plot does not generate a new window but clears the window and then adds the content. If kw reuse=true is used (or plot!(...)), the previous window content is not cleared. So for the "current window" the needed functionality is present.

However, if several windows have been generated, I did not find a way to, say, clear the first window and plot the content in the first window. This would be somehow possible, if one could define, that a window with a given handle is defined to be the current window. But I did not find such an option (plot!(h1, x,y, resuse=true) or plot!(h1, x,y, resuse=false) all add content to h1, without clearing the content beforehand; so reuse is ignored).

So, to summarize, it seems not possible that the first simulation generates 3 windows and in the second simulation these 3 windows are reused (so the previous content is removed and replaced by the result of the new simulation). Or is this possible "somehow" (note in PyPlot, this is of course a basic feature, but I did not find something of this kind when using PyPlot via Plots)?

[tshort]

I like removing PyPlot as a dependency. As for Plots, I've had good luck with the default GR backend on windows.

If we convert to the DiffEq common interface, plotting will come for free. Another option is to use a DiffEq recipe as a starting point.

[crlaugh]

@MartinOtter The authors of Makie have made a lot of progress in its pursuit of 1.0 compatibility, and it has recently been updated to reflect this work. I just installed a fresh copy of 1.0 on a 64-bit Windows machine (ver. 1803), installed Makie, and all of its tests (]test Makie) pass.

[MartinOtter]

@MartinOtter The authors of Makie have made a lot of progress in its pursuit of 1.0 compatibility, and it has recently been updated to reflect this work. I just installed a fresh copy of 1.0 on a 64-bit Windows machine (ver. 1803), installed Makie, and all of its tests (]test Makie) pass.

I just tried again and got errors during test (was terminated after some errors). There is also a message that ImageMagick is not properly installed and that Pkg.build("ImageMagick") has to be made. This also fails. I reported this previously as an issue to Makie.

I guess the installation of Makie worked for you, since you used Administrator rights for the installation (during installation symbolic links are set, and this requires Administrator rights).

O.k. what I could do is that I install Makie via the IT departement and then can at least perform evaluations and tests (however, for including it in a released version of ModiaMath, I would have to wait until the issue is fixed).

[MartinOtter]

I like removing PyPlot as a dependency. As for Plots, I've had good luck with the default GR backend on windows.

You are right GR seems to be a good default backend. However:

Using GR via Plots has the missing functionality as with other Plots backends (as described above: not possible to plot in several windows and then reuse these windows for other simulation runs and clearing the content in the figures).

I had a quick look to call GR directly. However, there is nearly no documentation and also looking in the source code did not help. I could not figure out, how to open several figures/windows. Using "supplot(..)" in the way as in matplotlib gave error messages.

[MartinOtter]

I have inspected Plots to figure out how Plots can cope with different backends. However, I could not figure this out. Anyway, I got some inspiration, and made the following small test package:

module TestPlot

export myplot

const PyPlotAvailable = fill(false,1)

function __init__()
    @eval Main begin
        try
            import PyPlot
        catch
            println("... PyPlot not installed (plot commands will be ignored)")
        end
    end

    PyPlotAvailable[1] = isdefined(Main, :PyPlot)
end
   

function myplot(x,y)
    if PyPlotAvailable[1]
        Main.PyPlot.plot(x,y)  
    else
        println("no plot, because PyPlot not installed")
    end
end

end # module

Once this package is added to the actual environment (]add TestPlot), then it can be used as:

   using TestPlot
   t=collect(0:0.01:10); y=sin.(t)
   myplot(t,y)

This package has no dependency on PyPlot and the reaction depends on whether PyPlot is in the actual environment or not:

• If PyPlot is in the actual environment, myplot(..) makes a PyPlot plot.
• If PyPlot is not in the actual environment, myplot(..) gives the information message that PyPlot is not installed.

[ChrisRackauckas]

Plots.jl has the quite necessary recipe functionality that PyPlot.jl doesn't have. Using recipes you don't have to worry about supporting units in your plotting code since you instead just output arrays with units and develop the plotting support for units separately. This works with any other type as well. So recipes are a 👍 for making this kind of code sane.

Plots.jl has a lot of faults though. It cannot do multiple windows. It cannot move legends very well. It will not do PDE plots very well because its surface plots do not allow full control of the polygons. Makie.jl fixes this but isn't fully released yet. The todo list is here: MakieOrg/Makie.jl#134 . Then you just rely on https://github.com/JuliaPlots/AbstractPlotting.jl

[ChrisRackauckas]

Evaling into Main to do optional dependencies is not safe for a few reasons related to precompilation and also some IDEs/dev packages take some liberty with Main in order to do clearing / workspace swapping. This used to be a big enough sin that it was one of the things that was looked for an disallowed in METADATA-registered Julia packages. Nowadays no one would check close enough for it, but I still wouldn't recommend doing it. If that's really really how you want to handle it, then use @requires which essentially is doing what you want (adds an eval into the init function) but has extra hooks into the compilation setup to make it actually correct.

But the reasons to not use Requires is moreso for the benefits of recipe systems when natively handling non-Float64 types, which is something that I believe ModiaMath is looking to do.

[MartinOtter]

Plots.jl has the quite necessary recipe functionality that PyPlot.jl doesn't have. Using recipes you don't have to worry about supporting units in your plotting code since you instead just output arrays with units and develop the plotting support for units separately. This works with any other type as well. So recipes are a +1 for making this kind of code sane.

Plots.jl has a lot of faults though. It cannot do multiple windows. It cannot move legends very well. It will not do PDE plots very well because its surface plots do not allow full control of the polygons. Makie.jl fixes this but isn't fully released yet. The todo list is here: MakieOrg/Makie.jl#134 . Then you just rely on https://github.com/JuliaPlots/AbstractPlotting.jl

Thanks for your comments. My conclusion:
Temporarily continue to use PyPlot in ModiaMath but if possible, do not have it in the distribution. Once Makie supports several windows, switch to Makie.

[ChrisRackauckas]

Yeah, I'd use Requires.jl for now just to get things done, with the intent on using AbstractPlotting.jl recipes to support Makie.jl in the near future once Makie is ready.

[MartinOtter]

Yeah, I'd use Requires.jl for now just to get things done, with the intent on using AbstractPlotting.jl recipes to support Makie.jl in the near future once Makie is ready.

Before trying the TestPlot package above, I tried to use Requires.jl as done in Plots. However, I did not manage to get the desired feature. O.k. will make another try.

[MartinOtter]

In ModiaMath version 0.2.2, PyPlot was removed from the distrubtion according to the sketch above. The implementation idea with Requires is described on discourse.