Example for how to plot without jupyter?

[Paethon]

Hi

Would be great to include an example on how to plot contours etc. without using Jupyter. I tried to find out how to do that but have no idea how to plot the AxPlotConfig outside of Jupyter.

Or is this just not supported?

[kkashin]

@Paethon - the plotting functionality was designed to work in both notebooks and in standalone html files that we use for generating "reports" that can be easily shared - so yes, other modes are definitely supported. We highlight the notebook usage on the site because in our experience that's the primary way users have interacted with the tool for visualizing the models & results.

Here's an example of how one can run the CNN on MNIST tutorial and get the contour plot in an html file:

import torch
import numpy as np

from ax.plot.contour import plot_contour
from ax.plot.trace import optimization_trace_single_method
from ax.service.managed_loop import optimize
from ax.utils.tutorials.cnn_utils import load_mnist, train, evaluate

# note these new imports!
from ax.plot.render import plot_config_to_html
from ax.utils.report.render import render_report_elements


dtype = torch.float
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

train_loader, valid_loader, test_loader = load_mnist()

def train_evaluate(parameterization):
    net = train(train_loader=train_loader, parameters=parameterization, dtype=dtype, device=device)
    return evaluate(
        net=net,
        data_loader=valid_loader,
        dtype=dtype,
        device=device,
    )

best_parameters, values, experiment, model = optimize(
    parameters=[
        {"name": "lr", "type": "range", "bounds": [1e-6, 0.4], "log_scale": True},
        {"name": "momentum", "type": "range", "bounds": [0.0, 1.0]},
    ],
    evaluation_function=train_evaluate,
    objective_name='accuracy',
)

plot_config = plot_contour(model=model, param_x='lr', param_y='momentum', metric_name='accuracy')

# create an Ax report
with open('report.html', 'w') as outfile:
    outfile.write(render_report_elements(
    "example_report", 
    html_elements=[plot_config_to_html(plot_config)], 
    header=False,
))

We're working on making the plotting a bit more modular (and not always depend on require.js), which will make it even easier to put together a barebones html file without using the render_report_elements that handles the dependency management for you. I should note that when using the report functionality, you get access to a couple other utilities that make it easy to compose reports for viewing of results (see, e.g., benchmarking reports)

Are there other ways of plotting that you had in mind, or does outputting an html file work for you?

[Paethon]

Thanks!

Generating an HTML report is actually the best solution for me anyway. I found plot_config_to_html(plot_config) and tried writing that directly to an html-file, but that did not really work out as intended 😄

[kkashin]

Great - happy that works for you. Once we do the refactor of the plotting, it should be possible to write plot_config_to_html(plot_config) directly to HTML file and for that to work. Issue now is that you need a bunch of dependencies in the HTML head in order for rendering to work.