New: matplotlib in submissions, not just the sandbox

[TomNaessens]

We recently expanded Dodona’s data-science support in TESTed: submissions can now generate matplotlib plots, have them graded, and show the resulting chart directly in the feedback.

pandas, NumPy and SciPy already worked in submitted solution, and matplotlib worked in the in-browser sandbox. What was missing was support for matplotlib-generated figures in submitted solutions. That is now in place.

This means an exercise can now ask students to create a plot, test both the underlying values and the visual output, and show the generated chart back to the student as part of the feedback.

We added two example exercises to show how everything fits together:

• Monthly rainfall chart
• Pokémon stats

How it looks

Let's walk through the first example, Monthly rainfall chart. In this exercise, students write a function that aggregates a year of measurements with pandas and draws a bar chart with matplotlib.

In the sandbox, pressing Run renders the chart directly in the browser. This already worked in previous versions of Dodona.

What is new is that students can now submit the same code. The submitted solution is then tested like any other Dodona submission. In this example, the twelve monthly totals are checked by value...

...and the chart drawn by the submitted code is captured and shown in a Your chart tab, next to the test results:

The Pokémon stats exercise uses the same mechanism for a busier chart, coloured by type and including value labels and a legend:

How it works, for exercise authors

Two practical questions come up when you want to include a plot in an exercise:

1. How do you test the plot?
2. How do you show the generated plot in the feedback?

Both example exercises live in the example-exercises repo. Below is how each part works.

Testing a chart in code

You do not need matplotlib-specific grading machinery to test a plot. One useful pattern is to have the student return the matplotlib Figure and then inspect it like any other object.

The Pokémon stats exercise does exactly that. The test suite checks the title, the number of bars, the bar heights, and the x-axis labels (suite.yaml):

- statement:
    python: "fig = submission.plot_top_pokemon('pokemon.csv', 5)"
- expression:
    python: "fig.axes[0].get_title()"
  return: "Top 5 Pokémon by total stats"
- expression:
    python: "len(fig.axes[0].patches)"
  return: 5
- expression:
    python: "[round(p.get_height()) for p in fig.axes[0].patches]"
  return: [680, 600, 540, 540, 534]
- expression:
    python: "[t.get_text() for t in fig.axes[0].get_xticklabels()]"
  return: ["Mewtwo", "Dragonite", "Snorlax", "Gyarados", "Charizard"]

The Figure object itself does not need to be serialised. Each check reads a plain serialisable property (a string, an integer, or a list) so TESTed can handle the result like any other value.

Showing the chart in the feedback

To show the generated chart in the feedback, we use a small custom-check oracle. The oracle cannot access the live matplotlib figure directly: it runs in the judge and receives serialised values. Instead, the test suite first saves the figure to chart.png. The oracle then reads that file, base64-encodes it, and returns it to Dodona as an HTML message (render.py):

with open(chart_path, "rb") as chart_file:
    encoded = base64.b64encode(chart_file.read()).decode("ascii")

image = f'<img src="data:image/png;base64,{encoded}" alt="Your chart">'

return EvaluationResult(result=True, messages=[Message(image, "html")])

That leaves one question: how does the figure end up in chart.png?

We deliberately used two different patterns in the example exercises, so both styles are illustrated.

1. The student returns the Figure (Pokémon). The test harness saves it directly (solution.py). This makes the figure easy to inspect in the tests, and the test suite can also save it directly before calling the oracle:

- statement:
    python: "fig = submission.plot_top_pokemon('pokemon.csv', 5)"
# ...the introspection checks from above...
- statement:
    python: "fig.savefig('chart.png')"
- expression:
    python: "True"
  return: !oracle
    oracle: "custom_check"
    file: "render.py"
    name: "render"

This is often the cleanest approach when you are designing a new plotting exercise from scratch: returning the Figure gives the test suite a clear handle on the object it needs to inspect and render.

2. The student just draws and calls plt.show(). In Monthly rainfall chart, the student uses the familiar matplotlib workflow: draw the chart and call plt.show(). No figure is returned.

During judging, matplotlib runs headlessly using the Agg backend. That means plt.show() is a harmless no-op: it does not open a window, and the figure remains available as the current figure. The test suite can then retrieve it with plt.gcf() and save it to chart.png (suite.yaml):

- statement: "import matplotlib.pyplot as plt"
- statement: "submission.rainfall_chart([[1, 49], [2, 41], [3, 38], ...])"
- statement: "plt.gcf().savefig('chart.png')"
- expression: "True"
  return: !oracle
    oracle: "custom_check"
    file: "render.py"
    name: "render"

This pattern is useful when you want students to write plotting code in the same style they would use in a notebook, a script, or the Dodona sandbox.

Where to find them

Both exercises (suite, solution and oracle) are available in example-exercises/tested. They are intended as starting points for your own data-visualisation exercises.

Let us know if you build something with them, or if you run into anything!