Agent inspection on mouse hover

[fbanning]

Closes JuliaDynamics/Agents.jl#538

This turned out to be a bigger endeavour than initially anticipated. I had to refactor the codebase a bit and most notably had to introduce a new Makie plot recipe.

The recipe introduces the ABMPlot type and its new generated functions abmplot and abmplot! which takes a list of agent positions as well as the model object and converts them into observables. Those are then accessible via the plot object itself (e.g. plot.model[]). The new type can be extended at will and we can easily create new methods for different use cases, e.g. for GraphSpace models later on.

Showcase

Same start for every showcase example:

using Agents
using InteractiveDynamics
using GLMakie

model, agent_step!, model_step! = Models.schelling()

That means I've only tested a simple 2D GridSpace model for now.

Then we try out the three main plotting functions.

## abm_plot with kwargs

ac(a) = a.group == 1 ? :red : :blue
fig, abmstepper, inspector = abm_plot(model; ac=ac)
fig

## abm_play

fig, abmstepper, inspector = abm_play(model, agent_step!, model_step!)
fig

abm_play.mp4

## abm_data_exploration
using Statistics

fig, adf, mdf = abm_data_exploration(model, agent_step!, model_step!, Dict(); 
    adata=[(:mood, mean)])

abm_data_exploration.mp4

Here we can see why it was necessary to introduce the new ABMPlot type. We can inspect scatter points as agents on the top left axis but still regularly inspect the scatter points on the observed variables on the right hand side. :)

Sidenote

I've also introduced the possibility to use a different colourscheme than the default one. Sorry about mixing it into this PR. Realised it a bit too late that it would have been nicer to split it into another PR. It's not a very big thing but is something that was on my to-do list for a while. :)

To do

• 3D models
  • Implement 3D plotting method (untested basics are already there)
  • Test
• 2D model with polygons
  • Implement poly plotting method
  • Test
  • Disable inspector until properly implemented
• Test with a ContinuousSpace model
• Test abm_video for 2D, 3D and poly plotting
• Truncate most floats
• Allow custom agent strings for tooltips.
• Update docstrings and add documentation for new features
• Test and document colorscheme keyword

Further ideas

• As proposed here, we could improve the tooltip formatting in cases with multiple agents on one position.
• Maybe modelobs is unnecessary now that we could just use plot.model[]?
• Maybe some of the Observable conversions (ac, am, as, etc.) can be dropped when they're integrated into the plot object as Attributes. Not sure about this as it might mess with the abmstepper flow.

That's it from my side. Happy to answer any questions, discuss function naming as well as your ideas for extensions/changes. :)

[fbanning]

I probably won't have time to work on this until the end of August. I've enabled the ability for repo maintainers to add commits, so please feel free to enhance, change and/or merge at will. :)

[Datseris]

I'm a bit behind here. So if I merge this, the polygon plotting goes away?

[fbanning]

No, polygon plots haven't been removed. They are just not yet

1. integrated into the new ABMPlot recipe system, meaning that there's no specialised inspector functionality for the poly! plots
2. tested at all

I would advise against merging this PR right now until the To-Do list above has been worked on. I will mark the PR as Draft for now, just in case.

[fbanning]

I've just tested a poly model, namely the one in Agents.Models.flocking. Plotting is working just fine although the mouse hover tooltip is of course not very helpful and just displays the coordinates of the bounding box of all polys.

abm_plot:

abm_play:

abm_play.mp4

abm_data_exploration:

abm_data_exploration.mp4

And here's a screenshot of the not-so-helpful-yet tooltip on mouse hover:

So yes, I can confirm that poly plots still work as they did before, although they aren't integrated into the new ABMPlot recipe system yet (which results in these useless tooltips).

But it's definitely not a showstopper and shouldn't block the PR from being merged.

However, if someone wants to contribute the recipe for poly plots, feel free to do so. :)

[fbanning]

I just run this in the Schelling model where I change each agent's plot style and I see wrong clipping

Hm, this is weird and hasn't happened to me during my tests. Will try to reproduce it again.

Anyways, I think this looks like a problem with the depth value of the inspector (see here). By default it is set to 9e3 but perhaps we need to put it...

[Libbum]

H: Ah, I see! Not sure we need to go that deep. It was confusing me that it was an 'obvious' choice, but could not be found in our code.

polygon tooltips and other features: No need to rush these anymore. We wanted to merge this on the weekend for the tutorial, but since we didn't end up doing that the time pressure is not so bad.

depth: I tried that locally, but setting it to 1e20 didn't really work at all. The plot even failed to show... Maybe I did it wrong though..

[fbanning]

depth: I tried that locally, but setting it to 1e20 didn't really work at all. The plot even failed to show... Maybe I did it wrong though..

Yep, same result here. Once I use a function as input for am, I get this overlap problem. Simply providing a different marker (e.g. am=:rect) seems to work fine. Not sure yet what's the root of the problem here but I'll do some more digging.

[fbanning]

* We need better parsing for more complicated data. I wonder if we can actually allow the user to provide a function like `show` of their agent?

Just tested this and it is already easily doable. Users only need to define a specialised method of the InteractiveDynamics.agent2string function and this will already work.

InteractiveDynamics.agent2string(agent::Models.SocietyMember) = "This is an agent string."

[Libbum]

define a specialised method of the InteractiveDynamics.agent2string function

Awesome! Lets export that function then, and provide a more 'complex' example to simplify the display of an agent like Bird.

This actually negates a lot of the floating point rounding requirements I was eluding to as well, since users can format their agent details as they like.

I assume this string can be mulitline right?

[fbanning]

Awesome! Lets export that function then, and provide a more 'complex' example to simplify the display of an agent like Bird.

Done. Went with an imaginary example that should still display what to do without relying on a specific example.

I assume this string can be mulitline right?

Sure thing.

[fbanning]

In b89cdb7 I've attempted to separate the initialisation of the abmstepper and the plots themselves. I think this makes the code a lot more maintainable in the long run and also contributes to keeping the functions a bit more "clean" as they now only have one major goal instead of doing multiple things at once. I've ran some tests (although not as extensive as all of those above) and everything seems to be in working order.

[fbanning]

Oh and in df542e2 I've changed the return values of the plotting functions back to what they were before. Initially I returned the inspector because I thought it might be helpful to have easy access to it but that's not really needed since it (the inspector) can be easily retrieved from the fig object. This avoids breaking other people's existing code.

[Libbum]

Thanks Fredrick! I'll have another once over tomorrow hopefully, but each of your changes sounds like a good idea.

[fbanning]

Yep, same result here. Once I use a function as input for am, I get this overlap problem. Simply providing a different marker (e.g. am=:rect) seems to work fine. Not sure yet what's the root of the problem here but I'll do some more digging.

I've poked around a bit and could not find out why the tooltip is displayed below the scatter points in the case of a function for am. Can somebody else please have a look? I'm out of ideas.

[Datseris]

I think only Makie people can answer this :(

[Datseris]

@fbanning have you contacted Makie people about this? In any case, to my understanding, the current functionality does not break anything, works for standard plotting and semi-works for function-as-marker plotting. It doesn't break it, it just shows the tooltip below the markers.

So, I think we should be able to merge and tag this, and let the remaining as a bugfix issue?

[fbanning]

have you contacted Makie people about this?

No, sorry, didn't find the time to do that. Will try to do it next week. Won't matter though if I refer to this PR or to another newly created bugfix issue.

I think we should be able to merge and tag this

Yes, thanks, I think so too. However, documentation needs to be updated before merging this PR, otherwise users won't know that (a) this functionality is there and (b) they can easily change the tooltip with an own method for their agent types. If somebody wants to write this up, please feel free to do so. Otherwise I'll also try to find an hour for that next week.

[Datseris]

I must admit I don't feel familiar enough to document how to do the custom agent tooltip so I'll leave that to you. Just tag me once you're done and we can merge.

[fbanning]

And I've just realised that I've never shown that the newly implemented colorscheme keyword actually works.

# same tests as above just with two additions
colorscheme = [:red, :blue, :green, :yellow, :grey]
ac(a) = a.group == 1 ? colorscheme[1] : colorscheme[2]

fig, adf, mdf = abm_data_exploration(model, agent_step!, model_step!, params; 
    adata=[(:mood, mean), (:mood, minimum)], ac, colorscheme)

I've already previously added it to the list of figure related keywords in the docstring of abm_plot, so it's documented enough, I think.

[fbanning]

@Datseris I think you can merge this now.