Suggestion: Show value indications on Fig 22.2

[zsunberg]

Hey guys, the book looks great :)

In Figure 22.2 it might be neat to include indications of the value estimate, Q(h, a) at each action node, perhaps with color. Then it will be easy to see that the branches with higher value are favored.

It may also be useful to indicate the number of times a branch has been simulated with the thickness of the line.

I'm not sure what the aesthetic consequences of doing this are though :)

I didn't look but I assume there is also a figure in the MDP section that could benefit similarly.

[mykelk]

Ooh... what do you think @tawheeler ?

[tawheeler]

We can certainly try line thickness. The weirdness there is that as far as I know, TikZ can't taper easily, so we wouldn't get smooth transitions between them without drawing node circles to cover them up.

I am reluctant to put a bunch of labels with numbers all over the image. If we are drawing node circles for the first bit, we could color them, and that would result in adding a horizontal colorbar - that sounds complicated. We can try this but let's make it low priority.

[zsunberg]

Yeah, it seems like numbers definitely don't belong on there - colored circles would be much more appropriate. I don't think you would necessarily even need a colorbar. Just say something like "the color of the action nodes indicates the Q estimate; green is higher, red is lower"

[tawheeler]

I now have a way to render gradient edges of varying thickness!
This might happen much sooner than I thought.

[tawheeler]

How about something like this:

Getting a good layout for varying thickness is proving more difficult than I'd like. Visitation counts are thus merely printed.
Note that this tree, to prevent it from being very tall (ie wide), does not show nodes with zero visitation counts.

(I can mess with it to get better colors)

[mykelk]

I'm happy if you're happy. yeah, the current color scheme looks like the new instagram logo.

[tawheeler]

Done! (for both MDP MCTS and POMDP MCTS)

[zsunberg]

I like this - the one thing that might make it easier to read is to not put the counts on the borders of the color patches, but that is a minor factor.