Add DataModel base class

[guillochon]

See #3674 (comment) for a new, better idea for this feature.

---

Hi all, I'm having an issue where line plots will sometimes be invisible when rendered. This error seems to be non-deterministic; when I run the same code repeatedly, the plots will sometimes render as normal, and sometimes render invisibly.

An example page that is currently showing these "invisible" plots is available here (2nd plot from the top): https://sne.space/sne/SN1993J-broken.html. One thing you can notice is that hovering over the plot still produces tooltips, and an examination of the data within the page source reveals that the line_color attributes are non-white, and that the data is actually available. But for some reason, the lines are not visible!

And here's the exact same page when the render does work: https://sne.space/sne/SN1993J.html. The file sizes are identical but the file contents are not identical. It seems like it might be a different command order in the two files?

Anyone know what's going on?

[birdsarah]

hi @guillochon, thanks for the report. I agree, I can see the tooltips but not the lines. Without the code that was used to produce the two examples "broken" and not it's impossible to know what's going on - I've certainly not seen this before and the console's not telling me anything useful if that helps.

[guillochon]

The code is what I posted here: #3671. The code used to produce both examples is identical, I literally just ran the code and ran it again. Sometimes the issue occurs, sometimes not. It's not deterministic!

[birdsarah]

I literally just ran the code and ran it again. Sometimes the issue occurs, sometimes not. It's not deterministic!

I had misunderstood - I thought it was at runtime - when the html loaded, not running your bokeh script.

I am not sure what's going on. My only guess is that HBox(vplot(hplot(p1,p2,vform(binslider,spacingslider)))) adds things in a non-deterministic order. I seem to remember there being something about order not being guaranteed for children of a document. ping @havocp

[guillochon]

So I got rid of vplot, hplot, and vform, and the problem still persists. I generate thousands of these pages in a loop, and it seems like when it's not working, all the plots are invisible, on every single page. I'll restart the loop several times, and randomly everything will render nicely.

Is it possible for someone to look at the broken link above to see the ordering of the output? It is different than the "good" plot, suggesting that the output order is affecting the display of the data. I have no idea how to fix this and it makes automation of the script impossible (I have to babysit it and make sure it properly generates the plots).

[birdsarah]

ping @bryevdv @mattpap

[havocp]

Possible clue: the working HTML page is 10x larger than the broken one.
What I would do is extract the JSON from each page into its own file, then pretty-format it with a JSON formatter, then do a diff. If the diff is obfuscated by different UUIDs, use a regex to strip all the UUIDs out perhaps.

[guillochon]

@havocp Since I first posted this issue more data was added to the plot. The file sizes were identical previously. I'll try to grab a broken example again the next time the script executes.

[guillochon]

This is kind of a deal-breaker for me for Bokeh, I can't have something that doesn't plot the data half the time. Does anyone have any idea of things I can try to fix this? What is Bokeh even doing here that would result in the non-deterministic output I'm seeing? Can I force Bokeh to output in a particular order, if that's what's causing the issue?

[havocp]

Do you have time to try the cleaned json diff I suggested? ideally two pages with the exact same data but one doesn't work ...

I have no idea what would cause this so step 1 for me would be to see what's different about the working and broken pages, by getting things human readable then diffing.

also maybe given a script to make the pretty diff you could automate generating a failure because you could regenerate, remove noise such as changed uuids, and then if the two pages aren't identical you know one of them is broken.

There's a "bokeh json" command too so if the problem is the json varies we could get the html out of the picture.

[havocp]

do you have a way to freeze the data so you can take variation in data out of the picture for debugging ?

[havocp]

Also useful would be to try to minimize a test case that's easy for anyone to run (includes all needed data files, etc).

[guillochon]

OK, I have two files that should not change now, one broken, one working. A diff is just a mess, the JSON is in a completely different order:

https://sne.space/sne/SN1990aa-broken.html
https://sne.space/sne/SN1990aa-working.html

[havocp]

I can't work on this tonight but I'd probably write a little json cleaner script to sort the keys and change all the uuids to just the string "uuid" or something and then pretty print.