Probabilistic programming

[lukego]

I'm taking my first baby-steps with using Kons-9 to visualize probabilistic programming results. It's fun!

Here's my very first plot that shows some random samples next to their underlying distribution, a slightly awkward test case of a Gaussian with a notch cut out on one side:

Even at this basic level it's already more fun for me to use Kons-9 than Blender, Julia, or R.

The next thing I need to figure out is how to draw a line/curve through a series of points and how to change colors 😀

Context/code: https://twitter.com/lukego/status/1585187363474485249

Thanks @kaveh808 for this already great tool!

[kaveh808]

Good stuff. Check out curve.lisp. You can set point colors, but not sure if curves render them. Let me know if you have questions.

[lukego]

Hey cool! kons-9 is helping me to debug a Sequential Monte Carlo (particle filter) simulation today 😁

The simulation calls kons-9 to plot its internal state in a new color after each step. This way I can watch the simulation proceeding in real time. Being able to pan/scale interactively also helps for making sense of information without endlessly tweaking plotting settings.

This is actually really helpful. The convergence is happening much more suddenly than I expected (from right to left below) and this gives me an idea about where to look for a bug that's causing bad results in other higher-dimensional simulations.

Thanks @kaveh808 for a great tool! Too bad you weren't at ELS2023 this week because @foretspaisibles and I would have loved to buy you a beer 🍻

[kaveh808]

Hi @lukego ,

Wonderful! This made my day. SO glad you're finding the system useful.

I just watched the videos from ELS on Twitch. Wish I could have been there. Someone suggested that I make a demonstration proposal for next year.

P.S. you can set the bg to black by doing (set-theme-dark)

[aykaramba]

I love seeing people use Kons9 in such interesting ways. Post more, would love to watch / read.

[lukego]

Thanks for the warm feedback! I posted some more notes on this little side-quest here: lukego/blog#37

[kayomarz]

This is really inspiring @lukego! I've been playing around with kons-9 since a couple of weeks and it is indeed awesome thanks to @kaveh808!

btw @lukego I like your way of using git issues for blogging and might do just the same:)

[lukego]

Here is another Monte Carlo diagnostic that takes better advantage of 3D space: lukego/blog#38

[lukego]

This might be starting to overlap with the domain of Paraview for visualizing scientific and engineering data. Though I think that non-interactive plots with e.g. R/ggplot2 are more common for the diagnostics that I'm doing.

[lukego]

Relatedly: There was a company at ELS who employ ~15 data scientists and use an in-house Emacs-based user interface for their tools. They had recently hired a dedicated person to maintain their Emacs tooling full time. This seems quite similar to building tooling on kons-9.

[kaveh808]

I'd be curious to know more about the company at ELS and their use of Emacs for data visualization.

[lukego]

I don't know exactly what they do with Emacs but the company is Machine Learning Programs. My understanding is that they do machine learning models for the insurance industry. They are using Python on the backend. Breanndán Ó Nualláin is the person I talked with (he was the local organizer in Amsterdam for ELS this year.)

[kaveh808]

Cool, thanks.

[lukego]

My next diagnostic turned out to fit pretty well with heightmaps: lukego/blog#39.