Scatterbrain rendering in vis, including shader generation by froyo-np · Pull Request #223 · AllenInstitute/vis

froyo-np · 2026-01-27T23:15:36Z

A Not quite feature complete implementation of basic scatterbrain scatterplot rendering

What

How

Replace this txt describing what kind of technical overlaying code changes were introduced here.

Screenshots

This section is optional if there are no visible changes

If possible add screenshots of the visible additions in the UI.
If there are changes in the UI, add Before and After Screenshots for quick overview.
If there was a Figma design, add a link to that here as well.
Hint : Drag and Drop any images you want to add to the PR. Also you can create a gif of an interactive version and add that!

PR Checklist

… demo. initial thoughts

…tor decide to continue traversal (or not) in addition to whatever effectful thing it would normally do

…out and start smaller...

… about how to generate these shaders in a less hard to modify way

froyo-np · 2026-01-28T17:01:10Z

packages/geometry/src/spatialIndexing/tree.ts

+export function visitBFSMaybe<Tree>(
+    tree: Tree,
+    children: (t: Tree) => ReadonlyArray<Tree>,
+    visitor: (tree: Tree) => boolean,


a slight variation on the previous visitBFS pattern - this one packs the choice to proceed into the visitor - the idea is that the visitor performs some side-effect, and returns true if recursion should proceed.

froyo-np · 2026-01-28T17:01:49Z

packages/scatterbrain/src/dataset.ts

+import type { ScatterbrainDataset, SlideviewScatterbrainDataset, TreeNode, volumeBound } from "./types";
+import reduce from "lodash/reduce";
+
+type Dataset = ScatterbrainDataset | SlideviewScatterbrainDataset


I tried to pull in and hopefully better organize the code for loading and traversing these datasets

froyo-np · 2026-01-28T17:02:02Z

packages/scatterbrain/src/dataset.ts

+
+export function loadDataset(raw: any): Dataset | undefined {
+    // index point attrs by name - its an array
+    // TODO zod validation first!


yup - I need to do this

froyo-np · 2026-01-28T17:03:31Z

packages/scatterbrain/src/renderer.ts

+type Content = Record<string, VBO>
+
+
+export function buildScatterbrainCacheClient(regl: REGL.Regl, cache: SharedPriorityCache, onDataArrived: () => void) {


here is our cache client implementation - it handles the fetching of the various types of data (genes or metadata) and uploading into GPU buffers

froyo-np · 2026-01-28T17:04:28Z

packages/scatterbrain/src/renderer.ts

+    }
+}
+
+function buildHelperThingy(regl: REGL.Regl, state: ShaderSettings) {


this short function has a silly name - its probably better to just move it into the one place its called

agree, I can't really come up with a better name than qtCellHelper :D

froyo-np · 2026-01-28T17:05:31Z

packages/scatterbrain/src/renderer.ts

+    }
+}
+
+/*      TODO features:


remove this comment

froyo-np · 2026-01-28T17:06:17Z

packages/scatterbrain/src/shader.ts

+import { Box2D, type vec4, type box2D, type Interval, type vec2 } from "@alleninstitute/vis-geometry";
+import { type Cacheable, type CachedVertexBuffer } from "@alleninstitute/vis-core";
+
+// the set of columns and what to do with them can vary


consider moving this commentary to a readme, or at least making it less rambly

I'm okay if this is here for now. I've been exploring producing documentation automatically from JSDoc comments, or having us enhance our README docs and get those deployed on the vis website (or maybe both options!).

Maybe turn into a JS Doc comment attached to the VBO class though?

froyo-np · 2026-01-28T17:07:49Z

packages/scatterbrain/src/shader.ts

+// get without a great deal of extra performance cost
+
+
+// scatterbrain does scatterplot rendering


this file is worth looking at closely. Its shorter and more centralized than the original ABC-atlas implementation, and I really hope its more readable. I think its got some nice guide-rails for how to think about making these, but because we still compose strings, it can be easy to make spelling or GLSL syntax mistakes.

froyo-np · 2026-01-28T17:08:33Z

packages/scatterbrain/src/shader.ts

+
+export function generate(config: Config): ScatterbrainShaderUtils {
+    const { mode, quantitativeColumns, categoricalColumns, categoricalTable, tableSize, gradientTable, positionColumn, colorByColumn } = config;
+    console.log('tableSize: ', tableSize)


remove console logs

froyo-np · 2026-01-28T17:09:30Z

packages/scatterbrain/src/shader.ts

+    const getDataPosition = /*glsl*/`return vec3(${positionColumn}+offset,0.0);`
+    const getClipPosition = /*glsl*/`return applyCamera(getDataPosition());`
+    const getPointSize = /*glsl*/`return mix(2.0,6.0,isHovered());` // todo!
+    // todo - use config options!


we are missing some features here - configurable point size,
and easily-configurable depth behavior

froyo-np · 2026-01-28T17:09:51Z

packages/scatterbrain/src/shader.ts

+    // produce an object that can be used to set up some internal config of the shader that would
+    // do the visualization
+    const { dataset, categoricalFilters, quantitativeFilters, colorBy, mode } = settings;
+    console.log('cat filters...', categoricalFilters)


froyo-np · 2026-01-28T17:10:17Z

packages/scatterbrain/src/types.ts

@@ -0,0 +1,88 @@
+// lets get a renderer up and rolling
+// then add features from there...


remove commentary

froyo-np · 2026-01-28T17:11:11Z

packages/scatterbrain/demo.ts

+import { ShaderSettings } from "./src/shader";
+import { vec4 } from "@alleninstitute/vis-geometry";
+
+const twoGB = 1024 * 1024 * 2000;


this is a demo I made in the package itself, because it was a lot easier than doing the same in the site/examples area. Now that we're here, I would be happy to delete it.

I'm not seeing it work right now, so yeah let's either fix it or delete it!

froyo-np · 2026-01-28T17:11:46Z

packages/scatterbrain/index.html

+      height: 100%;
+    }
+  </style>
+  <script defer src="demo.ts" type="module"></script>


this is the entrypoint for the "non-example" demo

froyo-np · 2026-01-28T17:13:09Z

packages/scatterbrain/readme.md

+multiple gene coloring / filtering
+slide-view / regular view
+
+to this end, we are gonna stick to the Renderer<> interface as given in packages/core/src/abstract/types.ts. We've seen that work with both the somewhat Ill-fated renderServer, as well as the new shared-cache


well - I didnt follow the advice on this line, but I think thats ok. update this commentary though!

chrisj · 2026-01-30T14:21:04Z

packages/scatterbrain/src/renderer.ts

+    }
+    // calling a texture as a function is REGL shorthand for total re-init of this texture, capable of resizing if needed
+    // warning - this is not likely to be fast
+    texture({ data, width: columns, height: rows });


is more performance here potentially needed?

I suspect not - changing filtering is a user action, so a few milliseconds of upload time is reasonable

chrisj · 2026-01-30T15:58:04Z

packages/scatterbrain/src/shader.ts

+function rangeFilterExpression(qColumns: readonly string[]) {
+    return qColumns.map(attrib =>/*glsl*/`within(${attrib},${rangeFor(attrib)})`).join(' * ')
+}
+function categoricalFilterExpression(cColumns: readonly string[], tableSize: vec2, tableName: string) {


I would be more verbose with the naming of qColumns and cColumns

chrisj · 2026-02-02T14:55:19Z

packages/scatterbrain/src/shader.ts

+    const categories = keys(categoricalFilters).toSorted()
+    const numCategories = categories.length;
+    const longest = reduce(keys(categoricalFilters), (highest, cur) => Math.max(highest, categoricalFilters[cur]), 0)
+    const qAttrs = reduce(quantitativeFilters.toSorted(), (acc, cur, i) => ({ ...acc, [cur]: `MEASURE_${i.toFixed(0)}` }), colorBy.kind === 'metadata' ? {} : { [colorBy.column]: 'COLOR_BY_MEASURE' } as Record<string, string>);


One alternative is:

const qAttrs: Record<string, string> = { ...(colorBy.kind === 'metadata' ? {} : { [colorBy.column]: 'COLOR_BY_MEASURE' }), ...Object.fromEntries(quantitativeFilters.toSorted().map((x, i) => [x,`MEASURE_${i.toFixed(0)}`])), };

I find it easier to read but I'm not as used to reduce

hmmm, fair enough - I'll see if I can improve on the legibility of this area a bit

chrisj · 2026-02-02T15:01:56Z

site/src/examples/scatterbrain/scatterbrain/renderer.ts

+        },
+        isValue: (v): v is Content => {
+            // TODO!!
+            // a problem here - unless we do some pre-cooking of shaders... the set of attrs we pass to the shader


I don't exactly follow the problem but this might be a good reason to follow a pattern similar to ShaderBuilder as I mentioned above. Neuroglancer is designed to generate the simplest shader necessary based on the user settings and we don't want to slow down the rendering in any way when wanting to also optionally support a more expensive setting.

lanesawyer

Haven't dug into the meat of it yet but have a few comments so far

lanesawyer · 2026-01-29T01:00:50Z

packages/geometry/src/spatialIndexing/tree.ts

+    while (frontier.length > 0) {
+        const cur = frontier.shift();
+        if (cur === undefined) {
+            // TODO: Consider logging a warning or error here, as this should never happen,


Hm. Maybe we make core a peerDependency of geometry and use the peerDependenciesMeta` to mark it as optional, and then write a bit of defensive code by checking if the logger exists before using it?

I hadn't know about the meta field until I tried to look for stuff to solve this issue just now, it's nice! Wish you could list why or what features are not going to work without it though...
https://docs.npmjs.com/cli/v10/configuring-npm/package-json?v=true#peerdependenciesmeta

ok - so I stared at this for a bit, and I'm convinced this todo is pretty clearly impossible to hit, so I dont think its worth making any changes now

I'll update the comment and remove the todo

lanesawyer · 2026-01-29T01:03:52Z

packages/geometry/package.json

        "type": "git",
        "url": "https://github.com/AllenInstitute/vis.git"
    },
+    "devDependencies": {


I don't think these changes are needed, it should pull from the root package.json for all of these. I wouldn't expect any changes to this file for the addition of the new BFS method.

yes, these are probably leftovers from an experiment

lanesawyer · 2026-01-29T01:22:29Z

packages/scatterbrain/package.json

+        "type": "git",
+        "url": "https://github.com/AllenInstitute/vis.git"
+    },
+    "devDependencies": {


These will all be provided by the workspace package.json, can remove the whole devDependencies section

lanesawyer · 2026-01-29T01:32:29Z

packages/scatterbrain/package.json

+        "@types/lodash": "4.17.21",
+        "parcel": "2.15.4",
+        "typescript": "5.9.3",
+        "vite": "7.3.0",


Although you'd actually need vite in here if you keep the demo in packages/scatterbrain around

lets remove it

lanesawyer · 2026-01-29T01:33:24Z

packages/scatterbrain/demo.ts

+import { ShaderSettings } from "./src/shader";
+import { vec4 } from "@alleninstitute/vis-geometry";
+
+const twoGB = 1024 * 1024 * 2000;


I'm not seeing it work right now, so yeah let's either fix it or delete it!

lanesawyer · 2026-01-29T01:34:27Z

site/src/examples/scatterbrain/demo.tsx

+            // make up random colors for the coloring, and add random filtering
+
+            setCategoricalLookupTableValues(categories, lookup)
+            const render = buildScatterbrainRenderFn(regl as any, {...settings,dataset});


yeahhhh the any is very weird. couldn't find any immediate fixes though. but it makes me nervous that there will be type issues when pulled into bkp-client so it would be nice to get this figured out

this is the same issue that is 'resolved' via the // @ ts-expect-error issues elsewhere. I'll change this instance to match those

lanesawyer · 2026-01-29T01:36:27Z

packages/scatterbrain/readme.md

We should probably pull these into the site. Not all packages have a README.md right now though. No need to do it in this PR, I'm just commenting to cement the idea better in my brain for later!

lanesawyer · 2026-01-29T01:38:04Z

packages/scatterbrain/package.json

+{
+    "name": "@alleninstitute/vis-scatterbrain",
+    "version": "0.0.1",
+    "contributors": [


Now here's a tough one. should contributors just be you, since you committed all the code in this PR? or should we figure out who helped write whatever you copy/pasted from the bkp-client?

… from geometry, fix commentary about a bit of dead code

lanesawyer

Looking good, demo still working just fine, couple a comments here and there for ya'

lanesawyer · 2026-02-04T01:15:00Z

packages/scatterbrain/src/renderer.ts

+            return proms;
+        },
+        isValue: (v): v is Content => {
+            // TODO


TODO for now or later?

lanesawyer · 2026-02-04T01:15:54Z

packages/scatterbrain/src/renderer.ts

+    return client;
+}
+
+type State = ShaderSettings & {


I'm not seeing this type used anywhere

lanesawyer · 2026-02-04T01:25:01Z

packages/scatterbrain/src/renderer.ts

+    texture: REGL.Texture2D
+) {
+    const { category, row, color, filteredIn } = update;
+    const col = categories.toSorted().indexOf(category)


I'm trying to figure out if the toSorted is for performance reasons, since maybe if we sort then the index gets hit soon, or if toSorted is to match the ordering elsewhere (like setCategoricalLookupTableValues, maybe, or in shader.ts? Or is there another reason I'm not thinking of?

If it's the former, I'm not totally convinced it would be faster.

If it's the latter, that feels a little dangerous to rely on the sort order of this being the same as it's sort order elsewhere and it would be nice to get that category access coming from a common place in the code so we don't introduce bugs in the future in case we need to access categories in a sorted manner again or tweak one of the places it's being sorted.

so what is happening here - first a little background:
the shader uses a lookup table to power categorical filtering and coloring - each column has to have a hard-coded index in the shader. that column's index in the table, and thus in the generated shader, corresponds to a vertex attribute that is used as the key to lookup color or filtering status of a point for that property.

what that means is that the set of columns used to create the texture must match the columns used to generate the shader - to my mind, this is made a bit more simple and stable by sorting the columns by name.

regarding performance - we tend to see just a few columns in use at a time - sorting a list of a few dozen strings repeatedly shouldn't be an issue. having the sorting done within the renderer / shader generator modules lets the external API simply pass a record of categories without having to bother with the sorting.

to your point - we could re-organize this so that the use of the lookup table is opaque... but then we'd have to diff the columns and handle the memory for the lookup table in a more conservative manner to avoid leaks... might be a good idea though, I'm on the fence

Makes sense, yeah we'd have to refactor it quite heavily to centralize the sorting/lookup logic. I'm confident in our ability to not break it as-is though, so I'm fine leaving it this way!

lanesawyer · 2026-02-04T01:25:41Z

packages/scatterbrain/src/renderer.ts

+    const { category, row, color, filteredIn } = update;
+    const col = categories.toSorted().indexOf(category)
+    if (texture.width <= col || texture.height <= row || row < 0 || col < 0) {
+        // todo - it might be better to let regl throw the same error... think about it


TODO here, are you saying regl will throw the error anyway if we get it wrong?

yes I think so

lanesawyer · 2026-02-04T01:26:38Z

packages/scatterbrain/src/shader.test.ts

lanesawyer · 2026-02-04T01:27:32Z

packages/scatterbrain/src/shader.ts

+import { Box2D, type vec4, type box2D, type Interval, type vec2 } from "@alleninstitute/vis-geometry";
+import { type Cacheable, type CachedVertexBuffer } from "@alleninstitute/vis-core";
+
+// the set of columns and what to do with them can vary


I'm okay if this is here for now. I've been exploring producing documentation automatically from JSDoc comments, or having us enhance our README docs and get those deployed on the vis website (or maybe both options!).

Maybe turn into a JS Doc comment attached to the VBO class though?

lanesawyer

Demo still works and new code changes look good. Yay for zod! That's slowly infecting all our codebases, but it's great 😁

I called out one spot that will fix the CI jobs, once those are passing, let's get this merged!

lanesawyer · 2026-02-10T18:41:53Z

site/src/content/docs/examples/scatterbrain.mdx

+tableOfContents: false
+---
+
+import { ScatterBrainDemo } from '../../../examples/Scatterbrain/demo.tsx';


All the CI jobs are failing, I think you just need to not capitalize Scatterbrain here

lanesawyer · 2026-02-10T18:45:32Z

site/src/examples/scatterbrain/demo.tsx

+import type { vec2, vec4 } from '@alleninstitute/vis-geometry';
+import { SharedCacheContext, SharedCacheProvider } from '../common/react/priority-cache-provider';
+import { useContext, useEffect, useRef, useState } from 'react';
+import { buildScatterbrainRenderFn, buildScatterbrainCacheClient, loadScatterbrainDataset, setCategoricalLookupTableValues, type Dataset, type ShaderSettings } from '@alleninstitute/vis-scatterbrain';


buildScatterbrainCacheClient is unused

site/src/examples/scatterbrain/demo.tsx

chrisj · 2026-02-10T22:26:28Z

packages/geometry/src/spatialIndexing/tree.ts

@@ -16,8 +16,7 @@ export function visitBFS<Tree>(
    while (frontier.length > 0) {
        const cur = frontier.shift();


I would use a ! assertion here but I understand if you prefer to avoid it

I like it - I tend to not use those, having stepped on that rake a few times, but its clearly reasonable here

Co-authored-by: Lane Sawyer <lane.sawyer@alleninstitute.org>

…itute/vis into noah/scatterbrain-port

lanesawyer

I'm happy with it, let's ship!

froyo-np added 15 commits November 19, 2025 13:04

move scatterbrain into vis for realsies - not just as a quick minimal…

31c6484

… demo. initial thoughts

add visitBFSmaybe, a variant of bfs tree-traversal that lets the visi…

214e441

…tor decide to continue traversal (or not) in addition to whatever effectful thing it would normally do

thinking about impl of scatterbrain in vis - might back some of this …

89cd1f9

…out and start smaller...

get the bare bones basics down for the cache client... start to think…

2be308d

… about how to generate these shaders in a less hard to modify way

thinking about ways to deal with shader generation...

0c70747

finally gettin somewhere

d487806

make it actually work, delete false-start stuff

19d734c

a list of remaning shader work...

09ec618

basic filtering exprs.

cb424a2

cleanup some of the simpler features of the shader

02711fc

much cleanup

5bc468d

commentary

127da48

move files out of silly folder name

66e47d6

export needed utils to build an example in the examples area

ddd0960

render scatterbrain demo in examples area

f3049fc

froyo-np commented Jan 28, 2026

View reviewed changes

packages/scatterbrain/src/renderer.ts Outdated

}

}

/* TODO features:

Copy link

Collaborator Author

froyo-np Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this comment

froyo-np commented Jan 28, 2026

View reviewed changes

chrisj reviewed Feb 2, 2026

View reviewed changes

lanesawyer reviewed Feb 2, 2026

View reviewed changes

froyo-np added 5 commits February 3, 2026 14:28

some more PR cleanup feedback, mostly removing stale comments or files.

3a6d53d

use ts-expect-error instead of casting to any, remove devDependencies…

c4f529d

… from geometry, fix commentary about a bit of dead code

merge conflict, pnpm install

73e9c56

better words

e3281fb

package lock updates (lodash types)

b0818de

lanesawyer reviewed Feb 4, 2026

View reviewed changes

froyo-np added 4 commits February 6, 2026 11:09

fix up important todo

fe037fe

replace a placeholder threshold

9a00456

add zod schema for loading dataset metadata

07826a9

a little cleanup

07ba25e

froyo-np marked this pull request as ready for review February 10, 2026 18:05

froyo-np requested a review from a team as a code owner February 10, 2026 18:05

froyo-np requested review from Jarbuckle and lanesawyer February 10, 2026 18:05

lanesawyer reviewed Feb 10, 2026

View reviewed changes

chrisj reviewed Feb 10, 2026

View reviewed changes

froyo-np and others added 8 commits February 11, 2026 08:59

Update site/src/examples/scatterbrain/demo.tsx

1a2dc74

Co-authored-by: Lane Sawyer <lane.sawyer@alleninstitute.org>

minor changes

4785102

format

cb5a519

lint

5e6eb8e

Merge branch 'noah/scatterbrain-port' of https://github.com/AllenInst…

3f56441

…itute/vis into noah/scatterbrain-port

fmt again

656cb27

lint

050e730

Merge branch 'main' into noah/scatterbrain-port

546f49b

froyo-np enabled auto-merge (squash) February 11, 2026 22:36

lanesawyer approved these changes Feb 11, 2026

View reviewed changes

froyo-np merged commit fb2ddce into main Feb 11, 2026
5 checks passed

froyo-np deleted the noah/scatterbrain-port branch February 11, 2026 22:38

		type Content = Record<string, VBO>


		export function buildScatterbrainCacheClient(regl: REGL.Regl, cache: SharedPriorityCache, onDataArrived: () => void) {

		// get without a great deal of extra performance cost


		// scatterbrain does scatterplot rendering

		@@ -0,0 +1,88 @@
		// lets get a renderer up and rolling
		// then add features from there...

		@@ -16,8 +16,7 @@ export function visitBFS<Tree>(
		while (frontier.length > 0) {
		const cur = frontier.shift();

Conversation

froyo-np commented Jan 27, 2026 • edited by chrisj Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

A Not quite feature complete implementation of basic scatterbrain scatterplot rendering

What

How

Screenshots

PR Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lanesawyer left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

froyo-np commented Jan 27, 2026 •

edited by chrisj

Loading

lanesawyer Jan 29, 2026 •

edited

Loading