waveform (ui grade) from audio buffer(s)

[tehn]

if possible, make this as generic as possible for multiple use cases (different engines)

[catfact]

i'll get to this soon (at least the SC side of it)
but i don't get a clear idea from "generic"
you mean:

• engines all use the same pool of buffers? (hmm...)
• some buffers are persistent? (maybe for input/output caputre?)
• engines use shared code? (of course, i will make an abstraction for the preparation of osc blobs from buffers)
• same API for all engines? (of course, it will be a type of Poll)

[catfact]

will do this asap but just a little concerned about the computation time of rendering thumbnails of arbitrary regions from scratch on each poll (instead of building up gradually as is common). sclang has to fetch buffer data from server over OSC and render it, then send more OSC to matron. (can't really see any other reasonable way.)

would love to know some idea of the minimum number of pixels in each region would be (pixel stride) and how poor an approximation is acceptable

am assuming the region that is thumbnailed should be of arbitrary size? makes it tricky.

some options of various craziness:

• the poorest approximation: just render every Nth sample, massively downsampling, who knows if you can recognize anything from this.

• the best approximation: for each pixel stride, keep a min/max of every intervening sample. this would become untenable if the region is very large.

• something in between: find local maxima/minima surrounding every Nth sample (where "local" is some optimal width.) this is the most work. might be ok.

could pick different approximation strategies depending on region size, i guess... oof

i guess should maybe start with simplest b/c better than nothing and then we can see.

[tehn]

despite my earlier comment suggesting this issue is important, i'm ok with deprioritizing this feature at the moment. (i was high on coffee)

agreed that it seems a difficult tradeoff to figure out.

[catfact]

since this issue was created we've added the ability to create polls with arbitrary updates, triggered either from matron or from crone.

so, we should add something to BufUtil to build data blobs for some approximation of the waveform, for a given region of a buffer. then it would be easy to, say, 1) ask for the waveform of current voice region, or 2) send the waveform when a clip is loaded.