Zoom optimisation for Waves and matching implementation for Spectrograms

[luke-harrison-personal]

Zoom optimisation for Waves and matching implementation for Spectrograms

Short description of changes:

My goal was to allow zooming on a spectrogram to be performed faster and with no maximum width. To do this I implemented optimisations in both the wavesurfer and the spectrogram with the intention of making zooming better for users.

Removing the maximum width

I took the multicanvas approach used in the wavesurfer drawer and applied it to the spectrogram so that the maximum size limit would be circumvented. In my approach all canvases are equally sized as I found this to be easier to manage, however the multicanvas drawer remains the same.

Improving Performance

Existing canvases can have their width changed in style without redrawing/recalcuation. This is an incredibly fast way to preview a wave/spectrogram at various zoom levels. I created a zooming() function to be called when the zoom level is changed, and only calling the original zoom() function when the zoom slider is released. This approach preserved the original behavior of zoom for older implementations.

The spectrogram is still a very computationally expensive task and so generating particularly large ones takes a lot of time. I scheduled canvases on a priority so that viewable and near-viewable canvases would be rendered before others. This process can be seen when zooming out quickly after zooming in. This was applied to the wavesurfer too, although it was much less common for long waves to generate the same level of lag as the spectrogram

An example of zooming while the audio is playing:

Breaking in the external API:

None that I know of

Breaking changes in the internal API:

None that I know of

Todos/Notes:

Todos

None

Notes

Some unrelated changes were necessary as a result of the approach taken. Since the drawer width changes with the zoom, I had to remove the check in drawer.js that prevented redrawing the wave if the zoom level remained unchanged.

I changed the example pages for zoom and spectrogram to reflect the new implementation.

This implementation adds/removes no files and requires no additional libraries.

Related Issues and other PRs:

None that I know of

[coveralls]

Coverage: 79.253% (-1.9%) from 81.108% when pulling 0c550f0 on luke-harrison-personal:zoom_optimisation into 5a772c8 on katspaugh:master.

[katspaugh]

I think it's a pretty clever solution, basically stretching an already drawn waveform while a new one is being calculated. Personally not a fan how the stretched one looks but it's definitely smoother!
The delayed rendering and prioritization of canvases is also a really good idea, regardless of zooming, actually.
And finally, adding multi-canvas to the spectrogram is no question an excellent addition.
Overall, while I think this comes with a lot of extra complexity, it's a solid UX improvement. 👍

[entonbiba]

@katspaugh @thijstriemstra @luke-harrison-personal been looking into this as well lately. We can look into adding css image-rendering: pixelated; so it doesn't blur the canvas while it's stretching it. There is also the globalConpositeOperation in canvas we could potentially utilize.
https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/globalCompositeOperation

However there is also the possibility of making these peaks as SVG in realtime as an idea.

[katspaugh]

That CSS rule is worth trying 👍
SVG, unfortunately, didn't fare well in terms of performance back when we used to have it as a rendering option. Things might’ve changed. In any case, it’s outside the scope of this PR.