-
Notifications
You must be signed in to change notification settings - Fork 55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
3.0 #101
Closed
Closed
3.0 #101
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bmschmidt
force-pushed
the
tileproxy-is-all-you-need
branch
from
February 13, 2024 03:12
869b646
to
6e18867
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Major refactor.
3.0.0 includes a number of pent-up breaking changes. The underlying motivation for many of these is to allow library to now fully pass typescript compilation tests, with all the stability benefits that provides.
Breaking changes:
The library is now structured as named exports,
rather than a single default export. Instead of
a typical first line will be
This allows the export of several useful types for advanced functions in scatterplots we've found useful at Nomic. The initial set of exported items are
{Dataset, Bitmask, Scatterplot}
.The distinction between QuadTile and ArrowTile
has been eliminated in favor of Tile, and with it the need to provide
generics around them through the system. Similarly, QuadTileDataset and ArrowDataset are both removed in favor of Dataset.
Instead, the TileProxy object is used to provide a wrapper than can turn anything into a
dataset. Although datasets are presumed to be quadtiles right now, formally they can be any
any collection of arrow batches structured as a tree. (This is increasingly how I've come to think of the data parts of deepscatter: as a system for navigating dataframes that consist of trees rather than of linear lists of points.)
Deepscatter no longer accepts strings as direct
arguments to
Scatterplot.plotAPI
in places where they were previously cast to functionsas lambdas, because linters rightfully get crazy mad about the unsafe use of
eval
. Ifyou want to use deepscatter in scrollytelling
contexts where definining functions as strings inside json is convenient (I still will do this myself in static sites) you must turn them
into functions before passing them into deepscatter.
Shortcuts for passing
position
andposition0
rather than naming thex
andy
dimensions explicitly have been removed.The behavior of categorical scales in certain circumstances has been tightened; it is possible, as a result, that places where it previously possible to treat categorical scales as numbers (referring to the underlying ints) will no longer work. I am not aware of specific such issues at the moment, and will act responsively to address any issues.
Dataset
andTile
objects can now be instantiated with amanifest
that allows listing all the tiles in a dataset. When passed, this allows a dataset to instantiate all tiles at creation time without actually loading any data. This represents a major change for any code that access theTile.record_batch
attributes, because they may now error on well-formed tiles since the presence of data is no longer necessary for something to be aTile
. Additionally, theTile.ready
promise has been retired; instead, to check if necessary data exists for a dataset, you must explicitly check ifTile.hasLoadedColumn('foo')
.(Another way of expressing this change is that where previously there was a 'primary record batch' and 'sidecar batches', in version 3.0 of deepscatter this distinction is much less important; it is possible, for example, to draw a scatterplot without loading the
x
andy
columns if other columns are passed toencoding.x
andencoding.y
.)The tile prioritization rules which previously applied to core tiles in a dataset now apply to all sidecars as well.
It is possible to aggressively load any columns to any depth in the dataset without loading other data using 'Dataset.spawnDownloads()' and
Dataset.runDownloads()
.The Dataset object is now more independent of the scatterplot, to the point that it can independently run in a non-browser environment like Node. See the unit tests for an example of this.