Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use custom (de)serialization for points/selection #11

Merged
merged 18 commits into from
Jun 30, 2021

Conversation

manzt
Copy link
Collaborator

@manzt manzt commented Jun 15, 2021

Changes points and selection to use different serde (rather than traitlets List). Currently assumes that points and selection will be 2D numpy arrays.

TODO:

  • implement custom JS deserialization
  • implement custom JS serialization

NOTE: regl-scatterplot requires data to be in a nested/interleaved structure: [[x0, y0, ...], [x1, y1, ...]]. I'm not sure how feasible it would be, but we could likely improve the overhead of data-transfer further if we could pass columnar data to scatterplot.draw(), e.g.:

scatterplot.draw({
    x: new Float32Array(...),
    y: new Float32Array(...),
    category: new Float32Array(...),
    value: new Float32Array(...),
});

In python, the underlying points/selection arrays could be column-major order (order="F"), which means that we could de-serialize the memoryview on the client into individual columnar typed arrays.

@manzt manzt changed the base branch from master to new-python-api June 15, 2021 16:53
@manzt manzt changed the title Manzt/bytes Use custom (de)serialization for points/selection Jun 15, 2021
@manzt manzt force-pushed the manzt/bytes branch 4 times, most recently from 8341eb0 to 56f717d Compare June 15, 2021 18:42
@manzt manzt marked this pull request as ready for review June 17, 2021 19:05
@manzt manzt force-pushed the manzt/bytes branch 3 times, most recently from 0547950 to 2e68c4a Compare June 28, 2021 22:26
@flekschas flekschas closed this Jun 29, 2021
@flekschas flekschas deleted the branch flekschas:master June 29, 2021 13:29
@flekschas flekschas reopened this Jun 29, 2021
@flekschas flekschas changed the base branch from new-python-api to master June 29, 2021 13:45
@flekschas flekschas self-requested a review June 29, 2021 13:45
@flekschas flekschas added the enhancement New feature or request label Jun 29, 2021
Copy link
Owner

@flekschas flekschas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks and works well! Thanks a lot 🎉

@flekschas flekschas merged commit 57ff4c9 into flekschas:master Jun 30, 2021
@manzt manzt deleted the manzt/bytes branch June 30, 2021 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants