Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to serialize massive JSON without blocking event loop #37

Closed
simonw opened this issue Oct 25, 2017 · 2 comments
Closed

Ability to serialize massive JSON without blocking event loop #37

simonw opened this issue Oct 25, 2017 · 2 comments
Labels

Comments

@simonw
Copy link
Owner

simonw commented Oct 25, 2017

We run the risk of someone attempting a select statement that returns thousands of rows and hence takes several seconds just to JSON encode the response, effectively blocking the event loop and pausing all other traffic.

The Twisted community have a solution for this, can we adapt that in some way? http://as.ynchrono.us/2010/06/asynchronous-json_18.html?m=1

@simonw simonw added this to the v1 stretch goals milestone Oct 25, 2017
@simonw
Copy link
Owner Author

simonw commented Oct 25, 2017

Could this be as simple as using the iterative JSON encoder and adding a yield statement in between each chunk?

@simonw simonw modified the milestones: v1 stretch goals, v2: visualization edition Nov 14, 2017
@simonw simonw removed this from the v2: visualization edition milestone Dec 10, 2017
@simonw
Copy link
Owner Author

simonw commented May 30, 2020

I'm not going to do this: 2.5 years later I have yet to run into anything that makes me think that JSON serialization performance is worth any extra work.

@simonw simonw closed this as completed May 30, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant