Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize diff cmd (horrendous memory usage) #353

Closed
neilotoole opened this issue Jan 14, 2024 · 1 comment
Closed

Optimize diff cmd (horrendous memory usage) #353

neilotoole opened this issue Jan 14, 2024 · 1 comment
Assignees

Comments

@neilotoole
Copy link
Owner

neilotoole commented Jan 14, 2024

Is your feature request related to a problem? Please describe.
sq diff uses a shocking amount of memory. The docs describe sq diff as being in beta for this reason. It's already noted in the source code that diff needs to be optimized.

It's pretty bad (memory usage circled in magenta):

sq_diff_mem_usage_shocker

Describe the solution you'd like
There are massive optimizations that can be made to the diff code. Some preliminary work is already in the source code.

@neilotoole neilotoole changed the title Optimize diff cmd (uses horrendous amount of resources) Optimize diff cmd (horrendous memory usage) Jan 14, 2024
@neilotoole neilotoole self-assigned this Feb 16, 2024
neilotoole added a commit that referenced this issue Feb 20, 2024
@neilotoole
Copy link
Owner Author

Release v0.48.1 tackles a bunch of the diff performance issues. What was once 22GB of ram is now more like 22MB. But there's still plenty more low-hanging fruit for diff perf.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant