-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Optimize Plot #18
Comments
Update: confirmed, displaying a small subset of my data (~2%, not more than a few hundred kb of floats) causes frame rate to become unusable on release + LTO-fat build, like 1-5hz.
After some small amount of brain-using, I think it is actually exactly because of the immediate mode architecture that there is no reason for this API to take ownership of the data, or even to require As for background work, all I've done for now is set an |
Ok after reviewing the repo locally, the story is a little more complicated. The original thinking was that because everything is immediate mode, there can never be any non-trivial borrowing patterns involved, so it would be safe to throw in lifetime bounds and references inside I think this may actually be true, but I'm unsure. This is going to depend on when and how long the ref can possibly be asked to live. I think because There are additional inefficiencies present here which have become obvious b.c. my data is so large. I was very surprised to come across so many (re)allocations in the hot paths of Take for example the reconstruction of A solution I can think of which basically solves this problem across the board is to modify E.g. when a mesh is constructed, the most recently dropped Would you accept a PR of this type? |
Before making assumptions about what is the slow part of the code, run an profiler and find out! You may be surprised. Each point in a line that the Plot has to show runs through several steps: transform to screen space, tessellated as a line to a If you make an optimization PR, make sure you create an e2e benchmark first (in If you are showing 50M points in one plot at the time (and that are all within the view bounds) you will never get great performance. You should consider down-sampling the data first before handing it to |
To get a very rough feel for where the milliseconds are being spent you can use |
At the very least, is it possible we could supply Plot with our own allocator? That would go some way towards this goal, but hopefully shouldn't be too intrusive on the codebase. |
You can start by trying out mimalloc and see how that affects performance |
Wish that worked on wasm though :( |
I tried to address this in emilk/egui#1816 but couldn't find a way to use non- |
We really need this, or maybe we can have some custom iterators to iterate through dense bulk arrays to avoid copying data. |
Are you sure? Do you have a profiler dump to share? |
If I am using a plot, I will not deal with some small static data but with dynamic data containing millions of points. |
@chengts95 your formatting on your comment isn't correct btw |
Sorry if this is slightly offtopic, but i ran into the "i need to plot a few million points with zoom and everything" issue as well and i have a solution that works, for my case, reasonably well now. ** in release builds anyway ** https://gist.github.com/mkalte666/a4944361a80c1014d098c0adb15ec9dd The essential idea is that every time the plot bounds are moved around, i grab from the big array My use case needs the copy, as the original data source might disappear and lifetimes are a mess otherwise anyway, however you could probably realise this with a seperatly-kept reference to the original data. Or even making the functions a trait, the cache something seperate, so it works on more or less arbitrary arrays. You would also replace the single downsampling function i use (which very explicitly grabs absolute maximum of the sampled range) and do something proper (which, however, might take too long?) or just throw away other samples. Anyway, this is what i use for myself for now, and it works. I hope this helps others with the issue, and maybe can serve as inspiration for others with this issue. |
For what it's worth, here is how I handled plotting millions of points in my app: https://gitlab.com/ygor.souza/poli/-/blob/583a3f0ba2d6b0b195db8adef9d7fb70f88eeee6/src/tool/dynamical_system_simulator.rs#L378-399 It basically samples the data vector at the desired resolution using PlotPoints::from_explicit_callback. The I limited the size to 1M points in my app because I'm also running the actual simulation in the UI thread for now, and I don't want it to lock the UI for more than a couple of seconds. But if I change it to 10M, I can still scroll smoothly through the results once the simulation finishes. |
Adding another comment here, as my method has evolved since then. Cant share the whole thing because i made it quite custom for the data im working with, but i can throw you my mipmap functions. Essentially i am now taking the hit of using twice the storage, and essentially use a 1d mipmap to sample the data. http://www.number-none.com/product/Mipmapping,%20Part%201/index.html Mip-mapping for graphics sometimes just uses gamma correction to combat this, but this isn't image data thats normalized between 0..1, so shrug https://gist.github.com/mkalte666/9797eb7e0eb31cdbe208fce994d3e45b Things you might wanna do
Hope this helps people, have a good day! |
Hi I am currently looking at optimizing the plotting of many data points. I have done profiling. There are a few problem as I see it with the current code that absolutely destroy performance on many data points
I think these items are low hanging and easy to address problems and I will add a PR for that in the near future.
Additionally, the compute takes place on a single core. Maybe it can be parallelized efficiently for native instances? |
Hi all who are interested in this, I have implemented plotting of borrowed data: Update: My pr is closed and won't be merged. More info here: emilk/egui#3849 |
* Part of https://github.com/emilk/egui/issues/1485 This adds a `rayon` feature to `epaint` and `egui` to parallelize tessellation of large shapes, such as high-resolution plot lines.
Hi!
Cloning
I'm trying to use the
Plot
andLine
API.This requires that I consume my data by value, I'd much rather keep a cached copy & provide a reference.
My plot data is fairly large (1M+ elements, sometimes 50M).
If this is not possible, can I add the API to accept something which can be
.as_ref()
to a&[(f64, f64)]
?I suspect the by-value nature of the API has something to do with the immediate mode design of
egui
... however, in cases like this it doesn't seem to actually make sense if it is driven only be design purity, ykwim? Your thoughts on this / advice would help me.Background Work
I am running massive simulations (~hours completion time) & interactively showing the results over time is the goal. I'd like to be able to do this in the background as mentioned. It is mentioned a few times in the documentation that long-running tasks should be deferred to background tasks in an async manner. I agree. I'm lost on how to accomplish this. I can think of a few clumsy ways, but I want to know if there is a proper way to do this that makes things easy and performant in
egui
.Thank you!
The text was updated successfully, but these errors were encountered: