-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
statmemprof is absent in OCaml 5.0 #11911
Comments
I would particularly appreciate input on the following questions:
|
My understanding is a bit different: Statmemprof required a very careful polishing of the runtime system to be able to tell, at each allocation point, how to handle the trigger of a statmemprof sampling event (can we run callbacks at this point and, if not, when is the next safepoint to run one?). This careful engineering, which happened in upstream 4.x after the forking of the Multicore codebase, was never back-ported into the Multicore OCaml branch because of lack of expertise (or prioritization) from the Multicore developers, and as a result it disappeared from the 5.x runtime. It would need to be restored to get Statmemprof back, even before asking about the behavior in multi-domain contexts. (@jhjourdan is the original author of statmemprof; @stedolan and @gadmm reviewed it carefully and could probably assist in those questions; related issues/PRs from @gadmm: #10915, #11057 )
Yes, because it forces you to address the issues above.
Whatever is simple and is compatible with good performance. A first proposal:
(The idea of "context" that you mention could come later, if there is need.) |
Here's my understanding of the history. Since Multicore OCaml was being developed as a fork, we typically caught up with mainline OCaml by rebasing the changes onto the Multicore OCaml branch one release at a time. Multicore OCaml typically remained a few releases behind the latest OCaml release. Hence, we could see the upstream developments several releases ahead of where Multicore OCaml was. Separately, in parallel to Multicore OCaml, Statmemprof was being developed and its API was refined over multiple OCaml releases. It was painful for the Multicore OCaml team to understand and integrate Statmemprof changes in a particular OCaml version with the knowledge that the subsequent version would change the statmemprof API (for the better). So we decided to not integrate Statmemprof until the API had stabilised (which it has now). @sadiqj is also a good person to comment on this issue since he reviewed a number of PRs by @gadmm which improved how OCaml 5.0 treats asynchronous actions. The work by @gadmm gets (most/all?) of the changes that were introduced to treat asynchronous actions in such a way that it is compatible with what Statmemprof requires. |
For the record, memprof.c on trunk is very nearly identical to 4.14, apart from the fact that it's #ifdeffed-out. There are literally three different lines (passing |
But statmemprof relies on a lot of small changes in the rest of the runtime to work well. See the changelog entries around the Statmemprof merge: Main PRs:
Buildup work:
Follow-ups:
As mentioned by @kayceesrk, it may be that the 5.x runtime is now in a good-enough state wrt. Statmemprof requirements (@gadmm would know), but this would be something to check. |
Thanks for all of this, these are exactly the sorts of pointers I need. The innards of memprof.c are also not at all thread-safe (in the C sense): there's a lot of global shared state all the way down to the random number generators. It's a SMOP to put all of that into some sort of thread-safe object, created on demand. It seems likely to me that this object should be per-domain, linked from the domain state (analogously to, say, the caml_intern_state). |
I would distinguish between:
(I don't remember the use-cases for "pausing" and restarting memprof sampling, it may be that those are used to protect critical sections, and therefore should be fast and per-domain as well.) |
This should be easy to do, at the time I contemplated to do exactly that on top of #11307.
We needed to go a bit further than 4 due to some subtleties (e.g. replacement of
I am not worried about needing support for user schedulers, etc. One example is memprof-limits, which attaches per-task context to callbacks. This is done by the library without need for special support from memprof. Concretely, I have a thread-safe map from Thread.id to per-task context (several tasks can be nested per thread; the thread id is enough to find the innermost context). I think that profilers, likewise, might want to be aware of user threads. If I want to support a notion of task tied to a user-side thread abstraction, I just need to make it parametric over the notion of thread (concretely have an interface that is parametric on a particular notion of thread id or thread-local storage). If possible it is better to avoid to make memprof dependent on the user-side threading abstractions; the profiler might need to worry about this but not memprof itself. Another point to consider is if a computation (continuation) is sent from one domain to another. I think it is much simpler to have the memprof status (started/stopped) and starting parameters be global, than having the profiler negociate with the user threading library how it should be started on every domain it manages. Another aspect of interaction with effects is whether one is allowed to perform effects inside memprof callbacks. Here, note that no effect can escape from a callback for the simple reason that the capture of C stack frames is not supported ATM (and it is unclear that it will be). Thus any effect will have to be handled inside the callback, which makes sense from a computational point of view. As for changing the parameters per-domain, for memprof-limits I do not need it but it does not bother me either (I would just not use it). On the other hand, a useful property of memprof is that it guarantees that delayed callbacks run on the right thread; this should be easily preserved for domains. This might not be doable for user threads. I have to think more about it, but it might be a useful information to have, whether a callback has been delayed or not, because the latter guarantees that it runs on the sampled thread. (In this example, directly querying the user-thread id from memprof at moment of sampling is not doable, because it might be impossible to run any OCaml code for the same reason that callbacks are delayed). There is a third option to solve this issue but this needs its own discussion thread.
AFAIR it was still marked as "experimental" and susceptible to change, but it would be good to stabilize it. (It would be good to have the change proposed at #9267; the discussion there is a bit long-winded but it is mostly subsumed by my papers and talks about asynchronous exceptions and progress with Stephen and Leo on this topic, so it could benefit from starting the discussion anew. Stephen told me verbally that he agrees with the change so I hope I can have positive reviews without too much discussion effort from the community.)
With moderate priority, I'd avoid a stop-the-world pause if possible.
Memprof wants to avoid sampling its own allocations (edit: those done inside memprof callbacks). Thus per domain (edit: this is already per-thread; furthermore one should not expect to have to make it aware of user threads because one cannot perform outside effects inside callbacks). |
I was thinking the simplest extension would be to keep a global rate but that the allocating domain's id would be available in the allocation callbacks.
I'm not entirely convinced we'd save that much time overall with single-domain. Happy to discuss in-front of a whiteboard though. |
And such a global/default rate can be made customizable per-domain later without breaking the API. I am not sure we need the allocating domain id. The current statmemprof implementation already guarantees that the callback runs on the same thread, so one just has to call Domain.self. |
I tend to agree. Actually, I think that trying to isolate domains is rather an additional feature than a simplification. To me, what this means is simply that we have a guarantee that the various callbacks for a block are all called in the allocating domain. This is clearly desirable in some use cases, but I would say that it adds some complexity, because once a block is in the major, the GC does not give us any guarantee in the general case on the deallocating domain for a given block (right?). Anyway, I'm happy to discuss and give some help. |
I do not think that we need more guarantees than currently exists w.r.t. threads. We already have some guarantees right? So it is probably more work to remove those guarantees. I do not think we need constraints on which domain calls a deallocating callback, this does not make much sense to me. I agree that simply reactivating memprof for a single domain is an intermediate step that can be done in parallel of discussions about the right multicore API. |
I think it does make sense, because client code will likely store information about tracked blocks in domain-local data structures. |
My non-expert understanding of the 5.x major GC is that each domain sweeps its domain-local shared heap, in which it promoted its domain-local minor heap. So it may be the case that we can call the deallocating callback on the allocating domain. |
Agree with this. OCaml 5.0 does not have a primitive notion of user-level schedulers. The language supports delimited continuations, which may be used for building lots of things including user-level threading. In the first iteration, it would be prudent to focus only on domains. What I suspect may happen is that once we have a good understanding of how things work with multiple domains, we may add some primitive features to the compiler so that support for user-level threads can be built outside the compiler. A good example of this is how we've developed runtime events. Runtime events don't understand user-level threading but are aware of domains. Recently, support for custom events was merged: #11474, which is used by the meio observability tool for the eio library, which has a user-level scheduler. I am hoping we can do something similar for Statmemprof. |
This is correct and why we can't guarantee the callbacks will always happen in the same domain. Adopted pools from terminated domains can be swept by a different domain eventually. |
But pools are adopted explicitly on domain shutdown, we could call another callback on sampled values there that would arrange ownership transition if necessary. (This could make adoption more costly, but it is not a performance-critical operation.) |
Pools are moved to the global available-for-adoption lists on shutdown but there isn't an explicit handoff to a new domain. That happens when another domain needs a new pool of the appropriate size and grabs one from the available-for-adoption lists (or we get to the end of a major cycle and there's still pools waiting for adoption). In both of those two cases the original domain the pool came from has been terminated. |
We could call a callback on the terminating domain when the sampled block is about to be orphaned, which would be in charge of moving the user-implemented tracking data from the domain-local pool to a global resource. This could be rather cheap if we track, per-pool, whether there exists one sampled block that has such a call-me-on-orphaning callback (common case: no block has this). |
(We could also have a callback on readoption, or not.) |
Doesn't this amount to writing in marble stone implementation details of the GC? These could change in the future. What are the alternatives for storing information on blocks? Is there a simple thread-safe data structure that could take advantage of the bias towards the current domain? |
Another API-changing aspect is that we really would like to sample stack allocations, now that it is treated like dynamically allocated memory. |
I see that the backtrace modules have diverged somewhat between 4 and 5, so memprof will also have to adapt to that. |
Agree with @gadmm here, we end up exposing a lot of the internals of the GC really. We'd end up with similar things needed for compaction too. Here's a potentially controversial proposal. Instead of keeping domain-affinity for callbacks we drop it and go further. We include the allocating domain in the tracking data that memprof keeps and then only a single domain (the one that registers callbacks) receives them. This has a couple of benefits. It doesn't require domains doing actual useful work to stop what they're doing and run callbacks and you can get away with using fast non-thread-safe data structures in the callbacks. The downside is you are limited by the throughput of a single domain's callback processing but I can't think of a realistic usecase where you'd hit that (you would reduce the sampling rate). |
I don't see the problem with exposing the detail that the GC uses per-domain major heaps. If users were expecting their statmemprof code to keep working flawlessly in all future OCaml versions, they must be seriously disappointed by now! (Also if we have an 'orphaning' callback, we can just stop calling it if the GC changes and does not orphan anymore, and things will keep working without changing the API.) |
@sadiqj I remember being told that memprof tried hard to run the callback immediately at the allocation point if possible. In addition this design would make it impossible for memprof-limits to raise an exception in the right thread. I think we may be overthinking this issue. I think there's a solution on the side of the user. You can use a domain-local single-threaded data-structure at first and then use @gasche There is no reason to not be as backwards-compatible as possible. I wonder how much the current API is designed for this. For instance, if the @NickBarnes It will also be necessary to add a mechanism to adopt orphaned pending memprof callbacks, like there is for the finalizer queue. |
Just to provide a use-case here: yes, it would be very useful if OCaml 5+ supported statmemprof even if restricted just to a single domain. This would've been a lot easier if statmemprof worked on OCaml 5+, even if just on a single domain. |
Fixed (finally!) by #12923. |
The statistical memory profiler, also known as "statmemprof", is currently stubbed-out. But it was tremendously useful in OCaml 4. This high-level issue exists to discuss, reason through, and then track the work to resurrect it, which I hope to complete during 2023Q1. There may be several subsidiary issues covering parts of that work.
Statmemprof was stubbed-out because of unanswered questions about its behaviour in the multicore world, both at the implementation level and for the API: what semantics are desired or even possible in the presence of multiple domains? We have to identify and answer those questions.
For reference:
runtime/memprof.c
;stdlib/gc.ml[i]
.I'm starting work on this by reading through the existing code (in 4.14) in fine detail, documenting it to my own satisfaction, with the intention of then putting together a proposal for multicore semantics and notes on implementing them. All of that work will end up in comments on this issue and/or in source code comments in PR(s) linked from here.
The text was updated successfully, but these errors were encountered: