New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proposed implementation of Threads in terms of Domain and Atomic #240
Conversation
@@ -74,8 +74,10 @@ static void init_segments(void) | |||
} | |||
|
|||
/* These are termination hooks used by the systhreads library */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably not the most friendly way to remove this logic. Should properly delete this logic if it's really obsolete.
@@ -128,10 +130,12 @@ value caml_startup_common(char_os **argv, int pooling) | |||
else | |||
exe_name = caml_search_exe_in_path(exe_name); | |||
caml_init_argv(exe_name, argv); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also here.
byterun/domain.c
Outdated
@@ -1055,7 +1054,9 @@ CAMLprim value caml_ml_domain_yield_until(value t) | |||
if (ts < caml_time_counter ()) { | |||
ret = Val_int(0); /* Domain.Sync.Timeout */ | |||
break; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My recollection is that this change was necessary to address an intermittently observed deadlock failure in the threads unit tests. Unfortunately, the test for pr5325 is still failing deterministically. Not sure why. It does seem to be deadlocking on this same call to caml_plat_timedwait
.
configure
Outdated
@@ -1280,7 +1280,7 @@ config UNIX_OR_WIN32 "$unix_or_win32" | |||
config UNIXLIB "$unixlib" | |||
config GRAPHLIB "$graphlib" | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The changes here are provisional. Not sure how this should be addressed properly.
otherlibs/dthreads/event.ml
Outdated
@@ -0,0 +1,276 @@ | |||
(**************************************************************************) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just basically copied the Event
library from systhreads. It's weird, actually, there are two Event
library implementations already. This is a copy-paste that adds a third.
otherlibs/dthreads/thread.mli
Outdated
@@ -0,0 +1,46 @@ | |||
(** Lightweight threads. *) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Needs Doxygen.
otherlibs/dthreads/mutex.mli
Outdated
@@ -0,0 +1,6 @@ | |||
type t |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Needs Doxygen.
otherlibs/dthreads/condition.mli
Outdated
@@ -0,0 +1,6 @@ | |||
type t |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Needs Doxygen.
2b09cc9
to
4940091
Compare
— This branch is not ready for merge. — I have reimplemented this work with two noteworthy improvements:
Also, the logic is peppered with verbose logging of a very idiosyncratic form. I would like to remove that logic before this branch is considered eligible for merging. Finally, worth noting: I had been using a slightly modified obsolescent version of Dune as another check on the correctness of this library, but sadly that tactic is no longer working for me because A) something gets into a deadlock trying to force a Lazy value, and B) the new Dune 2.0 releases require OCaml 4.07 or later, and multicore is still baselined on 4.06. |
After rebasing, many of the tests, which were previously failing for reasons I suspected were caused by other problems, are now passing. There are still some tests that need attending, and I'll get to those soonish. I don't feel like there is enormous pressure to get this working now now now, but if there is some urgency I'm not sensing, then please respond. |
And with a little bit of additional effort to make this Accordingly, I've now seen that this branch also builds |
Many thanks for the sustained efforts on this PR! We're going to review this soon as the rest of the GC is now pretty stable. |
I'm grateful for the opportunity to make what I hope will be a noteworthy contribution. I hope. This branch is ready for preliminary review. It is not remotely close to ready for merge. I suppose if I knew how to file a "draft" pull request, this would be more correctly categorized as that. |
Now that the rebasing to 4.10 is complete in the default branch, @Engil has been working on rebasing this PR from 4.06. His working tree is at https://github.com/Engil/ocaml-multicore/tree/parallel_minor_gc_4_10%2Bdthreads (to avoid any duplication of efforts). |
Hello, A small update on the rebase work to 4.10 parallel_minor: |
Thank you for doing this! I'll follow the new branch. Feel free to close this one when that once that one is made authoritative for this change. |
Closing this Pull Request, as #342 is now the authoritative PR for merging this contribution. |
This is a request for review and comment on a candidate for implementing the
Threads
library in terms of the newDomain
and andAtomic
modules in the Multicore OCaml standard library. This change, when combined with my other patch forUnix.system
to not useUnix.fork
makes the stock Dune 1.9.2 package compile and run properly on Multicore OCaml. (Or at least, it did until something in themaster
branch related to Lazy values introduced a regression.)Some caveats are worth noting up front:
Firstly, I've made a half-hearted attempt at introducing this as a third library without overwriting the existing vmthread and systhread implementations. I noticed this decision is visible when trying to compile the ounit external package, so I'm anticipating the need to revisit some of these interoperability concerns.
Secondly, I've tried to improve the assurance of correctness by using the unit tests, but unfortunately, this pull request does not currently pass all of them reliably, and moreover, the only reason it can pass most of them at all is that I inserted a call to
Gc.minor
in the blocking section entrance hook. Obviously, that's a suboptimal workaround. Also, theswapchan
unit test looks to my eye like it has a concurrency error of its own, and can only pass reliably if the Thread library is expected to have some guarantee about context switches that I don't think it actually promises. In short, I think it only accidentally passes reliably in the mainline monocore distribution.(This introductory message is updated from its previous content.)