Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.
Sign upMultithreading #8112
Multithreading #8112
Conversation
Move event_loop ivars to Thread Use libevent pthread support
…. Added main thread as a worker.
…ed. It now supports MT for send/receive and select operations.
…lation is fixed). Use simpler but effective RR scheduling instead.
… select statement
…nqueue_self as Scheduler#enqueue now always sends the fiber to the local queue.
…ode.
…dcoded value (otherwise the constructor of Time requires the value of UNIX_EPOCH already set to validate the parameters)
…ltithreading)
…r thread
Only positive integer values are accepted or empty string (that switches back to the default number of workers). Any other value will make the program to crash.
This comment has been minimized.
This comment has been minimized.
|
This won't compile because |
This comment has been minimized.
This comment has been minimized.
|
Good catch!! Actually that method is not used for a loooong time, since this commit: 56dec21 |
This comment has been minimized.
This comment has been minimized.
|
@waj I wonder if performance changes if we change these to be |
This comment has been minimized.
This comment has been minimized.
|
It does! It's obviously really small difference but everything counts. Thanks! |
This comment has been minimized.
This comment has been minimized.
|
I just tested with a multithread matrix multiplication example that heavily exercises inter thread communication without any IO involved and the difference is around 1% improvement. |
This comment has been minimized.
This comment has been minimized.
|
Cool! |
| private def self.event_base | ||
| Thread.current.event_base | ||
| end | ||
|
|
||
| private def self.loop_fiber |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
j8r
Aug 27, 2019
Contributor
I'm talking about this very private class method, not the instance method and variable.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
waj
Aug 27, 2019
Author
Member
The instance variable belongs to Thread. This static method is defined in Crystal::EventLoop module.
This comment has been minimized.
This comment has been minimized.
j8r
Aug 27, 2019
Contributor
Sure, but sorry I didn't undestand the code URL you mentioned, where EventLoop.loop_fiber is used then?
This comment has been minimized.
This comment has been minimized.
asterite
Aug 28, 2019
Member
Same file, line 26
crystal/src/crystal/event_loop.cr
Line 26 in 6c6cadd
This comment has been minimized.
This comment has been minimized.
j8r
Aug 28, 2019
Contributor
Oh ok sorry, thanks. I didn't know this language feature, to call a static method like that.
| module Crystal::EventLoop | ||
| {% unless flag?(:preview_mt) %} | ||
| def self.after_fork | ||
| Thread.current.event_base.reinit |
This comment has been minimized.
This comment has been minimized.
|
This looks fantastic! I'm excited about how much thought has been given to making it easy on application developers to get parallel execution while mitigating thread-safety issues! |
| @@ -58,8 +58,11 @@ end | |||
| # | |||
| # 2.times { ch.receive } | |||
| # ``` | |||
| def spawn(*, name : String? = nil, &block) | |||
| def spawn(*, name : String? = nil, same_thread = false, &block) | |||
This comment has been minimized.
This comment has been minimized.
jgaskins
Aug 28, 2019
Contributor
Such a small thing but sooooo crucial. I use spawn all the time to move work out-of-band but I rely on it waiting until the current fiber is yielded.
Something that's been in the back of my mind every time I've done it is "this isn't going to work when Crystal goes multithreaded" but this one argument, along with fibers staying within their threads, solves that problem!
This comment has been minimized.
This comment has been minimized.
jhass
Aug 28, 2019
Member
Just make sure nobody changes the contract from "stays on the same thread" to "is initially queued for the same thread" in case job stealing is implemented :D
This comment has been minimized.
This comment has been minimized.
bcardiff
Aug 28, 2019
Member
@jgaskins I would recommend using Channel to wait for the result. The same_thread is more to ensure the execution context than the scheduling.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
asterite
Aug 28, 2019
Member
@j8r Type restrictions are optional in Crystal. In this way one can pass a nilable thing and it will work. Restricting it to Bool will make it less flexible, forcing you to do things like !!exp.
I think we should embrace the way Crystal lets you omit type annotations. If something is not strictly required by the language then we shouldn't always push for it.
|
|
waj commentedAug 23, 2019
This is a big PR that adds initial and experimental support for multithreading. It works side by side with the current single thread implementation and must be enabled with the
preview_mtflag.There is an upcoming blog post explaining details about the implementation and decisions made on this work.
Things covered by this PR:
Channel: re-created from scratch because with the previous implementation it was harder to make a multi threadselectoperation.MutexThings not covered by this PR:
Arrayis not thread safe and must be protected by a mutex in case it's shared between fibers