Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite channel in Crystal #3920

Closed
firejox opened this issue Jan 20, 2017 · 3 comments
Closed

Rewrite channel in Crystal #3920

firejox opened this issue Jan 20, 2017 · 3 comments

Comments

@firejox
Copy link
Contributor

firejox commented Jan 20, 2017

refer to #3912
I want to rewrite channel since current channel is hard to be thread safe and there are some bugs in
select operation, see #3900 #3862. The new design is implemented by some simple I-Var. It can be
thread safe if the coroutine switch is thread safe. Also those bugs in select will be solved since that
will not queue the duplicate fiber into scheduler.

And this is the performance tested by threadring
https://gist.github.com/firejox/f9fb7c4b6d44501d0b1070c123a1fab6

@TheLonelyGhost
Copy link
Contributor

TheLonelyGhost commented Jan 20, 2017

As someone unfamiliar with comparing Crystal's efficiency to that of other languages', I have some stupid questions:

  1. In comparison to Go, why might Crystal have higher memory usage like this when the runtime is shorter for the same test?
  2. Why might this change have spiked the memory to be 4x the original amount from before the change?

I ask because I'm genuinely interested in concurrency, threading, and digging into this topic at a lower level than I have thus far. Crystal seems like as good of a medium to do so as any other. Also, it might give good background knowledge for your change on the associated pull request to those who might not have previous knowledge of the topic.

@firejox
Copy link
Contributor Author

firejox commented Jan 21, 2017

@TheLonelyGhost I am not familiar with Go, so I don't know how Go implement it. Maybe there are some optimization in Go such that have smaller memory usage in channel. The current channel in Crystal is implemented by two queues (one for sender fibers and another one for receiver fibers) and the share buffer for communicating.

Denote that this is impossible to make receiver queue and share buffer be both non-empty. One of
my changes is to combine the two part into one queue, and use a integer value to represent the status of channel. And The another queue in my code can be treated as the sender queue in origin.

The another change is the I-Var data structure, a special Future. It can set value only one times and get value as Future do. Therefore, it can notify the Fibers and transmit the data. And this change is the main reason for using a lot memory sinceI-Var stores the value, error, status, and Fiber. It actually requires more memory than before.

@asterite
Copy link
Member

Closing, but only because in the thread-support branch this has been re-written so touching the current Channel implementation by others probably isn't worth it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants