New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Easy performance wins with Scalaz #17

Open
runarorama opened this Issue Jun 19, 2015 · 10 comments

Comments

Projects
None yet
7 participants
@runarorama
Owner

runarorama commented Jun 19, 2015

No description provided.

@duduk

This comment has been minimized.

Show comment
Hide comment
@duduk

duduk Jun 19, 2015

Hello Rúnar,

Isn't it that, since ackermannF already returns Future[Int], there should be no Future.apply calls in last 2 cases? So, there should be:

        case (m, 0) => ackermannF (m - 1, 1)
        case (m, n) => ackermannF (m, n - 1).flatMap (x => ackermannF (m - 1, x))

instead of:

        case (m, 0) => Future (ackermannF (m - 1, 1))
        case (m, n) => Future (ackermannF (m, n - 1)).flatMap { x => Future (ackermannF (m - 1, x)) }

Regards,
Ivan

duduk commented Jun 19, 2015

Hello Rúnar,

Isn't it that, since ackermannF already returns Future[Int], there should be no Future.apply calls in last 2 cases? So, there should be:

        case (m, 0) => ackermannF (m - 1, 1)
        case (m, n) => ackermannF (m, n - 1).flatMap (x => ackermannF (m - 1, x))

instead of:

        case (m, 0) => Future (ackermannF (m - 1, 1))
        case (m, n) => Future (ackermannF (m, n - 1)).flatMap { x => Future (ackermannF (m - 1, x)) }

Regards,
Ivan

@runarorama

This comment has been minimized.

Show comment
Hide comment
@runarorama

runarorama Jun 19, 2015

Owner

@duduk You're totally right. Fixed!

Owner

runarorama commented Jun 19, 2015

@duduk You're totally right. Fixed!

@flicken

This comment has been minimized.

Show comment
Hide comment
@flicken

flicken Jun 22, 2015

Shouldn't the base case use Future#successful (analogous to Task#now):

case (0, _) => Future.successful(n + 1)

Instead of the current Future#apply?

case (0, _) => Future(n + 1)

flicken commented Jun 22, 2015

Shouldn't the base case use Future#successful (analogous to Task#now):

case (0, _) => Future.successful(n + 1)

Instead of the current Future#apply?

case (0, _) => Future(n + 1)
@magro

This comment has been minimized.

Show comment
Hide comment
@magro

magro Jun 25, 2015

With futures I could specify an execution context that runs the callback in the same thread, like this one: https://github.com/inoio/solrs/blob/master/src/main/scala/io/ino/concurrent/Execution.scala

Do you see a disadvantage with this approach?

magro commented Jun 25, 2015

With futures I could specify an execution context that runs the callback in the same thread, like this one: https://github.com/inoio/solrs/blob/master/src/main/scala/io/ino/concurrent/Execution.scala

Do you see a disadvantage with this approach?

@flicken

This comment has been minimized.

Show comment
Hide comment
@flicken

flicken Jun 25, 2015

@magro The sameThreadContextexecutes the Futureinstances in the same call stack, thus suffering from the same stack overflows as the original version (only much worse, because the ExecutionContext calls are also on the stack).

flicken commented Jun 25, 2015

@magro The sameThreadContextexecutes the Futureinstances in the same call stack, thus suffering from the same stack overflows as the original version (only much worse, because the ExecutionContext calls are also on the stack).

@tbfangel

This comment has been minimized.

Show comment
Hide comment
@tbfangel

tbfangel Oct 23, 2015

Hello Rúnar,

Very interesting article which touches upon some the performance issues we are currently investigating on an Akka project with a basic design strategy to do everything in a non-blocking fashion using Futures. In one of the components of the system we see a very, very high amount of context switches (~ 100k) which makes us a bit nervous. We have been trying to figure out if there's some blocking code hiding around in a corner of any 3rd party Java library used, but without success yet. So, we are now looking for other explanations than blocking code, and I'm wondering whether mapping on Futures can be part of the explanation.

The component in question does batch processing of millions of records and for each chunk it does some lookup of static data in a cached API (returned in a Future) subsequently some processing of this data and finally writes it to a report file using Camel. (And we have ruled out the I/O as a source of the context switches - it's still there if we disable writing the data to disk). Everything is done by hefty mapping on Futures. So, I would be really interested in seeing a graph of the number of context switches done for each implementation of the Ackermann algorithm in your post. Is the poor performance of the implementation using Scala's own Future directly reflected in the number of context switches? And likewise, can the better performance of the optimized algorithm using scalaz' Task be seen in the number of context switches?

What do you think?

Thanks,

Hello Rúnar,

Very interesting article which touches upon some the performance issues we are currently investigating on an Akka project with a basic design strategy to do everything in a non-blocking fashion using Futures. In one of the components of the system we see a very, very high amount of context switches (~ 100k) which makes us a bit nervous. We have been trying to figure out if there's some blocking code hiding around in a corner of any 3rd party Java library used, but without success yet. So, we are now looking for other explanations than blocking code, and I'm wondering whether mapping on Futures can be part of the explanation.

The component in question does batch processing of millions of records and for each chunk it does some lookup of static data in a cached API (returned in a Future) subsequently some processing of this data and finally writes it to a report file using Camel. (And we have ruled out the I/O as a source of the context switches - it's still there if we disable writing the data to disk). Everything is done by hefty mapping on Futures. So, I would be really interested in seeing a graph of the number of context switches done for each implementation of the Ackermann algorithm in your post. Is the poor performance of the implementation using Scala's own Future directly reflected in the number of context switches? And likewise, can the better performance of the optimized algorithm using scalaz' Task be seen in the number of context switches?

What do you think?

Thanks,

@runarorama

This comment has been minimized.

Show comment
Hide comment
@runarorama

runarorama Oct 23, 2015

Owner

@tbfangel The performance difference here is precisely because of the number of context switches. The trampoline involves no context switches at all.

Owner

runarorama commented Oct 23, 2015

@tbfangel The performance difference here is precisely because of the number of context switches. The trampoline involves no context switches at all.

@trylks

This comment has been minimized.

Show comment
Hide comment
@trylks

trylks Sep 28, 2016

Hello, I like this post a lot, it helps to understand quite a few concepts, thank you for the explanations. IMHO it is a bit too dense and packed and it should be possible to make easier to understand a few things. In particular, there are four options evaluated, Future, Task, Trampoline, and Optimized, but there are only three code snippets corresponding to only three of these options. I think that the one missing may be the one corresponding to Task, and it may be obvious that it is a small modification from Future, but it would be nice to have it anyway, or at least an explanation of how it should be, or at the very least a mention acknowledging that it is missing and left as an exercise to the reader.

As I said, maybe it is obvious, but I am not familiar with the Task monad and I spent some time trying to figure out that such snippet was missing and in fact its place is taken by the Trampoline snippet.

Thank you again.

trylks commented Sep 28, 2016

Hello, I like this post a lot, it helps to understand quite a few concepts, thank you for the explanations. IMHO it is a bit too dense and packed and it should be possible to make easier to understand a few things. In particular, there are four options evaluated, Future, Task, Trampoline, and Optimized, but there are only three code snippets corresponding to only three of these options. I think that the one missing may be the one corresponding to Task, and it may be obvious that it is a small modification from Future, but it would be nice to have it anyway, or at least an explanation of how it should be, or at the very least a mention acknowledging that it is missing and left as an exercise to the reader.

As I said, maybe it is obvious, but I am not familiar with the Task monad and I spent some time trying to figure out that such snippet was missing and in fact its place is taken by the Trampoline snippet.

Thank you again.

@yanns

This comment has been minimized.

Show comment
Hide comment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment