Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Memory leak in iteratees (recursive Future.flatMap) #1084

Closed
richdougherty opened this issue May 2, 2013 · 3 comments
Closed

Memory leak in iteratees (recursive Future.flatMap) #1084

richdougherty opened this issue May 2, 2013 · 3 comments
Assignees
@richdougherty
Milestone
2.3.0

Comments

@richdougherty
Copy link
Member

The following code results in an OutOfMemoryError:

Enumerator.repeat("hello") |>> Iteratee.ignore[String]

Raised by @jroper on the dev mailing list: https://groups.google.com/d/topic/play-framework-dev/58VZD-YXdJw/discussion

The problem seems to be caused by using calling Future.flatMap recursively, which causes chains of completion handlers to grow to an unbounded size. See the mailing list discussion for more info.

A solution may be to avoid Future.flatMap by building asynchronous computations out of some other type and converting it to a Future at the last minute. This will allow us to avoid building chains of completion handlers. See, eg: richdougherty@396e425
@ghost ghost assigned richdougherty May 2, 2013
@richdougherty
Copy link
Member Author

See Scala issue: https://issues.scala-lang.org/browse/SI-7336

@richdougherty richdougherty mentioned this issue Jun 23, 2013
Merged
@benmccann benmccann mentioned this issue Jul 29, 2013
Merged
@richdougherty
Copy link
Member Author

@jroper, @huntc: I think we can upgrade to Scala 2.10.3-RC2 on master now to get this fix in.

We could verify the fix with a test like this: https://github.com/richdougherty/scala/blob/48c677ceb3177d93e700b399c00af6b8bb6419e4/test/files/run/t7336

richdougherty added a commit to richdougherty/playframework that referenced this issue Sep 24, 2013
@richdougherty
Upgrade to Scala 2.10.3-RC3 and check iteratee memory leak is fixed
41b6c9b 
Scala 2.10.3 inclues a fix that allows memory to be freed when chaining
Future.flatMap calls (SI-7336). Iteratees make extensive use of
Future.flatMap, so upgrading to Scala 2.10.3 allows us to avoid memory
leaks when using iteratees (playframework#1084). Memory leaks can occur when
processing large amounts of data or when working with connections that
can remain open for a long time.
@richdougherty richdougherty mentioned this issue Sep 24, 2013
Merged
@richdougherty
Copy link
Member Author

Fixed by #1735.

@jroper jroper mentioned this issue Nov 12, 2013
Closed
@alexandru alexandru mentioned this issue Nov 1, 2017
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@richdougherty richdougherty

Labels
None yet
Projects
None yet
Milestone
2.3.0
Development

No branches or pull requests
1 participant
@richdougherty