Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stream cookbook #16498

Merged
merged 2 commits into from Dec 19, 2014
Merged

Stream cookbook #16498

merged 2 commits into from Dec 19, 2014

Conversation

drewhk
Copy link
Member

@drewhk drewhk commented Dec 9, 2014

NOT FOR MERGE

Just a quick preview, WIP. Feel free to recommend new recipes.

@akka-ci akka-ci added validating PR is currently being validated by Jenkins needs-attention Indicates a PR validation failure (set by CI infrastructure) and removed validating PR is currently being validated by Jenkins labels Dec 9, 2014
@akka-ci
Copy link

akka-ci commented Dec 9, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.akka.io/job/pr-validator-per-commit-jenkins/1524/

ktoso
ktoso reviewed Dec 10, 2014
View reviewed changes
akka-docs-dev/rst/scala/code/docs/stream/cookbook/RecipeCollectingMetrics.scala
currentLoad ~> collector.left
reportTicks ~> collector.right

collector.out ~> sink
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any zipping with time would need some kind of (in the text of the docs) hints about buffer sizes etc?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, thats the plan.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@ktoso
Copy link
Member

ktoso commented Dec 10, 2014

Looks good so far!

patriknw
patriknw reviewed Dec 15, 2014
View reviewed changes
akka-docs-dev/rst/scala/code/docs/stream/cookbook/RecipeByteStrings.scala
if (buffer.isEmpty) ctx.pull()
else {
val emit = buffer.take(chunkSize)
buffer = buffer.drop(chunkSize)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a splitAt?

@patriknw
Copy link
Member

lots of awesome stuff here!

@akka-ci akka-ci added validating PR is currently being validated by Jenkins needs-attention Indicates a PR validation failure (set by CI infrastructure) and removed needs-attention Indicates a PR validation failure (set by CI infrastructure) labels Dec 17, 2014
@akka-ci
Copy link

akka-ci commented Dec 17, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.akka.io/job/pr-validator-per-commit-jenkins/1598/

@akka-ci akka-ci removed the validating PR is currently being validated by Jenkins label Dec 17, 2014
@drewhk drewhk force-pushed the wip-15239-stream-cookbook-drewhk branch from 5dee3a9 to 6051f27 Compare December 17, 2014 16:19
@drewhk
Copy link
Member Author

drewhk commented Dec 17, 2014

Now added docs

@akka-ci akka-ci added validating PR is currently being validated by Jenkins and removed needs-attention Indicates a PR validation failure (set by CI infrastructure) labels Dec 17, 2014
@drewhk
Copy link
Member Author

drewhk commented Dec 17, 2014

there is a flaky test case still, and missing the Java part

ktoso
ktoso reviewed Dec 18, 2014
View reviewed changes
akka-docs-dev/rst/scala/cookbook.rst

We express our solution as a :class:`StatefulStage` because it has support for emitting multiple elements easily
through its ``emit(iterator, ctx)`` method. Since an incoming ByteString chunk might contain multiple lines (frames)
this feature comes handy.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comes in handy

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks

@patriknw
Copy link
Member

looking very good

@ktoso
Copy link
Member

ktoso commented Dec 18, 2014

LGTM after minor updates and perhaps adding the example Will mentioned (rate limiting multiple streams)

@drewhk
Copy link
Member Author

drewhk commented Dec 18, 2014

Added a rate limiter

@akka-ci akka-ci added validating PR is currently being validated by Jenkins needs-attention Indicates a PR validation failure (set by CI infrastructure) and removed needs-attention Indicates a PR validation failure (set by CI infrastructure) validating PR is currently being validated by Jenkins labels Dec 18, 2014
@akka-ci
Copy link

akka-ci commented Dec 18, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.akka.io/job/pr-validator-per-commit-jenkins/1615/

drewhk
drewhk reviewed Dec 19, 2014
View reviewed changes
akka-docs-dev/rst/scala/code/docs/stream/cookbook/RecipeReduceByKey.scala
}

// get a stream of word counts
val counts: Source[(String, Int)] = countedWords.mapAsync(identity)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can deadlock

@drewhk drewhk force-pushed the wip-15239-stream-cookbook-drewhk branch from 3a18f86 to 3a8b662 Compare December 19, 2014 10:42
@akka-ci akka-ci added validating PR is currently being validated by Jenkins tested PR that was successfully built and tested by Jenkins and removed needs-attention Indicates a PR validation failure (set by CI infrastructure) validating PR is currently being validated by Jenkins labels Dec 19, 2014
@akka-ci
Copy link

akka-ci commented Dec 19, 2014

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.akka.io/job/pr-validator-per-commit-jenkins/1637/

drewhk pushed a commit that referenced this pull request Dec 19, 2014
Merge pull request #16498 from drewhk/wip-15239-stream-cookbook-drewhk
45cd3c0 
Stream cookbook
@drewhk drewhk merged commit 45cd3c0 into akka:release-2.3-dev Dec 19, 2014
@drewhk drewhk deleted the wip-15239-stream-cookbook-drewhk branch May 12, 2015 14:01
ktoso pushed a commit to ktoso/akka that referenced this pull request Jan 11, 2016
Merge pull request akka#16498 from drewhk/wip-15239-stream-cookbook-d…
cf46428 
…rewhk

Stream cookbook
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
tested PR that was successfully built and tested by Jenkins
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@drewhk @akka-ci @ktoso @patriknw @rkuhn