Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SPARK-4817[STREAMING]Print the specified number of data and handle all of the elements in RDD #3662

Closed
wants to merge 2 commits into from

Conversation

surq
Copy link

@surq surq commented Dec 10, 2014

Dstream.print function:Print 10 elements and handle 11 elements.
A new function based on Dstream.print function is presented:
the new function:
Print the specified number of data and handle all of the elements in RDD.
there is a work scene:
val dstream = stream.map->filter->mapPartitions->print
the data after filter need update database in mapPartitions,but don't need print each data,only need to print the top 20 for view the data processing.

@surq surq changed the title Print the specified number of data and handle all of the elements in RDD [SPARK-4817][streaming]Print the specified number of data and handle all of the elements in RDD Dec 10, 2014
@srowen
Copy link
Member

srowen commented Dec 10, 2014

You should put SPARK-XXXX [STREAMING] in the title. But your original JIRA was a duplicate of https://issues.apache.org/jira/browse/SPARK-3325 so perhaps you can connect this to that JIRA.

@SparkQA
Copy link

SparkQA commented Dec 10, 2014

Test build #24306 has started for PR 3662 at commit 4e3f715.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Dec 10, 2014

Test build #24306 has finished for PR 3662 at commit 4e3f715.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24306/
Test PASSed.

@surq surq changed the title [SPARK-4817][streaming]Print the specified number of data and handle all of the elements in RDD SPARK-4817[STREAMING]Print the specified number of data and handle all of the elements in RDD Dec 11, 2014
@SparkQA
Copy link

SparkQA commented Dec 11, 2014

Test build #24358 has started for PR 3662 at commit 411b287.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Dec 11, 2014

Test build #24358 has finished for PR 3662 at commit 411b287.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24358/
Test PASSed.

@tdas
Copy link
Contributor

tdas commented Dec 24, 2014

As I commented on the JIRA, this isnt right way to do what you are trying to achieve. The JIRA has already been closed, so mind closing this PR?

@asfgit asfgit closed this in 534f24b Dec 27, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
6 participants