Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ROCKETMQ-81] Add the RocketMq plugin for the Apache Spark #4

Closed
wants to merge 7 commits into from
Closed

[ROCKETMQ-81] Add the RocketMq plugin for the Apache Spark #4

wants to merge 7 commits into from

Conversation

vesense
Copy link
Member

@vesense vesense commented Mar 28, 2017

https://issues.apache.org/jira/browse/ROCKETMQ-81

This is the init version for code review.
features:

  • RocketMQReceiver - which is no fault-tolerance guarantees
  • ReliableRocketMQReceiver - which is fault-tolerance guarantees

@vongosling
Copy link
Member

@dongeforever @shroman @stevenschew what's your opinion about it

@vesense vesense mentioned this pull request Apr 20, 2017
Closed
@vesense
Copy link
Member Author

vesense commented Apr 20, 2017 •
edited

NOTICE: PR #5 has merged into this one, the work are completed by both @hustfxj and I. And I have added the credit for @hustfxj too.

@zhouxinyu
Copy link
Member

@vesense thanks, please @hustfxj help review this incorporative PR.

@hustfxj
Copy link

hustfxj commented Apr 23, 2017

@vesense Thanks. Can you add add more tests for this plugin, especially the pull mode. Of course, it is difficult.

@vesense
Copy link
Member Author

vesense commented Apr 24, 2017

@hustfxj Yes, I will try to add more unit tests, especially pull mode.
@zhouxinyu Do you mind merging this firstly? I will add the tests maybe a few days later because of my time, and I don't want to block other peoples contributions.

shroman
shroman reviewed Apr 24, 2017
View reviewed changes
rocketmq-storm/README.md
@@ -0,0 +1,2 @@
# RocketMQ Storm Integration
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this pushed by mistake?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this is just a placeholder for rocketmq-storm new README.md file. I plan to add some description in this file. It may be better to do this in another PR.

hustfxj
hustfxj reviewed Apr 25, 2017
View reviewed changes
rocketmq-spark/src/main/scala/org/apache/rocketmq/spark/streaming/MQPullInputDStream.scala Outdated
/**
* start up timer thread to persis the OffsetStore
*/
private val scheduledExecutorService = ThreadUtils.newDaemonSingleThreadScheduledExecutor(
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add @transient for this val

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hustfxj Fixed.

@zhouxinyu
Copy link
Member

Hi, I will merge this PR and close #5.

@asfgit asfgit closed this in 7ceba1b Apr 27, 2017
@zhouxinyu
Copy link
Member

I have no idea about how to commit with two authors, so I thanked @hustfxj in the commit log, hope you don't mind ~~

hubot pushed a commit that referenced this pull request Apr 28, 2017
@StyleTang
Merge pull request #4 from tcrow/tcrow
a900f60 
update cluster UI
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shroman shroman shroman left review comments

@hustfxj hustfxj hustfxj left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@vesense @vongosling @zhouxinyu @hustfxj @shroman