New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ROCKETMQ-81] import rocketmq plugin for spark streaming #5

Closed
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
6 participants
@hustfxj

hustfxj commented Mar 28, 2017

This plugin is for spark streaming based on pull mode. And you can see READ.ME for more details. And we will add more tests for this plugin after that.

@vongosling

This comment has been minimized.

Member

vongosling commented Mar 30, 2017

@ranqiqiang

This comment has been minimized.

ranqiqiang commented Mar 30, 2017

感谢 这个东西 能正式使用了吗? 期待

@shroman

This comment has been minimized.

Contributor

shroman commented Mar 31, 2017

@vongosling There are two PRs for ROCKETMQ-81. I guess @hustfxj or @vesense will have to compile them in one, and they recheck it together :)

Guys, please make sure you assign JIRA issues to yourselves before you start working on tasks, and update them. For instance, @hustfxj if you are working on the issue, please change the status, so @vesense knows you are working on it and doesn't waste his time. Two pull requests are submitted but the issue has still "Open" status :D

In fact, I think it's pretty common to pick the issue and work on it if there is no progress on it for a long time.

@zhouxinyu

This comment has been minimized.

Member

zhouxinyu commented Mar 31, 2017

Agree with @shroman .
But this two PRs are different implementations, based on PUSH and PULL models respectively, may be both can coexist.

@shroman

This comment has been minimized.

Contributor

shroman commented Mar 31, 2017

@zhouxinyu yes, luckily they are :)
IMO, if the difference is just PUSH and PULL models, it's reasonable to merge them and improve. Any other ideas?

@vesense

This comment has been minimized.

Member

vesense commented Mar 31, 2017

I plan to merge these two PRs into one. @hustfxj What do you think about it?

@hustfxj

This comment has been minimized.

hustfxj commented Mar 31, 2017

@vesense It's great! thank you

@zhouxinyu

This comment has been minimized.

Member

zhouxinyu commented Apr 17, 2017

@vesense How much the merge progress has been made now?

@vesense

This comment has been minimized.

Member

vesense commented Apr 17, 2017

@zhouxinyu Almost complete. I will submit it in this week. Some delay due to my computer wasn't working in the past few days. 😄

@zhouxinyu

This comment has been minimized.

Member

zhouxinyu commented Apr 17, 2017

@vesense Thanks for your efforts~ 😄

@vesense

This comment has been minimized.

Member

vesense commented Apr 20, 2017

I have completed the merging. The PR after merged is here: #4. Now I think we can merge it initially.
The following is the change:

  • merge PR #5 to #4
  • fix the jdk version to 1.8
  • fix package path
  • merge common configs to java/org.apache.rocketmq.spark.RocketMQConfig
  • merge common utils to scala/org.apache.rocketmq.spark.RocketMqUtils
  • add RocketMQServerMock to improve testing
  • improve document

Thanks.

@asfgit asfgit closed this in 7ceba1b Apr 27, 2017

hubot pushed a commit that referenced this pull request Apr 28, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment