[cascading3] Migrate core, commons and related #1521
Conversation
|
@cchepelov take a look? |
| @@ -340,7 +336,7 @@ lazy val scaldingCommons = module("commons").settings( | |||
| "com.twitter" %% "bijection-core" % bijectionVersion, | |||
| "com.twitter" %% "algebird-core" % algebirdVersion, | |||
| "com.twitter" %% "chill" % chillVersion, | |||
| "com.twitter.elephantbird" % "elephant-bird-cascading2" % elephantbirdVersion, | |||
| "com.twitter.elephantbird" % "elephant-bird-cascading3" % elephantbirdVersion, | |||
There was a problem hiding this comment.
can we move this to where the versions are? elephant-bird-artifact so we can keep all these switches in one place?
|
Does this pass e2e tests or CI at Twitter? |
|
If we can get an isolated Cascading3 test case we can take a stab at promoting this from 'no longer supported' to 'bug' and then to 'resolved'. |
|
Hi @posco @rubanm Re. the spurious ".forceToDisk"; indeed, the code should do the right thing without. The transform facility @cwensel wrote about looks like the correct place to put the necessary Boundaries in place. -- Cyrille Le 19 févr. 2016 19:26, à 19:26, "P. Oscar Boykin" notifications@github.com a écrit:
|
Hadoop's -libjars doesn't support wildcards, with large class paths its easy to exhaust the max arg length for linux/os x when running commands. This acts as a filter above our interaction with the generic options parser to expand wildcards
|
@cwensel about the repro: It should be as easy as a cascading |
|
@johnynek This branch now passes e2e tests at Twitter (with a related EB change twitter/elephant-bird#465). I'm working on piloting some user jobs. |
|
@rubanm this is pretty amazing work! |
|
Amazing! |
|
looks good to me to merge into cascading3 branch. does this have all the changes from current develop branch? |
|
@johnynek Thanks for the review! RC6 is currently being released to twitter source. I plan to merge develop once that release is done so it's in tandem, with the joinWithTiny fix to follow. |
|
@rubanm sounds good. Way to push through on this! |
|
Wow, @rubanm, this is great, many thanks for doing the hard work! With apologies for the lack of testing on my side, been litterally Le 11/04/2016 17:41, Ruban Monu a écrit :
|
part of #1465
based on Cyrille's work in #1446
Most of the interesting changes are in:
Cascading fabric selection changes will be sent in a separate PR.