Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug fixes for and extensions of new Java API #542

Closed
wants to merge 5 commits into from

Conversation

fhueske
Copy link
Contributor

@fhueske fhueske commented Mar 7, 2014

  • Fix for Map operator translation (introduced by merge of 0038c9d)
  • Fix for TupleComparator (normalized key writing) @StephanEwen please verify
  • Fix for BasicTypeComparator (inverted sort order) @StephanEwen please verify
  • initial support for Group Sorting (position keys and Reduce operator only)
  • extended TypeExtractor and TupleTypeInfo to work with subclasses of Tuple classes.
  • added join function for semi joins. Support in runtime required to enable this feature.

Fabian Hueske added 3 commits March 7, 2014 11:20
TODO: Special exec strategy (runtime + optimizer) based on hash join required.
      Either no duplicates of filtering side in HT or filtered tuples removed from HT after first match.
@fhueske fhueske mentioned this pull request Mar 7, 2014
@rmetzger rmetzger added the JAPI label Mar 7, 2014
}

@Override
protected BinaryNodeTranslation translateToDataFlow() {
// TODO: Runtime support required. Each left tuple may be returned only once.
// Special exec strategy (runtime + optimizer) based on hash join required.
// Either no duplicates of right side in HT or left tuples removed from HT after first match.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't you implement that using CoGroup?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is almost easier to do this with an extra runtime op. Those are
line 20 lines only as well...
Am 07.03.2014 20:58 schrieb "Robert Metzger" notifications@github.com:

In
stratosphere-java/src/main/java/eu/stratosphere/api/java/operators/JoinOperator.java:

    }

    @Override
    protected BinaryNodeTranslation translateToDataFlow() {
  •       // TODO: Runtime support required. Each left tuple may be returned only once.
    
  •       //       Special exec strategy (runtime + optimizer) based on hash join required.
    
  •       //       Either no duplicates of right side in HT or left tuples removed from HT after first match.
    

Can't you implement that using CoGroup?


Reply to this email directly or view it on GitHubhttps://github.com//pull/542/files#r10397365
.

@fhueske
Copy link
Contributor Author

fhueske commented Mar 12, 2014

Fixes #543

@fhueske
Copy link
Contributor Author

fhueske commented Mar 13, 2014

Rebased to stratosphere:staging branch in #565

@fhueske fhueske closed this Mar 13, 2014
@fhueske fhueske deleted the japi branch March 13, 2014 09:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants