Skip to content

[HUDI-371] : Supporting combine input format RT tables#1503

Merged
bvaradar merged 1 commit intoapache:masterfrom
n3nash:combine_input_format_rt_tables_3
Apr 21, 2020
Merged

[HUDI-371] : Supporting combine input format RT tables#1503
bvaradar merged 1 commit intoapache:masterfrom
n3nash:combine_input_format_rt_tables_3

Conversation

@n3nash
Copy link
Contributor

@n3nash n3nash commented Apr 9, 2020

Tips

What is the purpose of the pull request

We current support HoodieCombineHiveInputFormat for CopyOnWrite tables to ensure that there is a limit on the number of mappers spawned for any query. We currently do not have support for this feature on Realtime tables and this PR adds this support

Committer checklist

  • Has a corresponding JIRA in PR title & commit

  • Commit message is descriptive of the change

  • CI is green

  • Necessary doc changes done or have another open PR

  • For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@n3nash n3nash added the status:in-progress Work in progress label Apr 9, 2020
@n3nash n3nash changed the title Supporting combine input format RT tables [HUDI-371] : Supporting combine input format RT tables Apr 9, 2020
@n3nash n3nash force-pushed the combine_input_format_rt_tables_3 branch 6 times, most recently from 00ff08a to 946df4a Compare April 10, 2020 16:13
@n3nash n3nash removed the status:in-progress Work in progress label Apr 10, 2020
@n3nash
Copy link
Contributor Author

n3nash commented Apr 10, 2020

Closed PR -> #1053 in favor of this one. @bvaradar PTAL

Copy link
Contributor

@bvaradar bvaradar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@n3nash Looks good. Once you resolve conflicts, we can merge.

@vinothchandar
Copy link
Member

@bvaradar @n3nash any code that is reused from other projects here? (asking since this is Hive and combine input splits)..

@n3nash
Copy link
Contributor Author

n3nash commented Apr 20, 2020

@vinothchandar no new class, for the existing class -> https://github.com/apache/incubator-hudi/blob/master/LICENSE#L206

@n3nash n3nash force-pushed the combine_input_format_rt_tables_3 branch from 0a9aa33 to 88e702f Compare April 20, 2020 17:09
@bvaradar bvaradar merged commit 332072b into apache:master Apr 21, 2020
kroushan-nit pushed a commit to kroushan-nit/hudi-oss-fork that referenced this pull request Aug 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants