Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[HUDI-2840] Fixed DeltaStreaemer to properly respect configuration passed t/h properties file #4090

Merged
merged 15 commits into from
Nov 25, 2021
Merged

[HUDI-2840] Fixed DeltaStreaemer to properly respect configuration passed t/h properties file #4090

merged 15 commits into from
Nov 25, 2021

Commits on Nov 23, 2021

  1. Rebased DFSPropertiesConfiguration to access Hadoop config in liue … 

    …of FS to avoid confusion
    Alexey Kudinkin committed Nov 23, 2021
    Configuration menu
    Copy the full SHA
    a42ee8f View commit details
    Browse the repository at this point in the history

  2. Fixed readConfig to take Hadoop's Configuration instead of FS; 

    Fixing usages
    Alexey Kudinkin committed Nov 23, 2021
    Configuration menu
    Copy the full SHA
    de8fbf8 View commit details
    Browse the repository at this point in the history

  3. Added test for local FS access

    Alexey Kudinkin committed Nov 23, 2021
    Configuration menu
    Copy the full SHA
    1458ca5 View commit details
    Browse the repository at this point in the history

  4. Rebase to use FSUtils.getFs

    Alexey Kudinkin committed Nov 23, 2021
    Configuration menu
    Copy the full SHA
    487e4ce View commit details
    Browse the repository at this point in the history

Commits on Nov 24, 2021

  1. Combine properties provided as a file along w/ overrides provided fro… 

    …m the CLI
    Alexey Kudinkin committed Nov 24, 2021
    Configuration menu
    Copy the full SHA
    0352d08 View commit details
    Browse the repository at this point in the history

  2. Added helper utilities to HoodieClusteringConfig; 

    Make sure corresponding config methods fallback to defaults;
    Alexey Kudinkin committed Nov 24, 2021
    Configuration menu
    Copy the full SHA
    2af3d92 View commit details
    Browse the repository at this point in the history

  3. Fixed DeltaStreamer usage to respect properly combined configuration; 

    Abstracted `HoodieClusteringConfig.from` convenience utility to init Clustering config from `Properties`
    Alexey Kudinkin committed Nov 24, 2021
    Configuration menu
    Copy the full SHA
    7a736b3 View commit details
    Browse the repository at this point in the history

  4. Tidying up

    Alexey Kudinkin committed Nov 24, 2021
    Configuration menu
    Copy the full SHA
    0d496ff View commit details
    Browse the repository at this point in the history

  5. lint

    Alexey Kudinkin committed Nov 24, 2021
    Configuration menu
    Copy the full SHA
    8bee589 View commit details
    Browse the repository at this point in the history

  6. Reverting changes to HoodieWriteConfig

    Alexey Kudinkin committed Nov 24, 2021
    Configuration menu
    Copy the full SHA
    ad022fa View commit details
    Browse the repository at this point in the history

  7. Tdiying up

    Alexey Kudinkin committed Nov 24, 2021
    Configuration menu
    Copy the full SHA
    869be7e View commit details
    Browse the repository at this point in the history

  8. Fixed incorrect merge of the props

    Alexey Kudinkin committed Nov 24, 2021
    Configuration menu
    Copy the full SHA
    7378e6e View commit details
    Browse the repository at this point in the history

  9. Converted HoodieConfig to wrap around Properties into `TypedPrope… 

    …rties`
    Alexey Kudinkin committed Nov 24, 2021
    Configuration menu
    Copy the full SHA
    7a34a66 View commit details
    Browse the repository at this point in the history

  10. Fixed compilation

    Alexey Kudinkin committed Nov 24, 2021
    Configuration menu
    Copy the full SHA
    2a01801 View commit details
    Browse the repository at this point in the history

  11. Fixed compilation

    Alexey Kudinkin committed Nov 24, 2021
    Configuration menu
    Copy the full SHA
    5795833 View commit details
    Browse the repository at this point in the history