Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HUDI-6908] Confirm e2e flow of the spark record #9817

Conversation

linliu-code
Copy link
Contributor

Change Logs

We created a functional test to write data and read data, meanwhile log the writers that are used to write data into local files.
We confirmed that only SparkParquetWriters are used to write internal rows into the disk.

This PR is used to confirm the e2e write paths of the internal rows. We will not try to land this PR into prod.

Impact

No.

Risk level (write none, low medium or high below)

No.

Documentation Update

Describe any necessary documentation update if there is any new feature, config, or user-facing change

  • The config description must be updated if new configs are added or the default value of the configs are changed
  • Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
    ticket number here and follow the instruction to make
    changes to the website.

Contributor's checklist

  • Read through contributor's guide
  • Change Logs and Impact were stated clearly
  • Adequate tests were added if applicable
  • CI passed

We created a functional test to write data and read data, meanwhile
log the writers that are used to write data into local files.

We confirmed that only SparkParquetWriters are used to write internal rows
into the disk.
@linliu-code linliu-code closed this Oct 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant