Skip to content

[MINOR] Expose filename when getting commit time from illegal filename#9304

Merged
yihua merged 3 commits intoapache:masterfrom
CTTY:ctty/committime-fix
Jul 28, 2023
Merged

[MINOR] Expose filename when getting commit time from illegal filename#9304
yihua merged 3 commits intoapache:masterfrom
CTTY:ctty/committime-fix

Conversation

@CTTY
Copy link
Contributor

@CTTY CTTY commented Jul 28, 2023

Change Logs

Sometimes users can get non-hudi files mixed in Hudi table causing ArrayIndexOutOfBoundsException when getting commit time from filename. This fix is to expose this error better to make it easier for users to identify problematic files.

Impact

None

Risk level (write none, low medium or high below)

None

Documentation Update

Describe any necessary documentation update if there is any new feature, config, or user-facing change

  • The config description must be updated if new configs are added or the default value of the configs are changed
  • Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
    ticket number here and follow the instruction to make
    changes to the website.

Contributor's checklist

  • Read through contributor's guide
  • Change Logs and Impact were stated clearly
  • Adequate tests were added if applicable
  • CI passed

@danny0405
Copy link
Contributor

Sometimes users can get non-hudi files mixed in Hudi table

How could this happen?

@CTTY
Copy link
Contributor Author

CTTY commented Jul 28, 2023

Hi @danny0405 , this would happen when some users accidently copy files to their table location or writer is not correctly set up like they forgot to use HoodieSparkSessionExtension. Either way I think we need to report this error better to help people understand the issue

@hudi-bot
Copy link
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@yihua yihua merged commit 64310f5 into apache:master Jul 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants