Skip to content

[MINOR] Fix flaky TestHoodieAvroDataBlock#12831

Merged
nsivabalan merged 1 commit intoapache:masterfrom
wombatu-kun:MINOR_fix_flaky_TestHoodieAvroDataBlock
Feb 12, 2025
Merged

[MINOR] Fix flaky TestHoodieAvroDataBlock#12831
nsivabalan merged 1 commit intoapache:masterfrom
wombatu-kun:MINOR_fix_flaky_TestHoodieAvroDataBlock

Conversation

@wombatu-kun
Copy link
Contributor

Change Logs

Recently i faced this failure:
Error: TestHoodieAvroDataBlock.testGetRecordIteratorWithInputStream:188->verifyRecords:360 Record count mismatch ==> expected: <0> but was: <1000>
That happened because of unnecessary random value generation in selectRandomRecords: as a number of selected records was randomly generated 0 value, so it returned an empty list recordsForFilter (and keysForFilter). Later, when we call dataBlock.getRecordIterator with empty list as keysForFilter, it returns all records, and it causes records verification to fail.

This PR fixes it by not using Random for generating number of selected records, just use value allRecordsNum/4.

Impact

fix flaky test

Risk level (write none, low medium or high below)

none

Documentation Update

none

  • The config description must be updated if new configs are added or the default value of the configs are changed
  • Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
    ticket number here and follow the instruction to make
    changes to the website.

Contributor's checklist

  • Read through contributor's guide
  • Change Logs and Impact were stated clearly
  • Adequate tests were added if applicable
  • CI passed

@github-actions github-actions bot added the size:XS PR with lines of changes in <= 10 label Feb 12, 2025
@hudi-bot
Copy link
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@nsivabalan nsivabalan merged commit c9003eb into apache:master Feb 12, 2025
43 checks passed
@voonhous
Copy link
Member

voonhous commented Apr 8, 2025

Removing release-1.0.2 tag for this as this PR is dependent on !11924

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XS PR with lines of changes in <= 10

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants