Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support converting non-inlined to inlined #15444

Merged
merged 1 commit into from
Aug 3, 2022

Conversation

harshavardhana
Copy link
Member

@harshavardhana harshavardhana commented Jul 31, 2022

Description

Add support converting non-inlined to inlined

Motivation and Context

This is a feature to allow for inode compaction on
large clusters that use a lot of small files spread
across a large hierarchy.

How to test this PR?

minio/mc#4175 use this to

~ mc admin heal --recursive --rewrite

To migrate old objects that are not inlined to be
inlined with newer requirements.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Optimization (provides speedup with no functional changes)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • Fixes a regression (If yes, please add commit-id or PR # here)
  • Documentation updated
  • Unit tests added/updated

@harshavardhana harshavardhana force-pushed the add-recreate branch 2 times, most recently from 677f2ee to 7b45181 Compare July 31, 2022 08:32
@harshavardhana harshavardhana changed the title Add support for migrating from noIndex(xl.meta) -> indexed(xl.meta) Add support converting non-inlined to inlined Jul 31, 2022
@harshavardhana harshavardhana marked this pull request as ready for review July 31, 2022 08:33
This is a feature to allow for inode compaction on
large clusters that use a lot of small files spread
across a large heirarchy.
@minio-trusted
Copy link
Contributor

Mint Automation

Test Result
mint-large-bucket.sh ✔️
mint-fs.sh ✔️
mint-gateway-s3.sh ✔️
mint-erasure.sh ✔️
mint-dist-erasure.sh ✔️
mint-compress-encrypt-dist-erasure.sh ✔️
mint-pools.sh ✔️
Deleting image on docker hub
Deleting image locally

@@ -560,6 +579,7 @@ func (er erasureObjects) healObject(ctx context.Context, bucket string, object s
})
if len(inlineBuffers) > 0 && inlineBuffers[i] != nil {
partsMetadata[i].Data = inlineBuffers[i].Bytes()
partsMetadata[i].SetInlineData()
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This a bug that is fixed to ensure that the partsMetadata contains the right set of headers to indicate after healing that this is an inlined data.

Copy link
Contributor

@kannappanr kannappanr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested. LGTM

@harshavardhana harshavardhana merged commit a6e0ec4 into minio:master Aug 3, 2022
@harshavardhana harshavardhana deleted the add-recreate branch August 3, 2022 06:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants