fix: change the way object metadata is stored in posix#1920
Merged
benmcclelland merged 1 commit intomainfrom Mar 6, 2026
Merged
fix: change the way object metadata is stored in posix#1920benmcclelland merged 1 commit intomainfrom
benmcclelland merged 1 commit intomainfrom
Conversation
Fixes #1909 Previously, the mapping between object metadata and posix object was as follows: for each metadata key, we stored a separate xattr with the `user.X-Amz-Meta.<key>` prefix. This resulted in syscall overhead when storing and deleting large numbers of metadata keys. In addition, very long metadata keys caused failures because most posix filesystems limit xattr key lengths to 127–255 bytes, while S3 does not enforce such a per-key limit. The logic has now been changed so that all object metadata is stored in a single xattr, `user.metadata`, as a JSON key/value object. For backward compatibility, metadata GET operations still fall back to the old mechanism (`metadata key -> xattr key`) when `user.metadata` is not present. A new CLI utility has been added to convert all legacy object metadata to the new metadata format within the provided directory. **Example usage:** ``` versitygw utils convert-xattr-metadata path/to/bucket ``` or ``` versitygw utils cxm path/to/bucket ``` It is recommended to run this command on bucket directories to convert all legacy metadata for every object in the bucket.
niksis02
force-pushed
the
sis/posix-object-metadata
branch
from
c282667 to
97bb705
Compare
benmcclelland
approved these changes
Mar 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1909
Previously, the mapping between object metadata and posix object was as follows: for each metadata key, we stored a separate xattr with the
user.X-Amz-Meta.<key>prefix. This resulted in syscall overhead when storing and deleting large numbers of metadata keys. In addition, very long metadata keys caused failures because most posix filesystems limit xattr key lengths to 127–255 bytes, while S3 does not enforce such a per-key limit.The logic has now been changed so that all object metadata is stored in a single xattr,
user.metadata, as a JSON key/value object. For backward compatibility, metadata GET operations still fall back to the old mechanism (metadata key -> xattr key) whenuser.metadatais not present.A new CLI utility has been added to convert all legacy object metadata to the new metadata format within the provided directory.
Example usage:
or
It is recommended to run this command on bucket directories to convert all legacy metadata for every object in the bucket.