Sync on S3 can set the owner to "" or "_" #11266
Labels
type-bug
This issue is about a bug
Comments
alluxio-bot
pushed a commit
that referenced
this issue
Apr 10, 2020
This PR includes 2 changes: - For S3, when `inherit_acl=false` any inodes created during loadmetadata should inherit owner/group from the parent directory in alluxio instead of being empty. This was broken because `setOwner` was called after `inheritOwnerAndGroupIfEmpty` in `InodeTree::createPath` - When syncing metadata with s3, an `updateMetadata` sync plan could lead to setting the owner and group to `_` in `DefaultFileSystemMaster::setAttributeSingleFile`. The fix is to not set the options in `SetAttributePOptions` to avoid setting to an empty string or `_`. This can be reproduced when a directory is created in alluxio only, the same directory is then later created in the ufs and then synced. Steps to reproduce: ``` $ aws s3 ls s3://adit-bucket-west/ PRE / $ afs mkdir /adit Successfully created directory /adit $ afs mount /adit/s3a s3a://adit-bucket-west/ Mounted s3a://adit-bucket-west/ at /adit/s3a $ afs mkdir /adit/s3a/nested Successfully created directory /adit/s3a/nested $ aws s3 cp world s3://adit-bucket-west/nested/file upload: ./world to s3://adit-bucket-west/nested/file $ afs ls -Dalluxio.user.file.metadata.sync.interval=0 /adit/s3a/nested -rwx------ _ _ 6 PERSISTED 04-09-2020 22:22:18:000 0% /adit/s3a/nested/file $ afs ls /adit/s3a/ drwx------ _ _ 1 PERSISTED 04-09-2020 22:22:18:000 DIR /adit/s3a/nested ``` Fixes: #11265, #11266 pr-link: #11257 change-id: cid-0407f53045c1d2d6f2342b6b62c8247d7d7ad9b2
alluxio-bot
pushed a commit
that referenced
this issue
Apr 10, 2020
This PR includes 2 changes: - For S3, when `inherit_acl=false` any inodes created during loadmetadata should inherit owner/group from the parent directory in alluxio instead of being empty. This was broken because `setOwner` was called after `inheritOwnerAndGroupIfEmpty` in `InodeTree::createPath` - When syncing metadata with s3, an `updateMetadata` sync plan could lead to setting the owner and group to `_` in `DefaultFileSystemMaster::setAttributeSingleFile`. The fix is to not set the options in `SetAttributePOptions` to avoid setting to an empty string or `_`. This can be reproduced when a directory is created in alluxio only, the same directory is then later created in the ufs and then synced. Steps to reproduce: ``` $ aws s3 ls s3://adit-bucket-west/ PRE / $ afs mkdir /adit Successfully created directory /adit $ afs mount /adit/s3a s3a://adit-bucket-west/ Mounted s3a://adit-bucket-west/ at /adit/s3a $ afs mkdir /adit/s3a/nested Successfully created directory /adit/s3a/nested $ aws s3 cp world s3://adit-bucket-west/nested/file upload: ./world to s3://adit-bucket-west/nested/file $ afs ls -Dalluxio.user.file.metadata.sync.interval=0 /adit/s3a/nested -rwx------ _ _ 6 PERSISTED 04-09-2020 22:22:18:000 0% /adit/s3a/nested/file $ afs ls /adit/s3a/ drwx------ _ _ 1 PERSISTED 04-09-2020 22:22:18:000 DIR /adit/s3a/nested ``` Fixes: #11265, #11266 pr-link: #11257 change-id: cid-0407f53045c1d2d6f2342b6b62c8247d7d7ad9b2
alluxio-bot
pushed a commit
that referenced
this issue
Apr 11, 2020
This PR includes 2 changes: For S3, when inherit_acl=false any inodes created during loadmetadata should inherit owner/group from the parent directory in alluxio instead of being empty. This was broken because setOwner was called after inheritOwnerAndGroupIfEmpty in InodeTree::createPath When syncing metadata with s3, an updateMetadata sync plan could lead to setting the owner and group to _ in DefaultFileSystemMaster::setAttributeSingleFile. The fix is to not set the options in SetAttributePOptions to avoid setting to an empty string or _. This can be reproduced when a directory is created in alluxio only, the same directory is then later created in the ufs and then synced. Steps to reproduce: $ aws s3 ls s3://adit-bucket-west/ PRE / $ afs mkdir /adit Successfully created directory /adit $ afs mount /adit/s3a s3a://adit-bucket-west/ Mounted s3a://adit-bucket-west/ at /adit/s3a $ afs mkdir /adit/s3a/nested Successfully created directory /adit/s3a/nested $ aws s3 cp world s3://adit-bucket-west/nested/file upload: ./world to s3://adit-bucket-west/nested/file $ afs ls -Dalluxio.user.file.metadata.sync.interval=0 /adit/s3a/nested -rwx------ _ _ 6 PERSISTED 04-09-2020 22:22:18:000 0% /adit/s3a/nested/file $ afs ls /adit/s3a/ drwx------ _ _ 1 PERSISTED 04-09-2020 22:22:18:000 DIR /adit/s3a/nested Fixes: #11265, #11266 [This is a manual PR to cherry-pick committed PR #11257 into target branch branch-2.8] pr-link: #11268 change-id: cid-6f5934c9be3c5c45d4be4175d0b92db161a87acc
|
This has been closed by #11257 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Alluxio Version:
2.2
Describe the bug
S3 does not set owner/group for objects when
inherit_acl=false. When syncing metadata with s3, anupdateMetadatasync plan could lead to setting the owner and group to_inDefaultFileSystemMaster::setAttributeSingleFileor to "" (empty) when updating inode entry during syncTo Reproduce
This can be reproduced when a directory is created in alluxio only, the same directory is then later created in the ufs and then synced.
Steps to reproduce:
Expected behavior
Owner and group inherited from parent dir previously should not be modified.
Urgency
Urgent
Additional context
N/A
The text was updated successfully, but these errors were encountered: