[WIP] Syncservice rebased onto HDFS-12090 #1726
Conversation
|
|
||
| import static org.apache.hadoop.hdfs.server.common.HdfsServerConstants.XATTR_SYNC_TO_EXTERNAL; | ||
|
|
||
| final class FSDirSyncNodeToRemoteStoreOp { |
There was a problem hiding this comment.
All these methods here seem to be doing the same thing? Will these methods be called by the SPS?
| } | ||
| INode inode = FSDirectory.resolveLastINode(iip); | ||
| if (inode.isFile() && inode.asFile().numBlocks() == 0) { | ||
| if (NameNode.LOG.isInfoEnabled()) { |
There was a problem hiding this comment.
Empty files would also have to be synced.
| + inode.getFullPathName() | ||
| + ", as this file/dir was already called for sync."); | ||
| } else { | ||
| XAttr syncToExternalXAttr = XAttrHelper |
There was a problem hiding this comment.
Call FSDirXAttrOp.setXAttr. It does all this for you.
| @@ -368,6 +368,7 @@ public void write(DataOutput out) throws IOException { | |||
| "system.hdfs.erasurecoding.policy"; | |||
|
|
|||
| String XATTR_SATISFY_STORAGE_POLICY = "user.hdfs.sps"; | |||
| String XATTR_SYNC_TO_EXTERNAL = "user.hdfs.sync"; | |||
There was a problem hiding this comment.
How is this XAttr used? I currently see this accessed only in FSDirSyncNodeToRemoteStoreOp
…locks. Original patch contributed by Ewan Higgs. Followup work and fixed contributed by Virajith Jalaparthi.
…de. Contributed by Ewan Higgs.
…od. Contributed by Ewan Higgs
ehiggs
force-pushed
the
syncservice-rebased
branch
from
26594f8
to
59803c2
Compare
|
Hi @ehiggs @virajith, any news on the state of sync service and HDFS-12090? |
|
Hi, @qboileau, based on this patch, we had developed provided storage sync write feature further. Now, the patch is almost in good shape. And we will post the patch to JIRA, expected in this month. |
|
Based on this patch from @ehiggs, we developed the feature to support provided storage write. Please review our patch in https://issues.apache.org/jira/browse/HDFS-15714. Thanks @ehiggs so much! |
@virajith @ashvina
This is a rebase of the sync service for HDFS-12090 rebased onto the current branch.