Skip to content

Conversation

@Wei-hao-Li
Copy link
Collaborator

1.Transfer stateful triggers when DataNode Status is Unknown;
2.Transfer stateful triggers when DataNode remove;
3.Has been tested, can see Trigger测试方案
4.Cooperate with @lancelly.

Wei-hao-Li and others added 11 commits October 14, 2022 16:00
Signed-off-by: Weihao Li <18110526956@163.com>
Signed-off-by: Weihao Li <18110526956@163.com>
Signed-off-by: Weihao Li <18110526956@163.com>
Signed-off-by: Weihao Li <18110526956@163.com>
Signed-off-by: Weihao Li <18110526956@163.com>
Signed-off-by: Weihao Li <18110526956@163.com>
…ss of RemovingNode

Signed-off-by: Weihao Li <18110526956@163.com>
Signed-off-by: Weihao Li <18110526956@163.com>
Copy link
Contributor

@CRZbulabula CRZbulabula left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL~

Comment on lines +260 to +267
// do transfer of the DataNodes before remove
if (configManager.transfer(removeDataNodePlan.getDataNodeLocations()).getCode()
!= TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
dataSet.setStatus(
new TSStatus(TSStatusCode.NODE_DELETE_FAILED_ERROR.getStatusCode())
.setMessage("Fail to do transfer of the DataNodes"));
return dataSet;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the transfer process should be such important that the DataNode remove process couldn't continue as long as if it fails down?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think we should ensure the safety of data.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove data process used RemoveDataProcedure and sub RegionMigrateProcedures,
Will triggers implement it later or using other methods?
Or some errors may happen when ConfigNode down/restart.

image

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We used to consider implement by Procedure, but later we found that we did not need to roll back when the transfer failed.

Copy link
Contributor

@CRZbulabula CRZbulabula left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Signed-off-by: Weihao Li <18110526956@163.com>
@JackieTien97 JackieTien97 merged commit 127263d into apache:master Oct 19, 2022
* @return result of transferTrigger
*/
public TSStatus transferTrigger(
List<TDataNodeLocation> newUnknownDataList,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
List<TDataNodeLocation> newUnknownDataList,
List<TDataNodeLocation> newUnknownDataNodeList,


public List<TSStatus> updateTriggerLocation(
String triggerName,
TDataNodeLocation dataNodeLocation,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parameter dataNodeLocation and dataNodeLocationMap are similar, maybe we can change dataNodeLocation to targetDataNode?

Comment on lines +260 to +267
// do transfer of the DataNodes before remove
if (configManager.transfer(removeDataNodePlan.getDataNodeLocations()).getCode()
!= TSStatusCode.SUCCESS_STATUS.getStatusCode()) {
dataSet.setStatus(
new TSStatus(TSStatusCode.NODE_DELETE_FAILED_ERROR.getStatusCode())
.setMessage("Fail to do transfer of the DataNodes"));
return dataSet;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove data process used RemoveDataProcedure and sub RegionMigrateProcedures,
Will triggers implement it later or using other methods?
Or some errors may happen when ConfigNode down/restart.

image

@Wei-hao-Li Wei-hao-Li deleted the TriggerTransfer branch October 20, 2022 04:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants