[IOTDB-188] Delete storage group by nlosilva1 · Pull Request #416 · apache/iotdb

nlosilva1 · 2019-09-24T04:19:58Z

Complete a function using session to delete storage groups in iotdb. Delete all the data of this storage group after calling this function.

Jira link: https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-188

qiaojialin · 2019-09-24T05:01:01Z

server/src/main/java/org/apache/iotdb/db/engine/StorageEngine.java

    return true;
  }

+  public void rmStorageGroupInfo(String storageGroupName) {


rename to deleteStorageGroup(), call deleteAllDataFilesInOneStorageGroup() first in this method.

qiaojialin · 2019-09-24T05:51:44Z

server/src/main/java/org/apache/iotdb/db/engine/storagegroup/StorageGroupProcessor.java

+    waitForAllCurrentTsFileProcessorsClosed();
+    writeLock();
+    try {
+      File storageGroupFolder = TSFileFactory.INSTANCE.getFile(systemDir, storageGroupName);


This may be the SystemFactory

qiaojialin · 2019-09-24T05:52:37Z

server/src/main/java/org/apache/iotdb/db/metadata/MManager.java

      case MetadataOperationType.SET_STORAGE_LEVEL_TO_MTREE:
        setStorageLevelToMTree(args[1]);
        break;
+      case MetadataOperationType.DELETE_STORAGE_LEVEL_TO_MTREE:


Suggested change

case MetadataOperationType.DELETE_STORAGE_LEVEL_TO_MTREE:

case MetadataOperationType.DELETE_STORAGE_GROUP_FROM_MTREE:

qiaojialin · 2019-09-24T05:52:50Z

server/src/main/java/org/apache/iotdb/db/metadata/MManager.java

@@ -212,6 +212,9 @@ private void operation(String cmd)
      case MetadataOperationType.SET_STORAGE_LEVEL_TO_MTREE:


Suggested change

case MetadataOperationType.SET_STORAGE_LEVEL_TO_MTREE:

case MetadataOperationType.SET_STORAGE_GROUP_TO_MTREE:

@qiaojialin Do you want to modify all STORAGE_LEVEL to STORAGE_GROUP? If in that case, the author needs to check all naming.

qiaojialin · 2019-09-24T05:53:16Z

server/src/main/java/org/apache/iotdb/db/metadata/MManager.java

+  /**
+   * function for deleting storage level of the given path to mTree.
+   */
+  public boolean deleteStorageLevelToMTree(String path) throws MetadataErrorException {


Suggested change

public boolean deleteStorageLevelToMTree(String path) throws MetadataErrorException {

public boolean deleteStorageGroupFromMTree(String path) throws MetadataErrorException {

qiaojialin · 2019-09-24T05:57:25Z

service-rpc/src/main/thrift/rpc.thrift

-  TSStatus deleteTimeseries(1:list<string> path)
+    TSStatus deleteTimeseries(1:list<string> path)
+
+    TSStatus deleteStorageGroup(1:TSDeleteDataReq req);


Suggested change

TSStatus deleteStorageGroup(1:TSDeleteDataReq req);

TSStatus deleteStorageGroup(1:string storageGroup);

I think it is best if you make it a list.

qiaojialin · 2019-09-24T05:57:42Z

session/src/main/java/org/apache/iotdb/session/Session.java

    }
  }

+  public synchronized TSStatus deleteStorageGroup(String storageGroupId, long time) throws IoTDBSessionException {


Suggested change

public synchronized TSStatus deleteStorageGroup(String storageGroupId, long time) throws IoTDBSessionException {

public synchronized TSStatus deleteStorageGroup(String storageGroupId) throws IoTDBSessionException {

qiaojialin · 2019-09-24T05:58:02Z

session/src/main/java/org/apache/iotdb/session/Session.java

+    List<String> paths = new ArrayList<>();
+    paths.add(storageGroupId);
+    request.setPaths(paths);
+    request.setTimestamp(time);


remove this 5 lines

qiaojialin · 2019-09-24T05:58:13Z

session/src/main/java/org/apache/iotdb/session/Session.java

+    request.setPaths(paths);
+    request.setTimestamp(time);
+    try {
+      return checkAndReturn(client.deleteStorageGroup(request));


Suggested change

return checkAndReturn(client.deleteStorageGroup(request));

return checkAndReturn(client.deleteStorageGroup(storageGroup));

qiaojialin · 2019-09-24T05:58:45Z

session/src/test/java/org/apache/iotdb/session/IoTDBSessionIT.java

    }
  }
+
+  public void deleteStorageGroupTest() throws ClassNotFoundException, SQLException, IoTDBSessionException {


Suggested change

public void deleteStorageGroupTest() throws ClassNotFoundException, SQLException, IoTDBSessionException {

public void deleteStorageGroup() throws ClassNotFoundException, SQLException, IoTDBSessionException {

server/src/main/java/org/apache/iotdb/db/metadata/MetadataOperationType.java

server/src/main/java/org/apache/iotdb/db/metadata/MGraph.java

fanhualta · 2019-09-24T06:49:01Z

server/src/main/java/org/apache/iotdb/db/metadata/MManager.java

@@ -212,6 +212,9 @@ private void operation(String cmd)
      case MetadataOperationType.SET_STORAGE_LEVEL_TO_MTREE:


@qiaojialin Do you want to modify all STORAGE_LEVEL to STORAGE_GROUP? If in that case, the author needs to check all naming.

server/src/main/java/org/apache/iotdb/db/metadata/MManager.java

server/src/main/java/org/apache/iotdb/db/service/TSServiceImpl.java

jt2594838

Great but with some small issues.

server/src/main/java/org/apache/iotdb/db/engine/StorageEngine.java

server/src/main/java/org/apache/iotdb/db/metadata/MTree.java

server/src/main/java/org/apache/iotdb/db/service/TSServiceImpl.java

jt2594838 · 2019-09-24T07:17:34Z

server/src/main/java/org/apache/iotdb/db/service/TSServiceImpl.java

+      logger.info(INFO_NOT_LOGIN, IoTDBConstant.GLOBAL_DB_NAME);
+      return new TSStatus(getStatus(TSStatusCode.NOT_LOGIN_ERROR));
+    }
+    MetadataPlan plan = new MetadataPlan(MetadataOperator.NamespaceType.DELETE_STORAGE_GROUP,new Path(req.getPaths().get(0)));


So only one storage group is supported at the same time? I guess it is not difficult to support deleting multiple groups with one request, try if you can.

As the issue defined in IOTDB-188, I just need to delete a single storage group at the same time, and Qiaojialin said to me that before we complete the batch deletion of storage groups, we should change some of the structure of the Metadata, so he suggested me to submit this version first.

You can always go beyond the issue as long as it is helpful, but it is fine to leave it as future work.

server/src/main/java/org/apache/iotdb/db/sync/receiver/SyncServiceImpl.java

jt2594838 · 2019-09-24T07:19:43Z

service-rpc/src/main/thrift/rpc.thrift

-  TSStatus deleteTimeseries(1:list<string> path)
+    TSStatus deleteTimeseries(1:list<string> path)
+
+    TSStatus deleteStorageGroup(1:TSDeleteDataReq req);


I think it is best if you make it a list.

jt2594838 · 2019-09-24T07:24:35Z

Rename the title to "[IOTDB-XXX]..." and provide the Jira link, please.

…e requested changes

nlosilva1 added 4 commits September 24, 2019 11:07

add simple constants and function framework

dd66dc9

fix some bugs in merging

9f858c6

Complete the deletion of the storage groups

f23d3e8

Fix some bugs about interface and test after merging

508b102

qiaojialin requested changes Sep 24, 2019

View reviewed changes

fanhualta requested changes Sep 24, 2019

View reviewed changes

Commit the suggestions except for the rename

e9b3c68

jt2594838 requested changes Sep 24, 2019

View reviewed changes

nlosilva1 changed the title ~~Delete storage group in session~~ [IOTDB-188] Delete storage group Sep 24, 2019

Rename 'storageLevel' as 'storageGroup'in the project, and finish som…

b85fac6

…e requested changes

nlosilva1 requested a review from jt2594838 September 24, 2019 11:14

jt2594838 approved these changes Sep 25, 2019

View reviewed changes

fanhualta approved these changes Sep 25, 2019

View reviewed changes

Rewrite the Session log, add the deletion of a list of storage groups

b56abdf

nlosilva1 requested review from jt2594838 and qiaojialin September 26, 2019 02:23

qiaojialin approved these changes Sep 26, 2019

View reviewed changes

qiaojialin merged commit 59f66f6 into apache:master Sep 26, 2019

	case MetadataOperationType.DELETE_STORAGE_LEVEL_TO_MTREE:
	case MetadataOperationType.DELETE_STORAGE_GROUP_FROM_MTREE:

		@@ -212,6 +212,9 @@ private void operation(String cmd)
		case MetadataOperationType.SET_STORAGE_LEVEL_TO_MTREE:

	case MetadataOperationType.SET_STORAGE_LEVEL_TO_MTREE:
	case MetadataOperationType.SET_STORAGE_GROUP_TO_MTREE:

	public boolean deleteStorageLevelToMTree(String path) throws MetadataErrorException {
	public boolean deleteStorageGroupFromMTree(String path) throws MetadataErrorException {

	TSStatus deleteStorageGroup(1:TSDeleteDataReq req);
	TSStatus deleteStorageGroup(1:string storageGroup);

	public synchronized TSStatus deleteStorageGroup(String storageGroupId, long time) throws IoTDBSessionException {
	public synchronized TSStatus deleteStorageGroup(String storageGroupId) throws IoTDBSessionException {

	return checkAndReturn(client.deleteStorageGroup(request));
	return checkAndReturn(client.deleteStorageGroup(storageGroup));

	public void deleteStorageGroupTest() throws ClassNotFoundException, SQLException, IoTDBSessionException {
	public void deleteStorageGroup() throws ClassNotFoundException, SQLException, IoTDBSessionException {

Conversation

nlosilva1 commented Sep 24, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jt2594838 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jt2594838 Sep 24, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jt2594838 commented Sep 24, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

nlosilva1 commented Sep 24, 2019 •

edited

Loading

jt2594838 Sep 24, 2019 •

edited

Loading

jt2594838 commented Sep 24, 2019 •

edited

Loading