-
Notifications
You must be signed in to change notification settings - Fork 974
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[IOTDB-1079] Virtual storage group first mile stone #2405
Conversation
…_partition # Conflicts: # server/src/main/java/org/apache/iotdb/db/engine/StorageEngine.java
…_partition # Conflicts: # server/src/main/java/org/apache/iotdb/db/engine/StorageEngine.java
…_partition_2_merge # Conflicts: # server/src/main/java/org/apache/iotdb/db/conf/IoTDBConfig.java # server/src/main/java/org/apache/iotdb/db/engine/StorageEngine.java
…_partition_2_merge
…_partition_2_merge
…_partition_2_merge
…_partition_2_merge
…_partition_2_merge
…_partition_2_merge
…_partition_2_merge
…_partition_2_merge # Conflicts: # server/src/main/java/org/apache/iotdb/db/engine/StorageEngine.java
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A great job, but I believe you can make it even better.
server/src/main/java/org/apache/iotdb/db/engine/StorageEngine.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/apache/iotdb/db/engine/StorageEngine.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/apache/iotdb/db/engine/StorageEngine.java
Outdated
Show resolved
Hide resolved
server/src/test/java/org/apache/iotdb/db/integration/IoTDBLoadExternalTsfileIT.java
Show resolved
Hide resolved
server/src/test/java/org/apache/iotdb/db/integration/IoTDBRemovePartitionIT.java
Outdated
Show resolved
Hide resolved
server/src/test/java/org/apache/iotdb/db/integration/IoTDBRestartIT.java
Show resolved
Hide resolved
server/src/test/java/org/apache/iotdb/db/sync/receiver/load/FileLoaderTest.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/apache/iotdb/db/engine/storagegroup/virtualSg/VirtualPartitioner.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/apache/iotdb/db/engine/storagegroup/StorageGroupProcessor.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/apache/iotdb/db/engine/storagegroup/StorageGroupProcessor.java
Outdated
Show resolved
Hide resolved
…_partition_2_merge # Conflicts: # server/src/main/java/org/apache/iotdb/db/engine/storagegroup/StorageGroupProcessor.java
server/src/main/java/org/apache/iotdb/db/engine/StorageEngine.java
Outdated
Show resolved
Hide resolved
logger.info("{} submit a compaction merge task", virtualstorageGroupName); | ||
logger.info("{} submit a compaction merge task", virtualstorageGroupId); | ||
try { | ||
CompactionMergeTaskPoolManager.getInstance() | ||
.submitTask( | ||
tsFileManagement.new CompactionRecoverTask(this::closeCompactionMergeCallBack)); | ||
} catch (RejectedExecutionException e) { | ||
this.closeCompactionMergeCallBack(); | ||
logger.error("{} compaction submit task failed", virtualstorageGroupName); | ||
logger.error("{} compaction submit task failed", virtualstorageGroupId); | ||
} | ||
} else { | ||
logger.error("{} compaction pool not started ,recover failed", | ||
virtualstorageGroupName); | ||
virtualstorageGroupId); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Storage group name is still missing in these logs. And please also check other places.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, I have fixed all the logs~
Kudos, SonarCloud Quality Gate passed!
|
Description
Currently out concurrency unit is storage group which can't make full use of cpu cores when user only set 1 or 2 storage group. We need virtual storage group to improve concurrent performance. This PR finish first part of the design.
JIRA:
https://issues.apache.org/jira/projects/IOTDB/issues/IOTDB-1079?filter=myopenissues
Design: https://cwiki.apache.org/confluence/display/IOTDB/Separate+memory+and+disk+management+in+storage+group+road+map+design
Performance report:
https://tower.im/teams/138444/todos/16237/
This PR has:
for an unfamiliar reader.
for code coverage.
Key changed/added classes (or packages if there are too many classes) in this PR