Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add FragmentInstanceStateMachine for FragmentInstance State change #5615

Merged
merged 10 commits into from
Apr 22, 2022

Conversation

JackieTien97
Copy link
Contributor

@JackieTien97 JackieTien97 commented Apr 20, 2022

  1. Let SinkHandle notify FragmentInstanceStateMachine to change to FAILED state while meeting errors in async send event.
  2. Rename Driver to IDriver and add another new Driver abstract class, extract duplicated code from DataDriver and SchemaDriver to their parent class Driver.
  3. Add FragmentInstanceStateMachine to manage FragmentInstance State change
  4. Add Inner Exclusive Lock called DriverLock in Driver to prevent it from closing by others while it is still running.

@@ -58,10 +64,51 @@ public class FragmentInstanceContext extends QueryContext {
// private final AtomicLong endFullGcCount = new AtomicLong(-1);
// private final AtomicLong endFullGcTimeNanos = new AtomicLong(-1);

public FragmentInstanceContext(
FragmentInstanceId id, AtomicReference<FragmentInstanceState> state) {
public static FragmentInstanceContext createFragmentInstance(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

createFragmentInstanceContext

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix

CounterStat failedInstances) {
FragmentInstanceExecution execution =
new FragmentInstanceExecution(scheduler, instanceId, context, driver, stateMachine);
execution.initialize(failedInstances, scheduler, instanceId, driver);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is better understood if failedInstances is changed to another name.

scheduler instanceId driver may be duplicated because they have been passed into constructor

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix

@@ -148,6 +162,19 @@ public FragmentInstanceInfo abortFragmentInstance(FragmentInstanceId fragmentIns
return null;
}

/** Cancels a FragmentInstance. */
public FragmentInstanceInfo cancelTask(FragmentInstanceId instanceId) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that abortFragmentInstance() do the same thing with cancelTask

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, cancel is called by user kill, abort is called by other FIs' failures in current query.

@coveralls
Copy link

coveralls commented Apr 20, 2022

Coverage Status

Coverage increased (+0.03%) to 54.138% when pulling 2fed1f0 on stable-mpp into b2520b3 on master.

@sonarcloud
Copy link

sonarcloud bot commented Apr 21, 2022

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 25 Code Smells

46.3% 46.3% Coverage
0.0% 0.0% Duplication

@JackieTien97 JackieTien97 merged commit f9226c3 into master Apr 22, 2022
@JackieTien97 JackieTien97 deleted the stable-mpp branch April 22, 2022 01:23
xinzhongtianxia pushed a commit to xinzhongtianxia/iotdb that referenced this pull request Apr 24, 2022
* remotes/upstream/master:
  Serialize measurement schema of insert node to wal entry (apache#5638)
  filter non schemaRegionDir (apache#5640)
  [IOTDB-2976] Add English and Chinese docs for count devices and count storage groups (apache#5635)
  change jenkins timeout from 2 hours to 3 hours
  [IOTDB-2740] Equal size bucket sampling UDFs: EQUAL_SIZE_BUCKET_RANDOM_SAMPLE, EQUAL_SIZE_BUCKET_AGG_SAMPLE, EQUAL_SIZE_BUCKET_M4_SAMPLE (apache#5518)
  Fix the issue that EndTime in FragmentInstanceContext is not set (apache#5636)
  fix concurrent bug of CachedMNodeContainer.putIfAbsent (apache#5632)
  [IOTDB-2880] Fix NPE occured in ci test (apache#5634)
  Fix CI (apache#5639)
  Add ColumnMerger to merge multipul input columns of same sensor into one column (apache#5630)
  Add block cancel when GetBlockTask throws exception (apache#5628)
  fix the bug when matching multi-wildcard in pattern tree (apache#5631)
  [IOTDB-2835]Fix empty page in selfcheck method of TsFileSequenceReader (apache#5552)
  Add FragmentInstanceStateMachine for FragmentInstance State change (apache#5615)
  [IOTDB-2880] Fix import check style (apache#5629)
  [IOTDB-2971] Fix sink handle memory leak (apache#5626)
  [rocksdb] updated the interface support (apache#5625)
  [IOTDB-2970] Code style: Avoid wildcard imports (apache#5622)
  [IOTDB-2880]Add procedure framework (apache#5477)
  [rocksdb] add rocksdb properties (apache#5588)

# Conflicts:
#	server/src/main/java/org/apache/iotdb/db/mpp/sql/planner/LocalExecutionPlanner.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants