Fix TsFileResource endTime error when insert empty tablet and flush

[HTHou]

Description

When data is written using the insertTablet() interface and both values are null, exactly the data written is flushed. An empty TsFile file will be generated, the space cannot be deleted, and the generated resource file will be incorrect, and the endTime of the device will be Long.MIN_VALUE. This in turn affects query and merge mechanisms.

Based on this, this PR has the following two optimizations:

1. In the flush process, it is added that if the resource file is empty, the generated TsFile and.resource file are deleted
2. Enhance the judgment logic when there is no valid data in MemTable. If there is no valid data, delete the related empty Device data from the resource.

How to test

Execute the following code.

  public static void main(String[] args)
      throws IoTDBConnectionException, StatementExecutionException {
    session =
        new Session.Builder()
            .host(LOCAL_HOST)
            .port(6667)
            .username("root")
            .password("root")
            .version(Version.V_1_0)
            .build();
    session.open(false);
    insertTablet();
    session.executeNonQueryStatement("flush");
    session.close();
  }


  private static void insertTablet() throws IoTDBConnectionException, StatementExecutionException {
    // The schema of measurements of one device
    // only measurementId and data type in MeasurementSchema take effects in Tablet
    List<MeasurementSchema> schemaList = new ArrayList<>();
    schemaList.add(new MeasurementSchema("s1", TSDataType.INT64));
    schemaList.add(new MeasurementSchema("s2", TSDataType.INT64));
    schemaList.add(new MeasurementSchema("s3", TSDataType.INT64));

    Tablet tablet = new Tablet(ROOT_SG1_D1, schemaList, 100);

    // Method 1 to add tablet data
    long timestamp = System.currentTimeMillis();

    for (long row = 0; row < 100; row++) {
      int rowIndex = tablet.rowSize++;
      tablet.addTimestamp(rowIndex, timestamp);
      for (int s = 0; s < 3; s++) {
        tablet.addValue(schemaList.get(s).getMeasurementId(), rowIndex, null);
      }
      if (tablet.rowSize == tablet.getMaxRowNumber()) {
        session.insertTablet(tablet, true);
        tablet.reset();
      }
      timestamp++;
    }

    if (tablet.rowSize != 0) {
      session.insertTablet(tablet);
      tablet.reset();
    }
  }

Then print the TsFileResource file, you will see the endTime is Long.MIN_VALUE.

[wangchao316]

hi , tsfile exsit and resource does not exsit, will delete tsfile?

[HTHou]

In this case, tsfile contains no data and resource file exists.
I'll update the PR description later.

[wangchao316]

ok

[sonarqubecloud]

SonarCloud Quality Gate failed.   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

0.0% Coverage
0.0% Duplication

Catch issues before they fail your Quality Gate with our IDE extension SonarLint

[wangchao316]

LGTM

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 48.59%. Comparing base (e0271e5) to head (856173d).
⚠️ Report is 3721 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #11404      +/-   ##
============================================
+ Coverage     48.58%   48.59%   +0.01%     
- Complexity    24434    24454      +20     
============================================
  Files          2817     2817              
  Lines        173009   173012       +3     
  Branches      20662    20663       +1     
============================================
+ Hits          84049    84072      +23     
+ Misses        88960    88940      -20     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.