Fix empty tsfile generated when all datatypes in one insert mismatching

[HTHou]

Description

If all the data types in the insertPlan are inconsistent with the registered metadata, memtable will consider all the columns written to fail when calculating the total number of points. In this case, totalPointNum will not increase and will remain 0. However, the calculation of memSize does not consider the case of fail. So the memSize will increase.

After flush, the system determines whether to delete the TsFile according to whether the tsfile is empty.

According to the previous write scenario, totalMemtableSize is not 0 and workMemTable is null. Therefore, the TsFile is considered not empty, resulting in that the TsFile is not deleted.

To fix it, before entering the TsFileProcessor, check that all data types in the write request do not match the registered metadata, and if they do, simply return and do not perform the subsequent write process.

How to test

1. create timeseries by sql create aligned timeseries root.sg1.d1(s1 boolean, s2 boolean, s3 boolean)
2. execute the following code to insert data and flush

  public static void main(String[] args)
      throws IoTDBConnectionException, StatementExecutionException {
    session =
        new Session.Builder()
            .host(LOCAL_HOST)
            .port(6667)
            .username("root")
            .password("root")
            .version(Version.V_1_0)
            .build();
    session.open(false);
    insertTablet1();
    session.executeNonQueryStatement("flush");
    session.close();
  }


  private static void insertTablet1() throws IoTDBConnectionException, StatementExecutionException {
    // The schema of measurements of one device
    // only measurementId and data type in MeasurementSchema take effects in Tablet
    List<MeasurementSchema> schemaList = new ArrayList<>();
    schemaList.add(new MeasurementSchema("s1", TSDataType.INT64));
    schemaList.add(new MeasurementSchema("s2", TSDataType.INT64));
    schemaList.add(new MeasurementSchema("s3", TSDataType.INT64));

    Tablet tablet = new Tablet(ROOT_SG1_D1, schemaList, 100);

    // Method 1 to add tablet data
    long timestamp = System.currentTimeMillis();

    for (long row = 0; row < 100; row++) {
      int rowIndex = tablet.rowSize++;
      tablet.addTimestamp(rowIndex, timestamp);
      for (int s = 0; s < 3; s++) {
        tablet.addValue(schemaList.get(s).getMeasurementId(), rowIndex, 1L);
      }
      if (tablet.rowSize == tablet.getMaxRowNumber()) {
        session.insertTablet(tablet, true);
        tablet.reset();
      }
      timestamp++;
    }

    if (tablet.rowSize != 0) {
      session.insertTablet(tablet);
      tablet.reset();
    }
  }

3. check if there are some empty TsFile generated.

[sonarqubecloud]

SonarCloud Quality Gate failed.   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

0.0% Coverage
0.0% Duplication

Catch issues before they fail your Quality Gate with our IDE extension SonarLint