[IOTDB-23] Restart state monitor module

[Alima777]

You can find the detail in [1].
[1] https://lists.apache.org/list.html?dev@iotdb.apache.org:lte=1M:Restart

Currently, the status monitor module of iotdb is abandoned and not maintained now. It's time for us to restart this module.

Before, it was designed including two child modules: 1. the writing data status monitor and 2. the file size monitor.

1. The writing data monitor module is responsible for collecting writing data statistics.

Actually, many parts of origin design are redundant and not meaningful for users. Therefore, I want to just keep TOTAL_POINTS here and remove others. And i did not come up with some good ideas about what needs to be added to this monitor while writing data. Welcome to any ideas here!

2. The file size monitor is concerned about how much space the data file takes. It's supported partly and very limited now.

I want to just remove it from iotdb.

I used the benchmark to do the performance test, using the default IoTDB configuration while 8G machine memoery , the benchmark configuration is:
----------------------Main Configurations----------------------
DB_SWITCH: IoTDB
OPERATION_PROPORTION: 1:0:0:0:0:0:0:0:0
ENABLE_THRIFT_COMPRESSION: false
INSERT_DATATYPE_PROPORTION: 1:1:1:1:1:1
ENCODING(BOOLEAN/INT32/INT64/FLOAT/DOUBLE/TEXT): PLAIN/PLAIN/PLAIN/PLAIN/PLAIN/PLAIN
IS_CLIENT_BIND: true
CLIENT_NUMBER: 20
GROUP_NUMBER: 10
DEVICE_NUMBER: 10
SENSOR_NUMBER: 100
BATCH_SIZE: 100
LOOP: 100000
POINT_STEP: 5000
QUERY_INTERVAL: 250000
IS_OVERFLOW: false
OVERFLOW_MODE: 0
OVERFLOW_RATIO: 0.5

The result is:

| Close monitor | Open monitor without writing timeseries | Open Monitor with writing time series |
| 1048.83 ms | 1049.40ms | 1064.90ms |
| | - 0.05% | -3% |

[jixuan1989]

I think some logic should refined:

We need to consider the case that the status of enableMonitor can be switched while IoTDB instance does not restart.
So, the recovery() and init() (create timeseries) should be called by StatMonitor itself.

[HTHou]

I think a more clear user guide is necessary for this module.

[HTHou]

I pulled this pr and tried to use the monitor module on my local machine.
Looks good.

[HTHou]

Plz fix the conflict.

[Alima777]

Plz fix the conflict.

Fixed.

[jixuan1989]

Data Monitoring

This module is for providing some statistics info about the writing operations:

• the data size (in bytes) in IoTDB, the number of data points in IoTDB;
• how many operations are successful or failed executed.

Enable/disable the module

Users can choose to enable or disable the feature of data statistics monitoring (set the enable_stat_monitor item in the configuration file).

Statistics Data Storing

By default, the statistics data is only saved in memory and can be accessed using Jconsole.

The data can also be written as some time series on disk. To enable it, set enable_monitor_series_write=true in the configuration file. If so, using select statement in IoTDB-cli can query these time series.

Note:
if enable_monitor_series_write=true, when IoTDB is restarted, the previous statistics data will be recovered into memory.
if enable_monitor_series_write=false, IoTDB will forget all statistics data after the instance is restarted.

Writing Data Monitor

At present, the monitor system can be divided into two modules: global writing statistics and storage group writing statistics. The global statistics records the number of total points and requests, and the storage group statistics counts the write data of each storage group.

The system sets the collection granularity of the monitoring module to update the statistical information once one data file is flushed into the disk, so the data accuracy may be different from the actual situation. To obtain accurate information, Please call the flush method before querying.

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities (and 0 Security Hotspots to review)
32 Code Smells

No Coverage information
0.3% Duplication