Add feature verify-seriesfile

[GikaWong]

Is your feature request related to a problem? Please describe.

Add feature series files integrity verification.

Describe the solution you'd like

Verifies the integrity of series files.

more info

[rick-200]

Purpose: Verifies the integrity of series files.

--> .\cnosdb_inspect verify-seriesfile --help
Verifies the integrity of Series files.

Usage:
  cnosdb_inspect verify-seriesfile [flags]

Flags:
      --c int                Specifies the number of concurrent workers to run for this command. Default is equal to the value of GOMAXPROCS. If performance is adversely impacted, you can set a lower value.
      --db string            Restricts verifying series files to the specified database in the data directory.
      --dir string           Specifies the root data path. Defaults to '$HOME/data'.
  -h, --help                 help for verify-seriesfile
      --series-file string   Path to a specific series file; overrides -db and -dir.
      --v                    Enables verbose logging.

example 1:

--> .\cnosdb_inspect verify-seriesfile
verify-seriesfile completed, 0/2 invalid seriesfiles.

example 2:

--> .\cnosdb_inspect verify-seriesfile --db mydb
2022-03-04T12:01:48.162274Z     error   Panic parsing key       {"log_id": "0_1ClSVG000", "path": "C:\\Users\\15112\\.cnosdb\\data\\mydb\\_series", "partition": "00", "segment": "0000", "key": "00", "offset": 5, "recovered": "runtime error: index out of range [1] with length 0"}
verify-seriesfile completed, seriesfile is invalid.

example 3:

--> .\cnosdb_inspect verify-seriesfile --db mydb --v
2022-03-04T12:13:04.538192Z     info    Verifying series seriesFile     {"log_id": "0_1DPjbW000", "path": "C:\\Users\\15112\\.cnosdb\\data\\mydb\\_series"}
2022-03-04T12:13:04.538713Z     info    Verifying partition     {"log_id": "0_1DPjbW000", "path": "C:\\Users\\15112\\.cnosdb\\data\\mydb\\_series", "partition": "00"}
2022-03-04T12:13:04.539252Z     info    Verifying partition     {"log_id": "0_1DPjbW000", "path": "C:\\Users\\15112\\.cnosdb\\data\\mydb\\_series", "partition": "06"}
2022-03-04T12:13:04.539252Z     info    Verifying partition     {"log_id": "0_1DPjbW000", "path": "C:\\Users\\15112\\.cnosdb\\data\\mydb\\_series", "partition": "07"}
2022-03-04T12:13:04.538713Z     info    Verifying partition     {"log_id": "0_1DPjbW000", "path": "C:\\Users\\15112\\.cnosdb\\data\\mydb\\_series", "partition": "01"}
2022-03-04T12:13:04.538713Z     info    Verifying partition     {"log_id": "0_1DPjbW000", "path": "C:\\Users\\15112\\.cnosdb\\data\\mydb\\_series", "partition": "03"}
2022-03-04T12:13:04.538713Z     info    Verifying partition     {"log_id": "0_1DPjbW000", "path": "C:\\Users\\15112\\.cnosdb\\data\\mydb\\_series", "partition": "04"}
2022-03-04T12:13:04.538713Z     info    Verifying partition     {"log_id": "0_1DPjbW000", "path": "C:\\Users\\15112\\.cnosdb\\data\\mydb\\_series", "partition": "02"}
2022-03-04T12:13:04.538713Z     info    Verifying partition     {"log_id": "0_1DPjbW000", "path": "C:\\Users\\15112\\.cnosdb\\data\\mydb\\_series", "partition": "05"}
2022-03-04T12:13:04.541163Z     info    Verifying segment       {"log_id": "0_1DPjbW000", "path": "C:\\Users\\15112\\.cnosdb\\data\\mydb\\_series", "partition": "00", "segment": "0000"}
2022-03-04T12:13:04.541683Z     error   Panic parsing key       {"log_id": "0_1DPjbW000", "path": "C:\\Users\\15112\\.cnosdb\\data\\mydb\\_series", "partition": "00", "segment": "0000", "key": "00", "offset": 5, "recovered": "runtime error: index out of range [1] with length 0"}
verify-seriesfile completed, seriesfile is invalid.

example 4:

--> .\cnosdb_inspect verify-seriesfile --series-file "D:/test_v_s" --v --c 1
2022-03-04T12:18:51.535900Z     info    Verifying series seriesFile     {"log_id": "0_1Djv3l000", "path": "D:/test_v_s"}
2022-03-04T12:18:51.535900Z     info    Verifying partition     {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "00"}
2022-03-04T12:18:51.536596Z     info    Verifying segment       {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "00", "segment": "0000"}
2022-03-04T12:18:51.537155Z     info    Verifying index {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "00"}
2022-03-04T12:18:51.537372Z     info    Verifying partition     {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "01"}
2022-03-04T12:18:51.537875Z     info    Verifying segment       {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "01", "segment": "0000"}
2022-03-04T12:18:51.537924Z     info    Verifying index {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "01"}
2022-03-04T12:18:51.538526Z     info    Verifying partition     {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "02"}
2022-03-04T12:18:51.538526Z     info    Verifying segment       {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "02", "segment": "0000"}
2022-03-04T12:18:51.539348Z     info    Verifying index {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "02"}
2022-03-04T12:18:51.539556Z     info    Verifying partition     {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "03"}
2022-03-04T12:18:51.540060Z     info    Verifying segment       {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "03", "segment": "0000"}
2022-03-04T12:18:51.541193Z     info    Verifying index {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "03"}
2022-03-04T12:18:51.541297Z     info    Verifying partition     {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "04"}
2022-03-04T12:18:51.541297Z     info    Verifying segment       {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "04", "segment": "0000"}
2022-03-04T12:18:51.542361Z     info    Verifying index {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "04"}
2022-03-04T12:18:51.542591Z     info    Verifying partition     {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "05"}
2022-03-04T12:18:51.542591Z     info    Verifying segment       {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "05", "segment": "0000"}
2022-03-04T12:18:51.544375Z     info    Verifying index {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "05"}
2022-03-04T12:18:51.548137Z     info    Verifying partition     {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "06"}
2022-03-04T12:18:51.552485Z     info    Verifying segment       {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "06", "segment": "0000"}
2022-03-04T12:18:51.553087Z     info    Verifying index {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "06"}
2022-03-04T12:18:51.553087Z     info    Verifying partition     {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "07"}
2022-03-04T12:18:51.553087Z     info    Verifying segment       {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "07", "segment": "0000"}
2022-03-04T12:18:51.554083Z     info    Verifying index {"log_id": "0_1Djv3l000", "path": "D:/test_v_s", "partition": "07"}
verify-seriesfile completed, seriesfile is valid.

example 5:

--> .\cnosdb_inspect verify-seriesfile --dir "D:/test_dir" --v --c 1
2022-03-04T12:27:44.988332Z     info    Verifying series seriesFile     {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series"}
2022-03-04T12:27:44.988332Z     info    Verifying partition     {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "00"}
2022-03-04T12:27:44.989102Z     info    Verifying segment       {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "00", "segment": "0000"}
2022-03-04T12:27:44.989781Z     info    Verifying index {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "00"}
2022-03-04T12:27:44.989859Z     info    Verifying partition     {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "01"}
2022-03-04T12:27:44.990374Z     info    Verifying segment       {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "01", "segment": "0000"}
2022-03-04T12:27:44.990935Z     info    Verifying index {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "01"}
2022-03-04T12:27:44.990935Z     info    Verifying partition     {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "02"}
2022-03-04T12:27:44.991501Z     info    Verifying segment       {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "02", "segment": "0000"}
2022-03-04T12:27:44.992008Z     info    Verifying index {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "02"}
2022-03-04T12:27:44.992067Z     info    Verifying partition     {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "03"}
2022-03-04T12:27:44.992575Z     info    Verifying segment       {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "03", "segment": "0000"}
2022-03-04T12:27:44.992635Z     info    Verifying index {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "03"}
2022-03-04T12:27:44.993275Z     info    Verifying partition     {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "04"}
2022-03-04T12:27:44.993275Z     info    Verifying segment       {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "04", "segment": "0000"}
2022-03-04T12:27:44.993801Z     info    Verifying index {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "04"}
2022-03-04T12:27:44.994338Z     info    Verifying partition     {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "05"}
2022-03-04T12:27:44.994338Z     info    Verifying segment       {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "05", "segment": "0000"}
2022-03-04T12:27:44.995406Z     info    Verifying index {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "05"}
2022-03-04T12:27:44.995406Z     info    Verifying partition     {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "06"}
2022-03-04T12:27:44.995945Z     info    Verifying segment       {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "06", "segment": "0000"}
2022-03-04T12:27:44.996553Z     info    Verifying index {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "06"}
2022-03-04T12:27:44.997085Z     info    Verifying partition     {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "07"}
2022-03-04T12:27:44.997613Z     info    Verifying segment       {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "07", "segment": "0000"}
2022-03-04T12:27:44.998422Z     info    Verifying index {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\NOAA_water_database\\_series", "partition": "07"}
2022-03-04T12:27:44.999087Z     info    Verifying series seriesFile     {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\mydb\\_series"}
2022-03-04T12:27:44.999087Z     info    Verifying partition     {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\mydb\\_series", "partition": "00"}
2022-03-04T12:27:44.999820Z     info    Verifying segment       {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\mydb\\_series", "partition": "00", "segment": "0000"}
2022-03-04T12:27:45.000358Z     error   Panic parsing key       {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\mydb\\_series", "partition": "00", "segment": "0000", "key": "00", "offset": 5, "recovered": "runtime error: index out of range [1] with length 0"}
2022-03-04T12:27:45.000358Z     info    Verifying partition     {"log_id": "0_1EFTrl000", "path": "D:\\test_dir\\mydb\\_series", "partition": "01"}
verify-seriesfile completed, 1/2 invalid seriesfiles.