Skip to content

Change FileTimeIndexCache to region level#13353

Merged
jt2594838 merged 5 commits intomasterfrom
regionLevelFileTimeIndexCache
Sep 3, 2024
Merged

Change FileTimeIndexCache to region level#13353
jt2594838 merged 5 commits intomasterfrom
regionLevelFileTimeIndexCache

Conversation

@HTHou
Copy link
Copy Markdown
Contributor

@HTHou HTHou commented Aug 30, 2024

Description

FileTimeIndexCache may open too much file if there is too much time partitions.

@codecov
Copy link
Copy Markdown

codecov bot commented Aug 30, 2024

Codecov Report

Attention: Patch coverage is 44.00000% with 42 lines in your changes missing coverage. Please review.

Project coverage is 40.98%. Comparing base (c547916) to head (4a15df7).
Report is 14 commits behind head on master.

Files with missing lines Patch % Lines
...n/tsfile/timeindex/FileTimeIndexCacheRecorder.java 32.50% 27 Missing ⚠️
.../iotdb/db/storageengine/dataregion/DataRegion.java 52.00% 12 Missing ⚠️
...s/fileTimeIndexCache/FileTimeIndexCacheReader.java 0.00% 2 Missing ⚠️
...storageengine/dataregion/tsfile/TsFileManager.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##             master   #13353   +/-   ##
=========================================
  Coverage     40.97%   40.98%           
  Complexity       71       71           
=========================================
  Files          3929     3929           
  Lines        242315   242308    -7     
  Branches      29251    29248    -3     
=========================================
+ Hits          99296    99308   +12     
+ Misses       143019   143000   -19     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@sonarqubecloud
Copy link
Copy Markdown


private final Map<Integer, Map<Long, FileTimeIndexCacheWriter>> writerMap =
new ConcurrentHashMap<>();
private final Map<Integer, FileTimeIndexCacheWriter> writerMap = new ConcurrentHashMap<>();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a comment to explain that the key is the data region id.

@jt2594838
Copy link
Copy Markdown
Contributor

One potential improvement: the compaction task clears the cache file at first. If a restart happens at that time, there will be no available cache file for this restart, and the system may suffer from a long restart time.
Maybe it would be better to adopt the write-a-temp-file-and-replace method.

@jt2594838 jt2594838 merged commit 64cf384 into master Sep 3, 2024
HTHou added a commit that referenced this pull request Sep 5, 2024
* Check FileTimeIndexCache to region level

* add log

* fix ut

* fix it error

* fix error log
jt2594838 pushed a commit that referenced this pull request Sep 5, 2024
* Check FileTimeIndexCache to region level

* add log

* fix ut

* fix it error

* fix error log
@HTHou HTHou deleted the regionLevelFileTimeIndexCache branch September 23, 2024 09:29
Caideyipi pushed a commit to Caideyipi/iotdb that referenced this pull request Mar 25, 2026
* Check FileTimeIndexCache to region level

* add log

* fix ut

* fix it error

* fix error log
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants