Skip to content

[IOTDB-6262] Pipe: fix NPE while deserializing WAL (caused by non-atomic WAL rename operation during pipe read)#11753

Merged
SteveYurongSu merged 1 commit intoapache:masterfrom
yschengzi:IOTDB-6262
Dec 20, 2023
Merged

[IOTDB-6262] Pipe: fix NPE while deserializing WAL (caused by non-atomic WAL rename operation during pipe read)#11753
SteveYurongSu merged 1 commit intoapache:masterfrom
yschengzi:IOTDB-6262

Conversation

@yschengzi
Copy link
Copy Markdown
Contributor

@yschengzi yschengzi commented Dec 20, 2023

Problem:

  • Due to the concurrent execution of the renaming of the wal file and the reading of the wal file by the pipe, the pipe may not be able to fetch the wal file when it reads the wal file, resulting in an NPE.

Solution:

  1. change Files.renameTo in wal code to Files.move for better atomicity and visibility.
  2. add logging when wal fetches a file
  3. add logging when pipe fetches a wal value to print the memtable id and wal file version id that it failed to fetch.

问题:

  • 由于wal 文件的重命名和 pipe 读取 wal 文件存在并发执行的情况,导致 pipe 读取 wal 文件时可能无法获取 wal 文件,导致 NPE

解决方案:

  1. 将 wal 代码中的 Files.renameTo 改成原子性和可见性更好的 Files.move
  2. 增加 wal 获取文件时的日志
  3. 增加 pipe 获取 wal 值时的日志,打印未能获取的 memtable id 和 wal file version id

image

…nd throw exception when can not find wal file
@SteveYurongSu SteveYurongSu self-assigned this Dec 20, 2023
@SteveYurongSu SteveYurongSu changed the title [IOTDB-6262] Pipe: NPE while deserialize WAL [IOTDB-6262] Pipe: fix NPE while deserializing WAL (caused by non-atomic WAL rename operation during pipe read) Dec 20, 2023
@SteveYurongSu SteveYurongSu merged commit a0ac60e into apache:master Dec 20, 2023
SzyWilliam pushed a commit to SzyWilliam/iotdb that referenced this pull request Nov 26, 2024
…mic WAL rename operation during pipe read) (apache#11753)

Problem:
- Due to the concurrent execution of the renaming of the wal file and the reading of the wal file by the pipe, the pipe may not be able to fetch the wal file when it reads the wal file, resulting in an NPE.
 
Solution:
1. change Files.renameTo in wal code to Files.move for better atomicity and visibility.
2. add logging when wal fetches a file
3. add logging when pipe fetches a wal value to print the memtable id and wal file version id that it failed to fetch.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants