Skip to content

[To rel/0.12] Fix upgrade tool cannot load old tsfile if time partition enabled in 0.11#3154

Merged
qiaojialin merged 2 commits intorel/0.12from
partition_upgrade_bug
May 13, 2021
Merged

[To rel/0.12] Fix upgrade tool cannot load old tsfile if time partition enabled in 0.11#3154
qiaojialin merged 2 commits intorel/0.12from
partition_upgrade_bug

Conversation

@HTHou
Copy link
Copy Markdown
Contributor

@HTHou HTHou commented May 10, 2021

Description

Reproduce

  1. change the configs in 0.11
# whether enable data partition. If disabled, all data belongs to partition 0
enable_partition=true

# time range for partitioning data inside each storage group, the unit is second
partition_interval=1
  1. start 0.11 iotdb, execute these statements.
IoTDB> insert into root.sg.d1(time,s1,s2) values(1001,1,2)
Msg: The statement is executed successfully.
IoTDB> insert into root.sg.d1(time,s1,s2) values(2001,1,2)
Msg: The statement is executed successfully.
IoTDB> flush

The tsfile file paths looks like

$ tree data
data
├── sequence
│   └── root.sg
│       ├── 1
│       │   ├── 1620653607439-1-0.tsfile
│       │   └── 1620653607439-1-0.tsfile.resource
│       └── 2
│           ├── 1620653626029-1-0.tsfile
│           └── 1620653626029-1-0.tsfile.resource
└── unsequence

  1. Move the data files 0.12 and start server with same partition configs.
  2. Execute Select * from root and get empty result set.

Reason

In 0.12, we introduce the virtual storage group, and there should be a directory names 0 under the storage group directory.

The upgrade tool should move all files of each partition to this virtual storage group directory before upgrading, otherwise, the server cannot load the old files.

There is no problem if time partition disabled in 0.11. The reason is a directory names 0 under the storage group folder exists.


This PR has:

  • been self-reviewed.
    • concurrent read
    • concurrent write
    • concurrent read and write
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods.
  • added or updated version, license, or notice information
  • added comments explaining the "why" and the intent of the code wherever would not be obvious
    for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold
    for code coverage.
  • added integration tests.
  • been tested in a test IoTDB cluster.

Key changed/added classes (or packages if there are too many classes) in this PR

@HTHou HTHou changed the title Fix upgrade tool cannot load old tsfile if time partition enabled in 0.11 [To rel/0.12] Fix upgrade tool cannot load old tsfile if time partition enabled in 0.11 May 10, 2021
@sonarqubecloud
Copy link
Copy Markdown

@qiaojialin qiaojialin merged commit 676b1f1 into rel/0.12 May 13, 2021
@qiaojialin qiaojialin deleted the partition_upgrade_bug branch May 13, 2021 06:06
HTHou added a commit that referenced this pull request May 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants