Skip to content

mergemaster0729#13055

Merged
JackieTien97 merged 29 commits intoty/TableModelGrammarfrom
ty/mergemaster0729
Jul 29, 2024
Merged

mergemaster0729#13055
JackieTien97 merged 29 commits intoty/TableModelGrammarfrom
ty/mergemaster0729

Conversation

@JackieTien97
Copy link
Copy Markdown
Contributor

No description provided.

SpriCoder and others added 29 commits July 29, 2024 12:03
* fix memory concurrency problem

* spotless

Signed-off-by: OneSizeFitQuorum <tanxinyu@apache.org>
Co-authored-by: OneSizeFitQuorum <tanxinyu@apache.org>
(cherry picked from commit 76ba733)
… enable_auto_create_schema is false #12990

Signed-off-by: OneSizeFitQuorum <tanxinyu@apache.org>
(cherry picked from commit 392625f)
…ernative (#12982)

The property `maven.multiModuleProjectDirectory` is not officially support and can be removed at any time, it has been removed when using mvnd.
The current alternative with Maven 3 is using path traversal from the current module's directory

Fixes: #12981
(cherry picked from commit cbabf87)
* reduce error log when compaction interrupted

* fix thread pool

(cherry picked from commit 27c2f13)
…#12986)

* Remove UNKNOWN from WALFileVersion which fails recovering V1 WAL file

* fix ut

* Detail print

(cherry picked from commit abb1f6a)
…ay cause NPE on receiver side (#12994)

Co-authored-by: Steve Yurong Su <rong@apache.org>
(cherry picked from commit a45adbc)
NOTE: This does NOT set Surefire's `forkCount` property, that will be done on an isolated commit
This is to ensure these changes don't negatively affect the existing build.

Details on changes:

- Change to test working directory

  The working directory was changed from `${project.build.directory}` to `${project.build.directory}/fork_${surefire.forkNumber}` (single fork builds would be `<module_name>/target/fork_1`),
  This isolates any test output written to relative paths.  Long term tests may want to be more explicit where they write output to, (e.g. using JUnit 5's `@TempDir` annotation)

- IOTDB_CONF property

  Becuse the working directory change, the IOTDB_CONF directory needs to resolve differently, however, even without parallel tests, this property _should_ use an absolute path

- Test run `mkdirs`

  The tests assumed that the `./target` directory was always present (relative to the working directory), a change to the working directory invalidated this assumption, so any directories a test uses should be created.
  This is probably a good idea even without the parallel test concern. (but in the future this could be replaced with something like JUnit 5's `@TempDir` annotation)

- Hardcoded Ports

  TestUtils, was using a hardcodes sequentially from `6001`, it now uses free JUnit 5's `@TempDir` annotation ports

Fixes: #12979
(cherry picked from commit 77b4131)
…on receivers (DN / CN) (#12927)

(cherry picked from commit 623fa5b)
* error -> warn

* warn -> info

* check file exist before delete

* done

(cherry picked from commit 3f112b9)
* commit for test

* re run

* update logs

(cherry picked from commit c866c0f)
… Format During CN and DN Upgrade (#13006)

(cherry picked from commit 7d94bba)
…d realtime extractor for tsfile format topic (#12996)

> Special thanks to Chang Xue for providing the test scenarios.

### Shared consumption semantics after this PR

During the process of converting a tablet to tsfile, it is possible that the generated tsfiles contain duplicate data. **Therefore, under normal consumption (no re-consumption situation), the tsfile format data CANNOT ensure that the total consumed data equals the written data. It can only be guaranteed that the total consumed data in the dataset format equals the written data**.

### Issue analysis

It should be noted that the parameter `updateFunction` of the `getAndUpdate` method of `AtomicReference` is required to be a side-effect-free function, as `updateFunction` may be execute concurrently.

If multiple threads call `batch.onEvent` at the same time, and if one thread returns true and seals the batch, the remaining threads still blocked on the `batch.onEvent` call may try to call `batch.onEvent` again, which could lead to sealing the same batch multiple times.

(cherry picked from commit b540636)
(cherry picked from commit ddd8d4d)
Co-authored-by: Steve Yurong Su <rong@apache.org>
(cherry picked from commit cb07658)
)

* add retry

* use TSStatus

* use TSStatus

* modify if judgment

* merge exception

(cherry picked from commit 97ce3e8)
* is it work?

* no it's not

* all wal UT pass, maybe it's work

* feel's like leetcode medium

(cherry picked from commit d5e409d)
* hot reload

* fix comment

(cherry picked from commit 25ed791)
(cherry picked from commit f0363a7)
* fix 833

* fix review(rename typo)

(cherry picked from commit 2a06d48)
@JackieTien97 JackieTien97 merged commit 80f93ce into ty/TableModelGrammar Jul 29, 2024
@JackieTien97 JackieTien97 deleted the ty/mergemaster0729 branch July 29, 2024 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.