Skip to content

Hdds 7279#3957

Closed
hemantk-12 wants to merge 11 commits intoapache:masterfrom
hemantk-12:HDDS-7279
Closed

Hdds 7279#3957
hemantk-12 wants to merge 11 commits intoapache:masterfrom
hemantk-12:HDDS-7279

Conversation

@hemantk-12
Copy link
Contributor

What changes were proposed in this pull request?

The OmRequest double buffer flush thread flushes the entire buffer as a batch. Since follower OM's will flush batches with different contents, snapshots can't stay consistent between the leader and the followers.
This change is to flush double buffer snapshot aware and flush all the transactions before snapshot create request.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-7279

How was this patch tested?

  • TBD

neils-dev and others added 11 commits August 3, 2022 17:11
* rebased

* fixed equals

* trigger new CI check

* fix merge from master

* checkstyle

* use the UUID for the directory name

* changed protocol from snapshotPath to vol/bucket

* added official SNAPSHOT_LOCK

* fixed the snapshot lock resource name

* fixed snapshot lock methods

* checkstyle

* cleanup

* now throws on failure to create directories

Co-authored-by: George Jahad <george@georgejahad.com>
* HDDS-7224. Create a new RocksDBCheckpoint Diff utility.

* HDDS-7224. Renamed RocksDBCheckpointDiffer to rocksdb-checkpoint-differ

* HDDS-7224. Fixing CI errors.

* HDDS-7224. Fixing CI errors.

* Add license headers.

Change-Id: I65f15273d97863bf2703debba0a0bdb6e256b9b8

* findbugs: add empty hadoop-hdds/rocksdb-checkpoint-differ/dev-support/findbugsExcludeFile.xml

Change-Id: Ia75af4a7ee3d33b5235572187d4cfd56e185586a

* Add new dependency licenses (remove them later if unneeded).

Change-Id: I57a7992912af330cbe4225c78f14755a150bdace

* HDDS-7224. Fixing CI errors.

* HDDS-7224. Fixing CI errors.

* Empty commit. Looks like fingbugs is not picking up the correct line
numbers.

Change-Id: Iacec58a628daa0e502fdd578164245d1fa34b9aa

* Address findbugs

Change-Id: I2e73578328f5a14c73da756a0eb6a785c1dccdc4

* artifactId rename to `rocksdb-checkpoint-differ`

Change-Id: I11b58b682eabaa13b636ba6ae80015b2270df9e3

* Correct file name to `RelationshipEdge`

Change-Id: Idb148a86d11e4ba99925def26792d37608265f34

* Checkstyle: unused import

Change-Id: I227c24209d5360320f6a9c7d450dfb01611e8fe7

* override enforcer rule in submodule `rocksdb-checkpoint-differ`

Change-Id: I4ef2a68e9d799ab1bdcb80ab8136cb6aa2ccf600

* Update jar-report.txt

Change-Id: I26affe5da495969e7bff40f8590d812b5914c5e9

* Update hadoop-hdds/pom.xml

Co-authored-by: Doroszlai, Attila <6454655+adoroszlai@users.noreply.github.com>

* Address Nanda's comments.

Change-Id: I31206a447da88215e5fa5e9c4c0e216635235fec

Co-authored-by: Prashant Pogde <prashantpogde@apache.org>
Co-authored-by: Prashant Pogde <ppogde@cloudera.com>
Co-authored-by: Siyao Meng <50227127+smengcl@users.noreply.github.com>
Co-authored-by: Doroszlai, Attila <6454655+adoroszlai@users.noreply.github.com>
…#3729)

* updated feature branch

* cleanup

* checkstyle

* cleaned up Bucket handling

* trigger new CI check

* trigger new CI check

* trigger new CI check

* trigger new CI check

* remove unneeded delete test

* trigger new CI check

* Update hadoop-hdds/common/src/main/resources/ozone-default.xml

Co-authored-by: Siyao Meng <50227127+smengcl@users.noreply.github.com>

* Update hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/IOmMetadataReader.java

Co-authored-by: Siyao Meng <50227127+smengcl@users.noreply.github.com>

* Update hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmMetadataManagerImpl.java

Co-authored-by: Siyao Meng <50227127+smengcl@users.noreply.github.com>

* Update hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmMetadataManagerImpl.java

Co-authored-by: Siyao Meng <50227127+smengcl@users.noreply.github.com>

* Update hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOmSnapshot.java

Co-authored-by: Siyao Meng <50227127+smengcl@users.noreply.github.com>

* Update hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java

Co-authored-by: Siyao Meng <50227127+smengcl@users.noreply.github.com>

* fixups

* add TimeUnit

* trigger new CI check

* renamed OmLock to IOzoneManagerLock

* cleanup

* cleanup

* fix timeout

* remove failcount

* checkstyle

* now close on eviction

* fix merge conflict

* trigger new CI check

Co-authored-by: George Jahad <george@georgejahad.com>
Co-authored-by: Siyao Meng <50227127+smengcl@users.noreply.github.com>
@hemantk-12 hemantk-12 closed this Nov 14, 2022
@hemantk-12
Copy link
Contributor Author

Created PR on wrong branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants