Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ledger storage] DbLedgerStorage should do periodical flush #1843

Merged
merged 1 commit into from
Nov 28, 2018

Conversation

sijie
Copy link
Member

@sijie sijie commented Nov 28, 2018

Descriptions of the changes in this PR:

Motivation

DbLedgerStorage doesn't drive checkpoint itself. so currently DbLedgerStorage flush only
happens either when write-cache is full or entry log file rotated. The correctness is still
maintained. However the behavior is different from original yahoo behavior.

Changes

Revert the behavior back to original periodical flush


In order to uphold a high standard for quality for code contributions, Apache BookKeeper runs various precommit
checks for pull requests. A pull request can only be merged when it passes precommit checks. However running all
the precommit checks can take a long time, some trivial changes don't need to run all the precommit checks. You
can check following list to skip the tests that don't need to run for your pull request. Leave them unchecked if
you are not sure, committers will help you:

  • [skip bookkeeper-server bookie tests]: skip testing org.apache.bookkeeper.bookie in bookkeeper-server module.
  • [skip bookkeeper-server client tests]: skip testing org.apache.bookkeeper.client in bookkeeper-server module.
  • [skip bookkeeper-server replication tests]: skip testing org.apache.bookkeeper.replication in bookkeeper-server module.
  • [skip bookkeeper-server tls tests]: skip testing org.apache.bookkeeper.tls in bookkeeper-server module.
  • [skip bookkeeper-server remaining tests]: skip testing all other tests in bookkeeper-server module.
  • [skip integration tests]: skip docker based integration tests. if you make java code changes, you shouldn't skip integration tests.
  • [skip build java8]: skip build on java8. ONLY skip this when ONLY changing files under documentation under site.
  • [skip build java9]: skip build on java9. ONLY skip this when ONLY changing files under documentation under site.


Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

If this PR is a BookKeeper Proposal (BP):

  • Make sure the PR title is formatted like:
    <BP-#>: Description of bookkeeper proposal
    e.g. BP-1: 64 bits ledger is support
  • Attach the master issue link in the description of this PR.
  • Attach the google doc link if the BP is written in Google Doc.

Otherwise:

  • Make sure the PR title is formatted like:
    <Issue #>: Description of pull request
    e.g. Issue 123: Description ...
  • Make sure tests pass via mvn clean apache-rat:check install spotbugs:check.
  • Replace <Issue #> in the title with the actual Issue number.

*Motivation*

DbLedgerStorage doesn't drive checkpoint itself. so currently DbLedgerStorage flush only
happens either when write-cache is full or entry log file rotated. The correctness is still
maintained. However the behavior is different from original yahoo behavior.

*Changes*

Revert the behavior back to original periodical flush
Copy link
Contributor

@merlimat merlimat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change LGTM. Maybe we could also have a method in the LedgerStorage interface that each implementation can use to declare if it wants to have the sync thread to be scheduled or not.

@sijie
Copy link
Member Author

sijie commented Nov 28, 2018

@merlimat

Maybe we could also have a method in the LedgerStorage interface that each implementation can use to declare if it wants to have the sync thread to be scheduled or not.

was thinking about doing that in a subsequent change. because this change is going to all 3 branches, so better to keep the change simple, so I don't encounter any issue when cherry-pick it to branches.

@merlimat
Copy link
Contributor

was thinking about doing that in a subsequent change. because this change is going to all 3 branches, so better to keep the change simple, so I don't encounter any issue when cherry-pick it to branches.

Sure, let's do that in master only.

@sijie
Copy link
Member Author

sijie commented Nov 28, 2018

run integration tests

@sijie sijie added the type/bug label Nov 28, 2018
@sijie sijie merged commit d7fffac into apache:master Nov 28, 2018
sijie added a commit that referenced this pull request Nov 28, 2018
Descriptions of the changes in this PR:

*Motivation*

DbLedgerStorage doesn't drive checkpoint itself. so currently DbLedgerStorage flush only
happens either when write-cache is full or entry log file rotated. The correctness is still
maintained. However the behavior is different from original yahoo behavior.

*Changes*

Revert the behavior back to original periodical flush

Reviewers: Matteo Merli <mmerli@apache.org>, Jia Zhai <None>

This closes #1843 from sijie/dbledgerstorage_sync

(cherry picked from commit d7fffac)
Signed-off-by: Sijie Guo <sijie@apache.org>
sijie added a commit that referenced this pull request Nov 28, 2018
Descriptions of the changes in this PR:

*Motivation*

DbLedgerStorage doesn't drive checkpoint itself. so currently DbLedgerStorage flush only
happens either when write-cache is full or entry log file rotated. The correctness is still
maintained. However the behavior is different from original yahoo behavior.

*Changes*

Revert the behavior back to original periodical flush

Reviewers: Matteo Merli <mmerli@apache.org>, Jia Zhai <None>

This closes #1843 from sijie/dbledgerstorage_sync

(cherry picked from commit d7fffac)
Signed-off-by: Sijie Guo <sijie@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants