Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: state listener observe writes at wrong time (backport #13516) #14138

Merged
merged 8 commits into from Dec 6, 2022

Commits on Dec 2, 2022

  1. fix: state listener observe writes at wrong time (#13516)

    * fix: state listener observe writes at wrong time
    
    Closes: #13457
    
    Currently state listener is notified when the cache store write, which happens in commit event only, which breaks the current design.
    The solution (as discussed in the issue) is to listen state writes on rootmulti store only.
    
    It also changes the file streamer to output single data file for the writes in the whole block, since we can't distinguish writes from different stage of abci events.
    
    It adds new config items for file streamer:
    - streamers.file.output-metadata
    - streamers.file.stop-node-on-error
    - streamers.file.fsync
    
    * synchronous abci call, and format doc
    
    * fix comment
    
    * update file streamer readme and fix typos
    
    * typo
    
    * fix: state listener observe writes at wrong time
    
    Closes: #13457
    
    Currently state listener is notified when the cache store write, which happens in commit event only, which breaks the current design.
    The solution (as discussed in the issue) is to listen state writes on rootmulti store only.
    
    It also changes the file streamer to output single data file for the writes in the whole block, since we can't distinguish writes from different stage of abci events.
    
    It adds new config items for file streamer:
    - streamers.file.output-metadata
    - streamers.file.stop-node-on-error
    - streamers.file.fsync
    
    synchronous abci call, and format doc
    
    fix comment
    
    update file streamer readme and fix typos
    
    typo
    
    * improve UX of file streamer, make it immediately usable after enabled
    
    - set default value to write_dir.
    - make write_dir based on home directory by default.
    - auto-create the directory if not exists.
    
    * get homePage from opts
    
    Co-authored-by: Marko <marbar3778@yahoo.com>
    (cherry picked from commit 1f91ee2)
    
    # Conflicts:
    #	CHANGELOG.md
    #	api/cosmos/base/store/v1beta1/listening.pulsar.go
    #	baseapp/streaming.go
    #	docs/architecture/adr-038-state-listening.md
    #	server/config/toml.go
    #	simapp/app_v2.go
    #	store/cachemulti/store.go
    #	store/iavl/store.go
    #	store/mem/store.go
    #	store/streaming/constructor.go
    #	store/streaming/file/service.go
    #	store/streaming/file/service_test.go
    #	store/types/listening.pb.go
    #	store/types/store.go
    yihuang authored and mergify[bot] committed Dec 2, 2022
    Configuration menu
    Copy the full SHA
    d64de1f View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    43c5018 View commit details
    Browse the repository at this point in the history

Commits on Dec 3, 2022

  1. fix conflicts

    yihuang committed Dec 3, 2022
    Configuration menu
    Copy the full SHA
    7ee85ac View commit details
    Browse the repository at this point in the history
  2. fix conflicts

    yihuang committed Dec 3, 2022
    Configuration menu
    Copy the full SHA
    33b13dc View commit details
    Browse the repository at this point in the history

Commits on Dec 5, 2022

  1. revert api breaking change

    yihuang committed Dec 5, 2022
    Configuration menu
    Copy the full SHA
    81e7184 View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2022

  1. fix build

    yihuang committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    bd77b58 View commit details
    Browse the repository at this point in the history
  2. fix unit test

    yihuang committed Dec 6, 2022
    Configuration menu
    Copy the full SHA
    532dacb View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    482987c View commit details
    Browse the repository at this point in the history