Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

retransmits shreds recovered from erasure codes (backport #19233) #20249

Merged
merged 7 commits into from
Sep 27, 2021
Merged

retransmits shreds recovered from erasure codes (backport #19233) #20249

merged 7 commits into from
Sep 27, 2021

Commits on Sep 27, 2021

  1. removes packet-count metrics from retransmit stage 

    Working towards sending shreds (instead of packets) to retransmit stage
so that shreds recovered from erasure codes are as well retransmitted.

Following commit will add these metrics back to window-service, earlier
in the pipeline.

(cherry picked from commit bf437b0)

# Conflicts:
#	core/src/retransmit_stage.rs
    @behzadnouri
    behzadnouri authored and mergify-bot committed Sep 27, 2021
    Configuration menu
    Copy the full SHA
    e941a07 View commit details
    Browse the repository at this point in the history

  2. adds packet/shred count stats to window-service 

    Adding back these metrics from the earlier commit which removed them
from retransmit stage.

(cherry picked from commit 8198a7e)
    @behzadnouri
    behzadnouri authored and mergify-bot committed Sep 27, 2021
    Configuration menu
    Copy the full SHA
    b922775 View commit details
    Browse the repository at this point in the history

  3. removes erroneous uses of Arc<...> from retransmit stage 

    (cherry picked from commit 6e41333)

# Conflicts:
#	core/src/retransmit_stage.rs
#	core/src/tvu.rs
    @behzadnouri
    behzadnouri authored and mergify-bot committed Sep 27, 2021
    Configuration menu
    Copy the full SHA
    c916136 View commit details
    Browse the repository at this point in the history

  4. sends shreds (instead of packets) to retransmit stage 

    Working towards channelling through shreds recovered from erasure codes
to retransmit stage.

(cherry picked from commit 3efccbf)

# Conflicts:
#	core/src/retransmit_stage.rs
    @behzadnouri
    behzadnouri authored and mergify-bot committed Sep 27, 2021
    Configuration menu
    Copy the full SHA
    a6e5318 View commit details
    Browse the repository at this point in the history

  5. returns completed-data-set-info from insert_data_shred 

    instead of opaque (u32, u32) which are then converted to
CompletedDataSetInfo at the call-site.

(cherry picked from commit 3c71670)

# Conflicts:
#	ledger/src/blockstore.rs
    @behzadnouri
    behzadnouri authored and mergify-bot committed Sep 27, 2021
    Configuration menu
    Copy the full SHA
    9e0fd67 View commit details
    Browse the repository at this point in the history

  6. retransmits shreds recovered from erasure codes 

    Shreds recovered from erasure codes have not been received from turbine
and have not been retransmitted to other nodes downstream. This results
in more repairs across the cluster which is slower.

This commit channels through recovered shreds to retransmit stage in
order to further broadcast the shreds to downstream nodes in the tree.

(cherry picked from commit 7a8807b)

# Conflicts:
#	core/src/retransmit_stage.rs
#	core/src/window_service.rs
    @behzadnouri
    behzadnouri authored and mergify-bot committed Sep 27, 2021
    Configuration menu
    Copy the full SHA
    b503e7d View commit details
    Browse the repository at this point in the history

  7. removes backport merge conflicts

    @behzadnouri
    behzadnouri committed Sep 27, 2021
    Configuration menu
    Copy the full SHA
    bd42581 View commit details
    Browse the repository at this point in the history