Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

retransmits shreds recovered from erasure codes (backport #19233) #20249

Merged
merged 7 commits into from
Sep 27, 2021

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Sep 27, 2021

This is an automatic backport of pull request #19233 done by Mergify.
Cherry-pick of bf437b0 has failed:

On branch mergify/bp/v1.7/pr-19233
Your branch is up to date with 'origin/v1.7'.

You are currently cherry-picking commit bf437b033.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   core/src/retransmit_stage.rs

no changes added to commit (use "git add" and/or "git commit -a")

Cherry-pick of 6e41333 has failed:

On branch mergify/bp/v1.7/pr-19233
Your branch is ahead of 'origin/v1.7' by 2 commits.
  (use "git push" to publish your local commits)

You are currently cherry-picking commit 6e413331b.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   core/benches/retransmit_stage.rs

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   core/src/retransmit_stage.rs
	both modified:   core/src/tvu.rs

Cherry-pick of 3efccbf has failed:

On branch mergify/bp/v1.7/pr-19233
Your branch is ahead of 'origin/v1.7' by 3 commits.
  (use "git push" to publish your local commits)

You are currently cherry-picking commit 3efccbffa.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   core/benches/retransmit_stage.rs
	modified:   core/src/packet_hasher.rs
	modified:   core/src/window_service.rs
	modified:   gossip/src/cluster_info.rs
	modified:   gossip/tests/gossip.rs

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   core/src/retransmit_stage.rs

Cherry-pick of 3c71670 has failed:

On branch mergify/bp/v1.7/pr-19233
Your branch is ahead of 'origin/v1.7' by 4 commits.
  (use "git push" to publish your local commits)

You are currently cherry-picking commit 3c71670bd.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   ledger/src/blockstore.rs

no changes added to commit (use "git add" and/or "git commit -a")

Cherry-pick of 7a8807b has failed:

On branch mergify/bp/v1.7/pr-19233
Your branch is ahead of 'origin/v1.7' by 5 commits.
  (use "git push" to publish your local commits)

You are currently cherry-picking commit 7a8807b8b.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   ledger/src/blockstore.rs

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   core/src/retransmit_stage.rs
	both modified:   core/src/window_service.rs

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/github/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally


Mergify commands and options

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

  • @Mergifyio refresh will re-evaluate the rules
  • @Mergifyio rebase will rebase this PR on its base branch
  • @Mergifyio update will merge the base branch into this PR
  • @Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

  • look at your merge queues
  • generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.io/

Working towards sending shreds (instead of packets) to retransmit stage
so that shreds recovered from erasure codes are as well retransmitted.

Following commit will add these metrics back to window-service, earlier
in the pipeline.

(cherry picked from commit bf437b0)

# Conflicts:
#	core/src/retransmit_stage.rs
Adding back these metrics from the earlier commit which removed them
from retransmit stage.

(cherry picked from commit 8198a7e)
(cherry picked from commit 6e41333)

# Conflicts:
#	core/src/retransmit_stage.rs
#	core/src/tvu.rs
Working towards channelling through shreds recovered from erasure codes
to retransmit stage.

(cherry picked from commit 3efccbf)

# Conflicts:
#	core/src/retransmit_stage.rs
instead of opaque (u32, u32) which are then converted to
CompletedDataSetInfo at the call-site.

(cherry picked from commit 3c71670)

# Conflicts:
#	ledger/src/blockstore.rs
Shreds recovered from erasure codes have not been received from turbine
and have not been retransmitted to other nodes downstream. This results
in more repairs across the cluster which is slower.

This commit channels through recovered shreds to retransmit stage in
order to further broadcast the shreds to downstream nodes in the tree.

(cherry picked from commit 7a8807b)

# Conflicts:
#	core/src/retransmit_stage.rs
#	core/src/window_service.rs
@mergify mergify bot added the conflicts label Sep 27, 2021
@mergify mergify bot added automerge Merge this Pull Request automatically once CI passes and removed automerge Merge this Pull Request automatically once CI passes labels Sep 27, 2021
@mergify
Copy link
Contributor Author

mergify bot commented Sep 27, 2021

automerge label removed due to a CI failure

@mergify mergify bot added automerge Merge this Pull Request automatically once CI passes and removed automerge Merge this Pull Request automatically once CI passes labels Sep 27, 2021
@mergify
Copy link
Contributor Author

mergify bot commented Sep 27, 2021

automerge label removed due to a CI failure

@mergify mergify bot added automerge Merge this Pull Request automatically once CI passes and removed automerge Merge this Pull Request automatically once CI passes labels Sep 27, 2021
@mergify
Copy link
Contributor Author

mergify bot commented Sep 27, 2021

automerge label removed due to a CI failure

@mergify mergify bot added the automerge Merge this Pull Request automatically once CI passes label Sep 27, 2021
@mergify
Copy link
Contributor Author

mergify bot commented Sep 27, 2021

automerge label removed due to a CI failure

@mergify mergify bot removed the automerge Merge this Pull Request automatically once CI passes label Sep 27, 2021
@behzadnouri behzadnouri merged commit a1a0c63 into v1.7 Sep 27, 2021
@behzadnouri behzadnouri deleted the mergify/bp/v1.7/pr-19233 branch September 27, 2021 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant