Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

federation: parallel sending per instance #4623

Open
wants to merge 29 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
539f06a
federation: parallel sending
phiresky Apr 13, 2024
491daab
federation: some comments
phiresky Apr 13, 2024
987174a
lint and set force_write true when a request fails
phiresky Apr 15, 2024
a66aec6
inbox_urls return vec
phiresky Apr 15, 2024
a3d705f
split inbox functions into separate file
phiresky Apr 15, 2024
7eedcb7
cleanup
phiresky Apr 15, 2024
e719baf
extract sending task code to separate file
phiresky Apr 15, 2024
5e986ef
move federation concurrent config to config file
phiresky Apr 15, 2024
c1932f9
off by one issue
phiresky Apr 15, 2024
a7c7abd
improve msg
phiresky Apr 30, 2024
13ff059
fix both permanent stopping of federation queues and multiple creatio…
phiresky May 29, 2024
7cb4e82
Merge branch 'fix-dupe-activity-sending' into federation-send-parallel
phiresky May 29, 2024
10d3b7d
fix after merge
phiresky May 30, 2024
ffb99cd
Merge remote-tracking branch 'origin/main' into federation-send-parallel
phiresky May 30, 2024
a0b0a7a
lint fix
phiresky May 30, 2024
cdff275
Update crates/federate/src/send.rs
phiresky May 30, 2024
175133f
comment about reverse ordering
phiresky May 30, 2024
2acdc78
remove crashable, comment
phiresky May 30, 2024
9d87921
comment
phiresky May 30, 2024
5538794
move comment
phiresky May 30, 2024
7ee63f4
run federation tests twice
phiresky May 31, 2024
3784b7f
fix test run
phiresky May 31, 2024
c2d18d3
prettier
phiresky May 31, 2024
5a418ac
fix config default
phiresky May 31, 2024
c66bf26
upgrade rust to 1.78 to fix diesel cli
phiresky May 31, 2024
1c1018b
Merge remote-tracking branch 'origin/upgrade-rust' into federation-se…
phiresky May 31, 2024
101901b
fix clippy
phiresky May 31, 2024
dfccf3e
delay
phiresky Jun 3, 2024
2dd7b71
Merge branch 'main' into federation-send-parallel
phiresky Jun 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 2 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ anyhow = { version = "1.0.81", features = [
diesel_ltree = "0.3.1"
typed-builder = "0.18.1"
serial_test = "2.0.0"
tokio = { version = "1.36.0", features = ["full"] }
tokio = { version = "1.37.0", features = ["full"] }
regex = "1.10.3"
once_cell = "1.19.0"
diesel-derive-newtype = "2.1.0"
Expand Down
7 changes: 2 additions & 5 deletions crates/federate/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -73,19 +73,16 @@ async fn start_stop_federation_workers(
// create new worker
let config = federation_config.clone();
let stats_sender = stats_sender.clone();
let pool = pool.clone();
workers.insert(
instance.id,
CancellableTask::spawn(WORKER_EXIT_TIMEOUT, move |stop| {
let instance = instance.clone();
let req_data = config.clone().to_request_data();
let config = config.clone();
let stats_sender = stats_sender.clone();
let pool = pool.clone();
async move {
InstanceWorker::init_and_loop(
instance,
req_data,
&mut DbPool::Pool(&pool),
config,
stop,
stats_sender,
)
Expand Down