Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parallel replication: clean up variables determining parallelism & convert to config knobs #306

Closed
2 tasks
problame opened this issue Apr 12, 2020 · 1 comment
Milestone

Comments

@problame
Copy link
Member

problame commented Apr 12, 2020

Current Situation

  • ZREPL_REPLICATION_EXPERIMENTAL_REPLICATION_CONCURRENCY controls overall per-filesystem concurrency
  • per-filesystem PlanFS uses
    • ZREPL_REPLICATION_MAX_CONCURRENT_SIZE_ESTIMATE for dry sends (and only those)
      • note that these have both CPU requirements, thus it makes sense to limit concurrency here
    • all other code is sequential
  • per-step doReplication is sequential (only 1 zfs operation at a time), and concurrency of multiple steps is controlled again by ZREPL_REPLICATION_EXPERIMENTAL_REPLICATION_CONCURRENCY

Desirable

replication:
  concurrency:
    planning: 
    replication:

Open Questions

  • vars like ZREPL_ENDPOINT_MAX_CONCURRENT_SEND are used to protect a passive side from an active side that sends lots of IO / CPU-intensive requests in parallel. What to do about them?
  • what about the Concurrency field in the endpoint.ListAbstractions query?
@problame problame created this issue from a note in Parallel Replication (Incoming) Apr 12, 2020
@problame problame added this to the 0.4 milestone Apr 12, 2020
@problame
Copy link
Member Author

fixed by 0ceea1b

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

No branches or pull requests

1 participant