Repository navigation
Replies: 1 comment
|
One area I really think we should prioritize also and discuss how it will go is the VFS work (#1043). It gives us a major opportunity to clean up and restructure parts of the codebase that have become difficult to follow and poorly documented after layering multiple security and bug fixes on top of each other (which will get worse after the work will be done for v4.0) . I'm honestly in favor of making VFS the master branch and porting our recent security fixes, regressions and enhancements over to it one by one making sure everything is well-documented and clean. That gives us a practical way to end up with clean, well-documented code that's easier to maintain and contribute to long-term. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
I want to start planning rsync 4.0. This is not currently intended to be a breaking release but it may very well have breaks where that genuinely improves a part of rsync. Compatibility should remain the default and the version number is not a reason to break existing commands, configurations or peers. If a particular change does need to break something, that should be discussed on its own merits. Work also does not need to wait for 4.0. Compatible fixes, improvements and optimisations can still go into 3.5 when they are ready. The point of 4.0 is to give us a clear target for finishing a broader pass over the project.
Backlog
Rsync has a very large backlog of issues and pull requests, including some which have been open for years.
I want to go through all of it. Some work will be merged, some will need a smaller or updated implementation and some no longer makes sense for the project. Broad ideas may be better kept in Discussions. Old reports which cannot be reproduced or reviewed should be closed. This does not mean every feature request will be implemented. It means everything should receive an actual decision rather than remaining open indefinitely.
The 4.0 milestone will track work which we decide should be resolved before the release.
This will include improvements to our documentation as I have noticed it is most certainly lacking in some places.
Architecture
There is quite a lot of old, duplicated and difficult code which deserves another look.
Areas worth reviewing include path handling, metadata, protocol messages, process cleanup, platform fallbacks, global state, error handling and duplicated filesystem policy.
The VFS work in #1043 is one possible direction. The same applies to larger proposals such as extended platform support, stream-source transfers, file flags, PAM support and new metadata handling.
I would rather take useful parts of these proposals than force the project through one enormous rewrite.
Performance
I also want a proper performance pass across rsync.
That should cover common workloads such as large directory trees, many small files, large files with small deltas, sparse files, hard links, metadata-heavy transfers, local copies, high-latency links and high-bandwidth links.
We should measure wall time, CPU use, memory, filesystem I/O and wire traffic before and after meaningful changes. Optimisations should address real bottlenecks rather than only making a small benchmark look better. Existing ideas around sparse writes, checksum work, file-list construction, readahead, metadata syscalls and transfer concurrency are all worth reviewing.
If rsync is unusually slow for a real workload, please share the command, platform, filesystem, rough data shape and any measurements you have. That will be much more useful than a general request to make it faster.
Testsuite and CI
The testsuite and CI work are also part of 4.0 -- #1122 gave us a much stronger base, though there is still cleanup and coverage work to do.
I want to continue improving test organisation, remove duplicate or meaningless tests, replace timing races with proper synchronisation and reduce unexplained platform exceptions. The supported operating systems should continue to run natively in CI. Historical peers, cross-platform transfers, release archives, installed binaries, sanitizers, Valgrind and fuzzing should remain part of qualification. The testsuite should also gain repeatable performance and resource benchmarks so optimisation work has a useful baseline.
Compatibility
4.0 should continue to work with older peers wherever practical. A product version change does not automatically require a wire-protocol change. New protocol features should be negotiated and should have clear behaviour with older peers. Existing scripts and configurations should not change meaning without a good reason and a documented way to migrate. Supported platforms should not be removed simply because doing so would make the code easier to clean up.
When 4.0 is ready
I expect 4.0 to be ready when:
After 4.0, I would like rsync to return to very conservative maintenance. Security fixes, correctness fixes, platform maintenance and worthwhile measured optimisations will continue. New features should have a clear use case and be maintainable for the long term.
Ideas
Please submit ideas if you would like.
You can reply here, link an existing issue or pull request or open a separate ideas discussion for something which needs more space. A patch is not required though a clear use case and examples will help. I am especially interested in real compatibility requirements, platforms which still matter to users, recurring performance problems and parts of rsync which are unnecessarily difficult to maintain.
All reactions