Implement file change coalescing and sequence number system

[guydols]

This pull request introduces a file change acknowledgment mechanism to the filesync protocol and codebase. The main goal is to enable clients and servers to track which file changes (by sequence number) have been received and acknowledged, improving reliability and observability in the sync process. It also bumps the protocol version and propagates change_sequence fields throughout file metadata and message types.

Protocol and Metadata Updates:

• Protocol version is incremented to 7, and a new ChangeAcknowledgment message type is added, allowing clients to acknowledge receipt of specific file changes (by sequence number) within a bundle. The FileMetadata struct now includes a change_sequence field. [1] [2] [3]

File Bundling and Streaming:

• The stream_messages and stream_large_file functions now accept an optional SyncEngine reference and populate the change_sequence field for each file or directory. This ensures each change is tracked with a unique sequence number. [1] [2] [3] [4] [5] [6]
• The manifest builder sets change_sequence to 0 for files and directories, ensuring compatibility. [1] [2]

Client Handling of Acknowledgments:

• The client now sends a ChangeAcknowledgment message after receiving and applying a bundle, listing all nonzero sequence numbers. It also logs received acknowledgments (for future use in tracking delivery guarantees). Large file acknowledgments are noted as a future improvement. [1] [2] [3]

Server Tracking and Processing:

• The server maintains a sent_bundles map to track which bundles (and their sequence numbers) have been sent to clients, who has acknowledged them, and when. The BundleTracking struct is introduced for this purpose, and acknowledgment handling is added to the server receive loop. [1] [2] [3] [4] [5] [6] [7] [8] [9]
• The local change broadcaster and flush logic now pass along the sent_bundles reference so that broadcasted bundles are tracked for acknowledgment. [1] [2] [3] [4] [5]

Change Coalescing and Stability:

• When a file is determined to be stable and ready for processing, its change history is cleared, ensuring proper coalescing and preventing redundant tracking.

---

Most important changes:

Protocol and Metadata:

• Added change_sequence to FileMetadata and a new ChangeAcknowledgment message to the protocol, incrementing PROTOCOL_VERSION to 7. [1] [2] [3]
• All file and directory metadata in manifests and bundles now include change_sequence. [1] [2] [3] [4] [5]

Client acknowledgment logic:

• After applying a bundle, the client sends a ChangeAcknowledgment message to the server, listing all sequence numbers in the bundle.
• Client logs incoming acknowledgments for possible future tracking.

Server acknowledgment tracking:

• Server tracks sent bundles and their sequence numbers per client using the new BundleTracking struct and sent_bundles map, recording acknowledgments as they arrive. [1] [2] [3]

Bundle broadcasting and tracking:

• Broadcasting of file changes now records sent bundles for acknowledgment, passing sent_bundles throughout the server's change flush logic. [1] [2] [3] [4]

Change coalescing:

• When a file is processed as stable, its change history is cleared to support correct coalescing.