feat(filesync): add native sync and virtual files - #272
Conversation
️✅ There are no secrets present in this pull request anymore.If these secrets were true positive and are still valid, we highly recommend you to revoke them. 🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request. |
veryCrunchy
added
area:files-sync
ChatGPT Codex Connector
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 831ef33be2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
🟡 Not ready to approve
There are confirmed correctness/privacy issues in the changed code (resource leak on writeback manifest failure; fixture strings should be made unambiguously synthetic per repo guidelines).
Once you've addressed the issues Copilot identified, you can request another Copilot review.
This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.
Pull request overview
This PR introduces a major “Filesync + Virtual Files” foundation across Android and desktop, adding durable folder-pair sync planning/execution rules, platform virtual-file hydration/caching models, and desktop tray + startup integration within the Nextcloud Native architecture.
Changes:
- Add selective-sync/ignore/priority planning to the shared sync coordinator and persist it through snapshots/stores.
- Add desktop virtual file backends (Linux FUSE + Windows Cloud Files scaffolding/tests) plus persistent range-cache + writeback recovery.
- Add desktop tray snapshot model + popup UI, startup (“start on login”) integration, and synthetic marketing/visual QA capture updates.
File summaries
| File | Description |
|---|---|
| website/public/screenshots/capture-manifest.json | Registers new sync/virtual-file and tray capture sources and adds new capture scenarios + hashes. |
| ui/src/desktopTest/kotlin/dev/obiente/nextcloudnative/nativeui/preview/MarketingCaptureOwnershipTest.kt | Updates test expectations for preserved marketing capture files. |
| ui/src/desktopTest/kotlin/dev/obiente/nextcloudnative/app/WindowsCloudFilesProviderTest.kt | Adds Windows Cloud Files provider tests for identity, hydration planning, and recovery behaviors. |
| ui/src/desktopTest/kotlin/dev/obiente/nextcloudnative/app/LinuxVirtualFileSystemTest.kt | Adds Linux FUSE virtual filesystem tests (read, metadata, writes, readdir, errors). |
| ui/src/desktopTest/kotlin/dev/obiente/nextcloudnative/app/DesktopVirtualRangeCacheTest.kt | Adds tests for persistent desktop range cache and lease-based eviction protection. |
| ui/src/desktopTest/kotlin/dev/obiente/nextcloudnative/app/DesktopStartOnLoginTest.kt | Adds tests for Linux autostart entry behavior and Windows Run-key registration. |
| ui/src/desktopTest/kotlin/dev/obiente/nextcloudnative/app/DesktopLinuxVirtualFileWritebackStoreTest.kt | Adds tests for durable Linux virtual-file writeback staging and recovery. |
| ui/src/desktopTest/kotlin/dev/obiente/nextcloudnative/app/DesktopFileSyncTrayStateTest.kt | Adds tests for tray tooltip formatting and snapshot validation. |
| ui/src/desktopTest/kotlin/dev/obiente/nextcloudnative/app/DesktopFileSyncStoreTest.kt | Adds tests for desktop sync state persistence and root-overlap detection helpers. |
| ui/src/desktopTest/kotlin/dev/obiente/nextcloudnative/app/DesktopFileSyncRuntimeConditionsTest.kt | Adds tests for metered/power gating logic used for desktop sync scheduling. |
| ui/src/desktopTest/kotlin/dev/obiente/nextcloudnative/app/DesktopFileSyncRemoteTreeTest.kt | Adds tests for DAV parsing (plus-sign handling, XXE rejection) and owned staging detection. |
| ui/src/desktopTest/kotlin/dev/obiente/nextcloudnative/app/DesktopFileSyncLocalTreeTest.kt | Adds tests for ignore/selective scanning, guarded replacement, and staging recovery on desktop. |
| ui/src/desktopMain/kotlin/dev/obiente/nextcloudnative/nativeui/preview/MarketingCaptureOwnership.kt | Preserves tray-capture output filename that is produced by a non-Compose capture harness. |
| ui/src/desktopMain/kotlin/dev/obiente/nextcloudnative/nativeui/preview/Main.kt | Adds desktop SystemTray integration, tray popup window, and lifecycle hooks for sync/VFS restore. |
| ui/src/desktopMain/kotlin/dev/obiente/nextcloudnative/nativeui/preview/FileSyncTrayVisualQaMain.kt | Adds a real-window synthetic tray popup renderer for deterministic tray visual QA capture. |
| ui/src/desktopMain/kotlin/dev/obiente/nextcloudnative/app/LinuxVirtualFileSystem.kt | Implements Linux FUSE virtual filesystem plus a revision-pinned backend bridging to range cache + writeback. |
| ui/src/desktopMain/kotlin/dev/obiente/nextcloudnative/app/DesktopVirtualRangeCache.kt | Implements a persistent exact-revision block cache for Linux virtual file hydration reads. |
| ui/src/desktopMain/kotlin/dev/obiente/nextcloudnative/app/DesktopStartOnLogin.kt | Implements desktop start-on-login configuration for Linux autostart and Windows Run key. |
| ui/src/desktopMain/kotlin/dev/obiente/nextcloudnative/app/DesktopLinuxVirtualFileWritebackStore.kt | Implements durable staged writeback storage + recovery for Linux virtual-file edits. |
| ui/src/desktopMain/kotlin/dev/obiente/nextcloudnative/app/DesktopFileSyncTrayState.kt | Adds tray snapshot domain model, validation, and tooltip formatting helpers. |
| ui/src/desktopMain/kotlin/dev/obiente/nextcloudnative/app/DesktopFileSyncTrayPopup.kt | Implements the Compose desktop tray popup UI surface for activity/progress/actions. |
| ui/src/desktopMain/kotlin/dev/obiente/nextcloudnative/app/DesktopFileSyncRuntimeConditions.kt | Adds desktop runtime condition probing (power/network) for sync gating, including Windows JNA probes. |
| ui/src/desktopMain/kotlin/dev/obiente/nextcloudnative/app/DesktopFileSyncRemoteTree.kt | Adds bounded, revision-guarded WebDAV remote tree adapter for desktop sync and VFS writeback. |
| ui/src/desktopMain/kotlin/dev/obiente/nextcloudnative/app/DesktopFileSyncLocalTree.kt | Extends desktop local scan to support selective/ignore filtering and subtree pruning. |
| ui/src/desktopMain/kotlin/dev/obiente/nextcloudnative/app/DesktopFileReadCache.kt | Adds virtual-file cache policy persistence + LRU updates + eviction planning and application. |
| ui/src/commonTest/kotlin/dev/obiente/nextcloudnative/app/VirtualFileStorageCenterTest.kt | Adds tests for virtual-file storage snapshot invariants and unsupported defaults. |
| ui/src/commonTest/kotlin/dev/obiente/nextcloudnative/app/VirtualFileCacheTest.kt | Adds tests for eviction planning and open/hydrate planning behaviors. |
| ui/src/commonTest/kotlin/dev/obiente/nextcloudnative/app/FileSyncPolicyTest.kt | Adds tests for portable glob matching, selective+ignore behavior, and priority rule ordering. |
| ui/src/commonTest/kotlin/dev/obiente/nextcloudnative/app/FileSyncDirectionPresentationTest.kt | Updates tests to use ASCII arrows in sync route labels. |
| ui/src/commonTest/kotlin/dev/obiente/nextcloudnative/app/FileSyncCoordinatorTest.kt | Adds tests for selective/ignored deletion suppression and priority ordering across folders. |
| ui/src/commonTest/kotlin/dev/obiente/nextcloudnative/app/FileSyncCoordinatorSnapshotTest.kt | Extends snapshot fixtures to include selection/ignore/priority configuration fields. |
| ui/src/commonMain/kotlin/dev/obiente/nextcloudnative/app/VirtualFileStorageCenter.kt | Introduces cross-platform virtual-file storage snapshot + action-result model and byte formatting. |
| ui/src/commonMain/kotlin/dev/obiente/nextcloudnative/app/VirtualFileCache.kt | Introduces platform-independent virtual-file cache policy, eviction planning, and open planning. |
| ui/src/commonMain/kotlin/dev/obiente/nextcloudnative/app/NextcloudPlatform.kt | Adds platform service hooks/capabilities for virtual-file storage and desktop start-on-login preference. |
| ui/src/commonMain/kotlin/dev/obiente/nextcloudnative/app/NextcloudNativeApp.kt | Adds settings UI for start-on-login and adds new marketing capture scenarios for sync/VFS. |
| ui/src/commonMain/kotlin/dev/obiente/nextcloudnative/app/FileSyncPlanning.kt | Adds selective sync paths, ignore globs, and priority rules to sync configuration and matching logic. |
| ui/src/commonMain/kotlin/dev/obiente/nextcloudnative/app/FileSyncCoordinatorSnapshot.kt | Persists selection/ignore/priority patterns in coordinator snapshots. |
| ui/src/commonMain/kotlin/dev/obiente/nextcloudnative/app/FileSyncCoordinator.kt | Scopes scans to includesSyncPath and adds priority-aware execution sorting. |
| ui/src/commonMain/kotlin/dev/obiente/nextcloudnative/app/FileSyncCenter.kt | Extends sync-center summaries to optionally include local root path. |
| ui/build.gradle.kts | Adds desktop dependencies (FUSE + JNA) and a new tray-visual-QA capture task. |
| changes/unreleased/11-native-file-sync.md | Adds an unreleased changelog entry describing sync + virtual files + tray outcome. |
| androidApp/src/test/kotlin/dev/obiente/nextcloudnative/AndroidVirtualFileProxyCallbackTest.kt | Adds unit tests for Android ProxyFileDescriptorCallback hydration/range behavior. |
| androidApp/src/test/kotlin/dev/obiente/nextcloudnative/AndroidFileSyncStoreTest.kt | Extends Android sync-store test fixture to include selection/ignore/priority rules. |
| androidApp/src/main/kotlin/dev/obiente/nextcloudnative/AndroidVirtualFileProxyCallback.kt | Adds Android seekable proxy callback with block hydration and complete-generation publish gating. |
| androidApp/src/main/kotlin/dev/obiente/nextcloudnative/AndroidNextcloudServices.kt | Implements virtual-file storage snapshot/actions on Android and enables supportsVirtualFileStorage. |
| androidApp/src/main/kotlin/dev/obiente/nextcloudnative/AndroidMediaStoreSyncLocalTree.kt | Adds selective/ignore filtering to MediaStore local tree scanning. |
| androidApp/src/main/kotlin/dev/obiente/nextcloudnative/AndroidFileSyncRemoteTree.kt | Adds selective/ignore filtering hook to remote scanning. |
| androidApp/src/main/kotlin/dev/obiente/nextcloudnative/AndroidFileSyncLocalTree.kt | Extends local tree interface/implementations to accept includes predicate for filtering. |
| androidApp/src/main/kotlin/dev/obiente/nextcloudnative/AndroidFileSyncEngine.kt | Wires includesSyncPath into Android engine scans so excluded/ignored content is not planned. |
| androidApp/src/androidTest/kotlin/dev/obiente/nextcloudnative/AndroidVirtualFileCacheInstrumentedTest.kt | Adds instrumented coverage for Android virtual-file cache hydration, corruption rejection, and writeback manifests. |
Review details
Suppressed comments (2)
ui/src/desktopMain/kotlin/dev/obiente/nextcloudnative/nativeui/preview/FileSyncTrayVisualQaMain.kt:44
- The fixture path segment
Stoppelhaenelooks like a real-world identifier. For privacy-safe synthetic captures, use obviously synthetic folder names (e.g.,ExampleEvent) so screenshots can never be mistaken for real user content.
ui/src/desktopMain/kotlin/dev/obiente/nextcloudnative/nativeui/preview/FileSyncTrayVisualQaMain.kt:69 - Same as above: keep the synthetic activity list free of real-world identifiers; update the second
Stoppelhaenepath to match the synthetic naming used elsewhere in this fixture.
- Files reviewed: 59/71 changed files
- Comments generated: 2
- Review effort level: Lite
We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.
There was a problem hiding this comment.
💡 Codex Review
https://github.com/Obiente/nc-native/blob/44cd314cc02a76e026b6d32e2958941116cd787c/ui/src/desktopMain/kotlin/dev/obiente/nextcloudnative/app/DesktopFileSyncLocalTree.kt#L185-L188
Every desktop scan treats any filename containing DOWNLOAD_MARKER as an interrupted internal download and deletes it, while any name containing BACKUP_MARKER is deleted or renamed. A legitimate user file such as .notes.nextcloud-native-download-archive is therefore silently removed on the next scan because the suffix is never checked for the UUID format generated by this adapter. Match the complete owned staging-name grammar before performing recovery cleanup.
AGENTS.md reference: AGENTS.md:L221-L223
https://github.com/Obiente/nc-native/blob/44cd314cc02a76e026b6d32e2958941116cd787c/ui/src/desktopMain/kotlin/dev/obiente/nextcloudnative/app/DesktopFileSyncLocalTree.kt#L216-L220
The desktop file revision uses lastModifiedTime().toMillis() and size but no content identity. An in-place same-size rewrite within the same millisecond, or an editor/tool that preserves timestamps, leaves the file key, timestamp, size, and path unchanged, so the coordinator accepts the old baseline and silently skips uploading the new bytes. Preserve the filesystem's full timestamp precision and use a strong content identity when metadata cannot prove the generation changed.
AGENTS.md reference: AGENTS.md:L221-L223
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e323c05138
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: acdb3be60f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 06f8d1b999
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2b958f342d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9b582d667f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
https://github.com/Obiente/nc-native/blob/c81adddd475b2377827a17796fcf12715ade64ae/androidApp/src/main/kotlin/dev/obiente/nextcloudnative/NextcloudDocumentsProvider.kt#L293
If another client renames a document while an external editor still holds its writable descriptor, this MOVE succeeds without checking or updating the active writeback. The close callback later commits to the captured old file.path, and both its durable manifest and automatic recovery continue targeting that now-missing path, leaving the edit permanently unable to publish at the renamed destination. Readdress the active writeback and manifest atomically, or reject the rename until the descriptor closes.
AGENTS.md reference: AGENTS.md:L221-L223
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3cb5c6dc58
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
https://github.com/Obiente/nc-native/blob/b10608a1eb1ae7a04a7e89f32d2d3b16f72de6b5/ui/src/commonMain/kotlin/dev/obiente/nextcloudnative/app/FileSyncPlanning.kt#L519-L523
Every keep-both operation for a given file and device generates the same two conflict-copy paths. After one conflict has populated those paths, a later simultaneous edit reuses them; Android rejects the first create because the destination exists, while desktop verifies that the old copy has different content and fails, so the new conflict can never be resolved. The same collision makes Android recovery fail after a partially published keep-both operation. Allocate generation-specific noncolliding names while preserving stable paths only for retries of that generation.
AGENTS.md reference: AGENTS.md:L221-L223
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
80bcca3 to
6050b82
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6050b82581
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (pending.conflict) { | ||
| pending.releaseActive() | ||
| return@runCatching | ||
| } |
There was a problem hiding this comment.
When recovery detects that the remote ETag changed, it marks the staged edit conflicted, and every subsequent load exits here without processing it; openWritableDocument also rejects recovered conflicts, while the storage UI only reports a count and exposes no keep-local, export, keep-both, or discard action. The user's only copy can therefore remain permanently trapped in app-private recovery storage after an ordinary writeback conflict. Provide a durable user-driven resolution path before treating this state as handled.
AGENTS.md reference: AGENTS.md:L221-L223
Useful? React with 👍 / 👎.
| } catch (_: EOFException) { | ||
| CacheState() | ||
| } catch (_: Exception) { | ||
| CacheState() |
There was a problem hiding this comment.
If the Android cache index is truncated, malformed, or otherwise fails validation, these branches return an empty state without inventorying the account directory. Existing .blob files then disappear from storage summaries and eviction plans, so freeUp cannot remove them and they can continue consuming substantial cache space until a later successful index save happens to clean the directory. Quarantine the invalid index and reconcile or delete unreferenced disposable blobs immediately.
AGENTS.md reference: AGENTS.md:L214-L214
Useful? React with 👍 / 👎.
| if (Files.isDirectory(localPath)) { | ||
| backend.createDirectory(relativePath) | ||
| } else { | ||
| backend.upload(relativePath, localPath.toFile(), expectedRemoteRevision = null) |
There was a problem hiding this comment.
When a large unmanaged file is still being copied after the 750 ms watcher delay, this streams the mutable file directly without recording or rechecking its generation. If the copy finishes after the request has consumed its declared bytes but before placeholder conversion, the queued modification event can subsequently be ignored because the path is no longer Absent, while lines 923-924 mark the shorter remote generation synchronized; Windows may then dehydrate the only complete local bytes. Upload an immutable snapshot, or compare a strong before/after file generation and requeue changes before converting or marking the file in sync.
AGENTS.md reference: AGENTS.md:L217-L218
Useful? React with 👍 / 👎.
| val plan = planVirtualFileEviction( | ||
| entries = entries, | ||
| policy = loadPolicy(), | ||
| availableFreeBytes = root.usableSpace.coerceAtLeast(0L), | ||
| nowEpochMillis = nowEpochMillis, | ||
| ) |
There was a problem hiding this comment.
When cached Android virtual files pass unusedFileAgeMillis without another hydration or policy edit, summary computes an eviction plan containing those expired entries but never applies it. loadVirtualFileStorage only calls this summary, and there is no background caller that executes the plan, so files advertised as automatically removed after the configured age can remain indefinitely until an unrelated cache publication occurs. Apply automatic eviction when loading the storage state or schedule periodic cache maintenance.
AGENTS.md reference: AGENTS.md:L214-L214
Useful? React with 👍 / 👎.
Outcome
Verification
cargo test --lockedValidation includes
:contractAcquisition:test,:ui:desktopTest,:androidApp:testDebugUnitTest,:androidApp:compileDebugAndroidTestKotlin,:ui:createDistributable, and:androidApp:assembleDebug. Focused desktop packaging and start-on-login regression runs also pass.Compatibility and risk
Android uses the system DocumentsProvider surface with editable proxy file descriptors and bounded range caching. Linux uses writable FUSE with staged, conflict-safe writeback. Windows uses Cloud Files sync roots, placeholders, hydration, pinning, dehydration, and guarded edit recovery through native JNA bindings.
Launch-time recovery restores enabled providers, reconciles interrupted writes, and resumes periodic background scanning. ETag and generation preconditions prevent stale writeback from silently overwriting remote changes. Platform state is persisted independently from the tray process.
The shared architecture keeps macOS and iOS provider implementations open, but those backends are not part of this change. Real Windows Explorer validation, privileged Linux mount validation, Android emulator instrumentation, and shell integration checks across GNOME, KDE, and Windows remain required before release qualification.
Visual changes
Synthetic desktop and mobile captures are included for sync setup, live status, rule priority, virtual-file storage, start-on-login settings, and the custom Linux tray. Side-by-side QA comparison boards are included under
design-qa-artifacts/filesync.Closes #108
Closes #117
Refs #109, #111, #122, and #125.
Advances #11