Skip to content

fix(task): use shared memory for fspy ipc#234

Merged
branchseer merged 25 commits intomainfrom
10-16-fix_task_use_shm_for_fspy_ipc
Oct 23, 2025
Merged

fix(task): use shared memory for fspy ipc#234
branchseer merged 25 commits intomainfrom
10-16-fix_task_use_shm_for_fspy_ipc

Conversation

@branchseer
Copy link
Copy Markdown
Member

@branchseer branchseer commented Oct 18, 2025

Implemented a shared memory IPC channel for fspy to improve performance and reliability when tracking file system accesses.

The channel implementation is cross-platform. In this PR it's only adopted in macOS/Linux version of fspy​. Windows version of fspy​ will switch to it in a separate PR.

Why make this change?

The previous IPC implementation uses file descriptor inheritance to pass down the IPC channel. It has the risk of descriptors being closed in the child processes. Some defensive logic was implemented to handle it, including:

It apparently missed some cases, and the IPC fd still got closed, leading to bad descriptor​ error in https://github.com/voidzero-dev/vibe-dashboard/actions/runs/18532354385/job/52818262992.

What changed?

  • Created a new shared memory-based IPC channel implementation for fspy that replaces the previous Unix socket approach.
    • The shared memory is looked up based on names instead of descriptors, making it impossible to be unexpectedly closed in child processes.
    • Proper file locking is added to ensure safe access to shared memory
    • Writing to the IPC channel becomes much faster as it now only does some atomic memory operations and doesn't trigger any syscalls.
  • Modified the semantics of fs access tracking:
    • The previous implementation automatically waits for the root target process and all its descendants to exit due to the nature of file descriptor inheritance.
    • The new implementation requires waiting accesses_future after the root target process has exited. If descendants are spawned after the root target process has exited, their fs accesses will be missed.

Note

Replaces Unix fspy IPC with a lock-backed shared-memory channel and updates payloads, readers, and callers; adds test utils and adjusts env/TLS dependencies.

  • IPC (core):
    • Introduces shared-memory IPC (fspy_shared::ipc::channel) with Sender/Receiver and lock-guarded ShmReader/ShmWriter for zero-syscall writes.
  • fspy (Unix/macOS):
    • Switches from FD/Unix-socket IPC to shared memory; payload now carries ChannelConf instead of an FD.
    • Refactors SpyInner::init_in, fixture handling, and path injection; reads frames via shared memory; lazy-locks receiver after child exit.
  • Preload (Unix):
    • Client sends PathAccess via shared memory; removes POSIX spawn FD-inheritance handling; minor argv handling fix.
  • Windows:
    • Keeps pipe IPC; small fix to NativeStr construction and a temporary task to avoid blocking.
  • API/Semantics & Callers:
    • accesses_future must be awaited after child exit; updates examples, tests, e2e, and vite_task execution flow accordingly.
    • Adds fspy_test_utils crate for cross-process test helpers.
  • Shared types:
    • Enhances NativeStr/NativeString for zero-copy (ANSI/wide) and cross-platform encoding; updates spawn payload/env usage.
  • Dependencies:
    • Adds shared_memory, memmap2, uuid, tracing; tweaks nix features; switches reqwest to native-tls-vendored (adds openssl-src).
  • Misc:
    • Cleans up obsolete files and adjusts test steps (platform filters).

Written by Cursor Bugbot for commit 8f35015. This will update automatically on new commits. Configure here.

Copy link
Copy Markdown
Member Author

branchseer commented Oct 18, 2025

@branchseer branchseer force-pushed the 10-16-fix_task_use_shm_for_fspy_ipc branch from 8a79947 to e524579 Compare October 18, 2025 15:56
@branchseer branchseer changed the title use shared memory for fspy ipc feat: implement shared memory IPC for path access tracking Oct 18, 2025
@branchseer branchseer changed the title feat: implement shared memory IPC for path access tracking fix(task): use shared memory for fspy ipc Oct 18, 2025
@branchseer branchseer changed the base branch from main to graphite-base/234 October 19, 2025 14:09
@branchseer branchseer force-pushed the 10-16-fix_task_use_shm_for_fspy_ipc branch from ac63d0a to 9301e45 Compare October 19, 2025 14:09
@branchseer branchseer changed the base branch from graphite-base/234 to 10-19-test_vitest_browser_mode October 19, 2025 14:09
@branchseer branchseer force-pushed the 10-16-fix_task_use_shm_for_fspy_ipc branch from 5a72a00 to fb36a7f Compare October 20, 2025 02:16
@branchseer branchseer force-pushed the 10-19-test_vitest_browser_mode branch from 8b46431 to 4049e8b Compare October 20, 2025 02:16
@branchseer branchseer force-pushed the 10-16-fix_task_use_shm_for_fspy_ipc branch from fb36a7f to 9ee81fa Compare October 20, 2025 02:19
@branchseer branchseer changed the base branch from 10-19-test_vitest_browser_mode to graphite-base/234 October 20, 2025 02:43
branchseer added a commit that referenced this pull request Oct 20, 2025
- Enable cli e2e test on Linux in ci and skip failed tests
- Add a test for caching vitest browser mode, stably reproducing the issue of https://github.com/voidzero-dev/vibe-dashboard/actions/runs/18532354385/job/52818262992. 

The added vitest browser mode test will be fixed and enabled in #234
@branchseer branchseer force-pushed the 10-16-fix_task_use_shm_for_fspy_ipc branch from 9ee81fa to 4f3b622 Compare October 20, 2025 02:44
@graphite-app graphite-app Bot changed the base branch from graphite-base/234 to main October 20, 2025 02:45
@branchseer branchseer force-pushed the 10-16-fix_task_use_shm_for_fspy_ipc branch from 4f3b622 to 5b562c0 Compare October 20, 2025 02:45
@branchseer branchseer marked this pull request as ready for review October 21, 2025 03:56
Copilot AI review requested due to automatic review settings October 21, 2025 03:56
Comment thread crates/fspy/examples/cli.rs
Comment thread crates/fspy_preload_unix/src/client/mod.rs
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR transitions the fspy IPC mechanism from file descriptor-based communication to a shared memory-based channel implementation. The change improves reliability and cross-platform compatibility by using shared memory for inter-process communication instead of passing file descriptors.

Key changes:

  • Implements a new lock-free shared memory writer/reader system
  • Replaces FD-passing IPC with shared memory channels
  • Removes Linux-specific memfd implementation in favor of unified filesystem-based approach

Reviewed Changes

Copilot reviewed 30 out of 31 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
packages/cli/snap-tests/vitest-browser-mode/steps.json Removes Linux from ignored platforms for browser mode tests
fspy/node_spawn.mjs Removes temporary test file
fspy/.gitignore Removes gitignore file
crates/vite_task/src/execute.rs Changes task execution to await path accesses after child completion instead of concurrently
crates/fspy_test_utils/src/lib.rs Adds new test utility crate for creating subprocess commands with encoded arguments
crates/fspy_test_utils/README.md Adds documentation for fspy_test_utils
crates/fspy_test_utils/Cargo.toml Adds dependencies for the test utilities crate
crates/fspy_shared_unix/src/spawn/macos.rs Updates to use as_os_str() method for path conversions
crates/fspy_shared_unix/src/spawn/linux/mod.rs Updates to use as_os_str() method for path conversions
crates/fspy_shared_unix/src/payload.rs Replaces ipc_fd with ipc_channel_conf in payload structure
crates/fspy_shared/src/ipc/shm/mod.rs Removes placeholder TODO file
crates/fspy_shared/src/ipc/native_str.rs Enhances NativeString to support both Unix and Windows platforms with improved API
crates/fspy_shared/src/ipc/mod.rs Updates module structure to include channel and export NativeString unconditionally
crates/fspy_shared/src/ipc/channel/shm_io.rs Implements lock-free concurrent shared memory writer and reader
crates/fspy_shared/src/ipc/channel/mod.rs Implements mpsc IPC channel based on shared memory
crates/fspy_shared/Cargo.toml Adds dependencies for shared memory and IPC channel functionality
crates/fspy_preload_windows/src/windows/detours/create_process.rs Updates to use from_ansi method for Windows ANSI strings
crates/fspy_preload_unix/src/interceptions/spawn/posix_spawn.rs Removes unused posix_spawn file actions handling and variables
crates/fspy_preload_unix/src/interceptions/spawn/exec/with_argv.rs Fixes type inconsistency in pointer type annotation
crates/fspy_preload_unix/src/client/mod.rs Replaces shared memory FD-passing IPC with channel-based implementation
crates/fspy_e2e/src/main.rs Changes to await accesses_future after child completion
crates/fspy/tests/test_utils.rs Changes to await accesses_future after child completion
crates/fspy/tests/node_fs.rs Changes to await accesses_future after child completion
crates/fspy/src/unix/mod.rs Replaces FD-passing IPC with shared memory channel implementation
crates/fspy/src/lib.rs Simplifies Spy initialization by using filesystem-based approach for all platforms
crates/fspy/src/fixture.rs Adds conditional compilation for unused fixture method
crates/fspy/examples/cli.rs Changes to await child completion before accessing results
crates/fspy/Cargo.toml Adds dependencies for fixture creation on all platforms
bench/Cargo.toml Reorders dependencies alphabetically
Cargo.toml Adds fspy_test_utils workspace member and updates reqwest features

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread crates/fspy_shared/src/ipc/native_str.rs Outdated
Comment thread crates/fspy_shared/src/ipc/native_str.rs Outdated
Comment thread crates/fspy/src/lib.rs Outdated
Copilot AI review requested due to automatic review settings October 21, 2025 07:09
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 30 out of 31 changed files in this pull request and generated 3 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread crates/fspy_shared/src/ipc/native_str.rs Outdated
Comment thread crates/fspy_shared/src/ipc/channel/shm_io.rs Outdated
Comment thread crates/fspy_shared/src/ipc/channel/shm_io.rs Outdated
Copilot AI review requested due to automatic review settings October 21, 2025 07:12
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 30 out of 31 changed files in this pull request and generated 1 comment.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread Cargo.toml Outdated
branchseer and others added 19 commits October 23, 2025 11:42
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	crates/fspy_preload_unix/src/interceptions/spawn/exec/with_argv.rs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: branchseer <3612422+branchseer@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: branchseer <3612422+branchseer@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: branchseer <3612422+branchseer@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: branchseer <3612422+branchseer@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: branchseer <3612422+branchseer@users.noreply.github.com>
Copilot AI review requested due to automatic review settings October 23, 2025 11:42
@branchseer branchseer force-pushed the 10-16-fix_task_use_shm_for_fspy_ipc branch from 21bdcf0 to 05cbeb6 Compare October 23, 2025 11:42
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 30 out of 31 changed files in this pull request and generated no new comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@branchseer branchseer merged commit 92c05a4 into main Oct 23, 2025
10 checks passed
@branchseer branchseer deleted the 10-16-fix_task_use_shm_for_fspy_ipc branch October 23, 2025 11:49
branchseer added a commit that referenced this pull request Oct 23, 2025
Chromium processes can now be successfully tracked after #234, and we have it monitored since #235.
branchseer added a commit that referenced this pull request Oct 25, 2025
# Fix seccomp filter IPC and add static executable support

This PR improves the seccomp filter IPC mechanism by using path-based Unix domain sockets instead of directly passing file descriptors. This approach is consistent with #234 and eliminates the need for pre-exec handling.

Key changes:

- Replace direct fd passing with Unix domain socket communication
- Fix handling of path overflow in CStrPtr by returning a boolean success indicator
- Add support for tracking static executables with tests
- Add a test binary crate that can be included as a static binary for testing
- Implement proper handling of the `open` syscall on x86_64 architecture
- Enable typeaware linting test on Linux

The PR also includes various code improvements and cleanup, such as better error handling and more robust syscall tracking.
branchseer pushed a commit that referenced this pull request Mar 13, 2026
Includes:
- ux: hint Escape key when no tasks match in interactive selector (#235)
- docs: overhaul README.md, CLAUDE.md and add CONTRIBUTING.md (#234)

https://claude.ai/code/session_0139LsSbe67hcD8NKKaK6Y8U
@branchseer branchseer mentioned this pull request Mar 13, 2026
2 tasks
branchseer added a commit that referenced this pull request Mar 13, 2026
## Summary
- Bump vite-task from `28f371a1` to `398a147f`
- Includes: hint Escape key when no tasks match in interactive selector
(#235), overhaul README/CLAUDE.md and add CONTRIBUTING.md (#234)

## Test plan
- [x] `cargo check` passes
- [ ] CI passes

https://claude.ai/code/session_0139LsSbe67hcD8NKKaK6Y8U

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants