Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 8 additions & 54 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,57 +12,8 @@ A hooking-based frame streaming tool with support for RetroArch shader processin
| :---: |
| ![showcase_crt_royale](showcase-crt-royale.png)|

Goggles captures Vulkan application frames and shares them across processes using Linux DMA-BUF with DRM format modifiers and cross-process GPU synchronization.

```text
┌───────────────────────────────────────┐
│ Target Application │
│ ┌─────────────┐ │
│ │ Swapchain │ │
│ └──────┬──────┘ │
│ │ vkQueuePresentKHR │
│ ▼ │
│ ┌─────────────────────────────────┐ │
│ │ Capture Layer │ │
│ │ Export DMA-BUF │ │
│ └──────────────┬──────────────────┘ │
└─────────────────┼─────────────────────┘
│ Unix Socket + Semaphore Sync
┌─────────────────┼─────────────────────┐
│ Goggles Viewer │ │
│ ┌──────────────┴──────────────────┐ │
│ │ CaptureReceiver │ │
│ └──────────────┬──────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────┐ │
│ │ VulkanBackend │ │
│ │ Import DMA-BUF ──► FilterChain │ │
│ └─────────────────────────────────┘ │
└───────────────────────────────────────┘
```

The filter chain transforms captured DMA-BUF images through a series of shader passes before presenting to the display. It supports RetroArch `.slangp` preset files which define multi-pass post-processing effects (CRT simulation, scanlines, etc.).

```text
Shader Preset (.slangp)
┌──────────────────────────────┼─────────────────────────────┐
│ ▼ │
│ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ │
│ │Original│────▶│ Pass 0 │────▶│ Pass 1 │────▶│ Pass N │ │
│ └────────┘ ┌─▶└────────┘ ┌─▶└────────┘ ┌─▶└────────┘ │
│ │ │ │ │ │ │ │ │
│ └──────┴───────┴──────┴───────┴──────┘ ▼ │
│ Output │
└────────────────────────────────────────────────────────────┘
┌───────────────────┐
│ Goggles Swapchain │
└───────────────────┘
```
Goggles captures Vulkan frames, runs a shader filter chain, and can optionally forward input via a
nested compositor.

## Shader Preset Compatibility Database

Expand All @@ -76,7 +27,7 @@ The filter chain transforms captured DMA-BUF images through a series of shader p
| **crt/crt-royale.slangp** | Pass | Partial | `Mesa: RDNA3` | Full verification pending after the shader parameter controlling support. |
| **crt/zfast-crt.slangp** | Pass | Verified | `Mesa: RDNA3`, `Proprietary: Ada` | |

*More reports pending validation...*
- [Shader Compatibility Report](docs/shader_compatibility.md) - Full compilation status for all RetroArch presets

## Build

Expand All @@ -102,8 +53,9 @@ target together. The preset defaults to `debug`.

```bash
# Quick smoke test (build + manifests as needed)
pixi run start vkcube --wsi xcb # preset=debug
pixi run start -p release vkcube --wsi xcb # preset=release
pixi run start -- vkcube --wsi xcb # preset=debug
pixi run start -p release -- vkcube --wsi xcb # preset=release
pixi run start --input-forwarding -- vkcube --wsi xcb

# Standard flow
pixi run build # 1. Build the project
Expand Down Expand Up @@ -131,6 +83,8 @@ See [docs/architecture.md](docs/architecture.md) for project architecture and de
Topic-specific docs:
- [Threading](docs/threading.md) - Concurrency model and job system
- [DMA-BUF Sharing](docs/dmabuf_sharing.md) - Cross-process GPU buffer sharing
- [Input Forwarding](docs/input_forwarding.md) - Forward keyboard/mouse to captured apps
- [Wayland + wlroots Input Primer](docs/wayland_wlroots_input_primer.md) - Concepts + code map for maintainers
- [Filter Chain](docs/filter_chain_workflow.md) - RetroArch shader pipeline
- [RetroArch](docs/retroarch.md) - Shader preset compatibility
- [Shader Compatibility Report](docs/shader_compatibility.md) - Full compilation status for all RetroArch presets
Expand Down
7 changes: 1 addition & 6 deletions cmake/Dependencies.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -62,13 +62,8 @@ find_package(slang REQUIRED CONFIG)

find_package(Vulkan REQUIRED)

# Input forwarding dependencies (X11/XWayland for input injection)
# Input forwarding dependencies (wlroots + XWayland for seat-based input delivery)
find_package(PkgConfig REQUIRED)
pkg_check_modules(wlroots REQUIRED IMPORTED_TARGET wlroots-0.18)
pkg_check_modules(wayland-server REQUIRED IMPORTED_TARGET wayland-server)
pkg_check_modules(xkbcommon REQUIRED IMPORTED_TARGET xkbcommon)
find_package(X11 REQUIRED)

if(NOT X11_XTest_FOUND)
message(FATAL_ERROR "libXtst (XTest extension) is required for input forwarding")
endif()
3 changes: 2 additions & 1 deletion config/goggles.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ backend = "vulkan_layer"
# =============================================================================
[input]
# Enable input forwarding into the captured app.
# When enabled, Goggles starts a nested XWayland server and forwards SDL events via XTest.
# When enabled, Goggles starts a nested wlroots compositor (headless Wayland + XWayland) and
# forwards SDL events via `wlr_seat_*` so both Wayland and X11 clients receive real input events.
# Default is false to match pre-input-forwarding behavior.
forwarding = false

Expand Down
60 changes: 32 additions & 28 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,33 +8,37 @@ High-level overview of the Goggles codebase for maintainers. Start here to under

Goggles captures frames from Vulkan applications and applies real-time shader effects before display.

```text
┌───────────────────────────────────────┐
│ Target Application │
│ ┌─────────────┐ │
│ │ Swapchain │ │
│ └──────┬──────┘ │
│ │ vkQueuePresentKHR │
│ ▼ │
│ ┌─────────────────────────────────┐ │
│ │ Capture Layer │ │
│ │ Export DMA-BUF │ │
│ └──────────────┬──────────────────┘ │
└─────────────────┼─────────────────────┘
│ Unix Socket + Semaphore Sync
┌─────────────────┼─────────────────────┐
│ Goggles Viewer │ │
│ ┌──────────────┴──────────────────┐ │
│ │ CaptureReceiver │ │
│ └──────────────┬──────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────┐ │
│ │ VulkanBackend │ │
│ │ Import DMA-BUF ──► FilterChain │ │
│ └─────────────────────────────────┘ │
└───────────────────────────────────────┘
```mermaid
flowchart TB
%% High-level overview: frame path + optional input forwarding.

subgraph AppProc["Target application process"]
App["Target application (Vulkan)"]
Layer["Vulkan capture layer"]
App --> Layer
end

subgraph IPC["Cross-process transport"]
Socket["Unix socket (frames + metadata)"]
Sync["Sync handles (frame_ready / frame_consumed)"]
end

subgraph Goggles["Goggles viewer process"]
Receiver["CaptureReceiver"]
Render["VulkanBackend"]
Chain["FilterChain"]
Window["Viewer window (SDL + swapchain)"]
Receiver --> Render --> Chain --> Window
end

Layer --> Socket --> Receiver
Layer --> Sync --> Render

subgraph Input["Optional: input forwarding"]
Window -->|"keyboard/mouse"| Forwarder["InputForwarder"]
Forwarder --> Nested["Nested compositor (wlroots headless + XWayland)"]
App <-->|"client connection"| Nested
end
```

## Module Overview
Expand Down Expand Up @@ -90,7 +94,7 @@ See: [threading.md](threading.md)

```
1. Target app renders frame
└─▶ vkQueuePresentKHR intercepted by capture layer
└─▶ present call intercepted by capture layer

2. Capture layer exports swapchain image as DMA-BUF
└─▶ Sends fd + metadata over Unix socket
Expand Down
73 changes: 38 additions & 35 deletions docs/dmabuf_sharing.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,33 +13,31 @@ This document explains how Goggles shares GPU textures between processes using L

## 1. Architecture Overview

```text
┌───────────────────────────────────────┐
│ Target Application │
│ ┌─────────────┐ │
│ │ Swapchain │ │
│ └──────┬──────┘ │
│ │ vkQueuePresentKHR │
│ ▼ │
│ ┌─────────────────────────────────┐ │
│ │ Capture Layer │ │
│ │ Export DMA-BUF │ │
│ └──────────────┬──────────────────┘ │
└─────────────────┼─────────────────────┘
│ Unix Socket + Semaphore Sync
┌─────────────────┼─────────────────────┐
│ Goggles Viewer │ │
│ ┌──────────────┴──────────────────┐ │
│ │ CaptureReceiver │ │
│ └──────────────┬──────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────┐ │
│ │ VulkanBackend │ │
│ │ Import DMA-BUF ──► FilterChain │ │
│ └─────────────────────────────────┘ │
└───────────────────────────────────────┘
```mermaid
flowchart TB
%% Frame transport only (input forwarding is covered in docs/input_forwarding.md).

subgraph AppProc["Target application process"]
App["Target application (Vulkan)"]
Layer["Vulkan capture layer"]
App --> Layer
end

subgraph IPC["Cross-process transport"]
Socket["Unix socket (DMA-BUF fd + metadata)"]
Sync["Sync handles (frame_ready / frame_consumed)"]
end

subgraph Goggles["Goggles viewer process"]
Receiver["CaptureReceiver"]
Backend["VulkanBackend (DMA-BUF import)"]
Chain["FilterChain"]
Present["Viewer swapchain present"]
Receiver --> Backend --> Chain --> Present
end

Layer --> Socket --> Receiver
Layer --> Sync --> Backend
```

---
Expand Down Expand Up @@ -132,14 +130,19 @@ Binary semaphores require one-to-one signal/wait pairing. Timeline semaphores us
| `frame_consumed` | Goggles viewer | Capture layer | Frame rendered (back-pressure) |

**Synchronization flow:**
```
Capture Layer Goggles Viewer
───────────── ──────────────
1. Wait frame_consumed[N-1]
2. Copy swapchain → export image
3. Signal frame_ready[N] ─────────────► 4. Wait frame_ready[N]
5. Import + render frame
◄───────────── 6. Signal frame_consumed[N]
```mermaid
sequenceDiagram
participant Layer as Capture layer (target process)
participant Viewer as Goggles viewer

Layer->>Layer: Wait frame_consumed[N-1]
Layer->>Layer: Produce/export frame (DMA-BUF)
Layer->>Viewer: Send metadata + DMA-BUF fd
Layer->>Viewer: Signal frame_ready[N]

Viewer->>Viewer: Wait frame_ready[N]
Viewer->>Viewer: Import + render frame
Viewer-->>Layer: Signal frame_consumed[N]
```

### Required Extensions
Expand Down
50 changes: 17 additions & 33 deletions docs/filter_chain_workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,23 +6,17 @@ This document describes the filter chain architecture for applying RetroArch sha

The filter chain transforms captured DMA-BUF images through a series of shader passes before presenting to the display. It supports RetroArch `.slangp` preset files which define multi-pass post-processing effects (CRT simulation, scanlines, etc.).

```text
Shader Preset (.slangp)
┌──────────────────────────────┼──────────────────────────────┐
│ ▼ │
│ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ │
│ │Original│────▶│ Pass 0 │────▶│ Pass 1 │────▶│ Pass N │ │
│ └────────┘ ┌─▶└────────┘ ┌─▶└────────┘ ┌─▶└────────┘ │
│ │ │ │ │ │ │ │ │
│ └──────┴───────┴──────┴───────┴──────┘ ▼ │
│ (Original available to all passes) Output │
└─────────────────────────────────────────────────────────────┘
┌───────────────────┐
│ Goggles Swapchain │
└───────────────────┘
```mermaid
flowchart LR
Preset["Shader preset (.slangp)"] --> Chain["FilterChain"]

Original["Original frame<br/>(imported image)"]
Chain --> P0["Pass 0"] --> P1["Pass 1"] --> PN["Pass N"] --> Out["OutputPass"] --> Swap["Viewer swapchain"]

%% Original texture is available to every pass (not just Pass 0).
Original -.-> P0
Original -.-> P1
Original -.-> PN
```

## Data Flow
Expand Down Expand Up @@ -121,22 +115,12 @@ Values are written at reflection-reported offsets, not hardcoded struct layout.

## Preset Lifecycle

```
load_shader_preset(path)
PresetParser::load() # Parse .slangp file
RetroArchPreprocessor # Resolve #include, extract parameters
FilterPass::init_from_sources() # For each pass
├──▶ ShaderRuntime::compile() # Slang → SPIR-V
├──▶ SlangReflect::reflect() # Extract bindings, push constants
├──▶ create_pipeline() # Vulkan graphics pipeline
└──▶ create_descriptor_resources()
```mermaid
flowchart TB
Load["Load preset path"] --> Parse["Parse .slangp"]
Parse --> Resolve["Resolve includes + parameters"]
Resolve --> Build["Build passes (compile + pipeline resources)"]
Build --> Ready["Ready to record frames"]
```

## Format Handling
Expand Down
Loading