Marlinspike Developer Architecture

Marlinspike Developer: Architecture

Technical architecture for the Marlinspike search and replace engine.

Overview

Marlinspike is the cross-tool search and replace engine for the Radoub toolset. It provides find/replace across Aurora Engine GFF file types with per-tool search providers, field-level filtering, and module-wide scanning. Part of Epic #42.

The engine spans three layers:

Radoub.Formats — Search providers, field registry, match models
Radoub.UI — Module search orchestration, progress, tool dispatch
Tool-specific — UI integration (SearchBar, ModuleSearchWindow, keyboard shortcuts)

Component Structure

graph TB
    subgraph Formats["Radoub.Formats / Search"]
        SPF[SearchProviderFactory]
        SFR[SearchFieldRegistry]
        DSP[DlgSearchProvider]
        GSP[GenericGffSearchProvider]
        SM[SearchMatch / SearchCriteria]
    end

    subgraph UI["Radoub.UI / Services / Search"]
        MSS[ModuleSearchService]
        BRS[BatchReplaceService]
        BKS[BackupService]
        TDS[ToolDispatchService]
        FSR[FileSearchResult]
        MSR[ModuleSearchResults]
        SP[ScanProgress]
    end

    subgraph Parley["Parley (Tool Integration)"]
        DSS[DialogSearchService]
        SB[SearchBar Control]
        MSW[ModuleSearchWindow]
        KSM[KeyboardShortcutManager]
    end

    KSM --> SB
    KSM --> MSW
    SB --> DSS
    DSS --> DSP
    MSW --> MSS
    MSW --> BRS
    BRS --> BKS
    BRS --> SPF
    MSS --> SPF
    MSS --> TDS
    SPF --> DSP
    SPF --> GSP
    DSP --> SFR
    GSP --> SFR
    MSS --> FSR
    MSS --> MSR

Layer Architecture

Radoub.Formats Search Layer

Radoub.Formats/Search/
├── Models/
│   ├── SearchCriteria.cs          # Pattern, options, field type filter
│   ├── SearchMatch.cs             # Match result with location context
│   ├── FieldDefinition.cs         # Field name, type, category, IsReplaceable
│   ├── ReplaceOperation.cs        # Single replace instruction (match + replacement text)
│   ├── ReplaceResult.cs           # Result of a single replace (success/skipped + old/new)
│   ├── DlgMatchLocation.cs        # DLG-specific: node type, index, display path
│   └── Enums.cs                   # SearchFieldType, SearchFieldCategory
├── Registry/
│   ├── SearchFieldRegistry.cs     # Central registry of searchable fields per resource type
│   └── FieldRegistrations.cs      # DLG, UTC, UTI, UTM, JRL, IFO, GIT field definitions
└── Providers/
    ├── IFileSearchProvider.cs     # Provider interface (Search + Replace)
    ├── SearchProviderBase.cs      # Shared search + replace helpers (ReplaceStringField, ReplaceLocStringField, ReplaceVarTableField)
    ├── SearchProviderFactory.cs   # Maps ResourceType → provider
    ├── DlgSearchProvider.cs       # Dialog tree search/replace
    ├── UtcSearchProvider.cs       # Creature/BIC search/replace
    ├── UtiSearchProvider.cs       # Item blueprint search/replace
    ├── UtmSearchProvider.cs       # Store/merchant search/replace
    ├── JrlSearchProvider.cs       # Journal search/replace
    ├── GitSearchProvider.cs       # Placed instances search/replace
    ├── UtpSearchProvider.cs       # Placeable search/replace (with inventory)
    ├── UtdSearchProvider.cs       # Door search/replace
    ├── AreSearchProvider.cs       # Area search/replace
    ├── ItpSearchProvider.cs       # Palette search (hierarchical paths)
    ├── FacSearchProvider.cs       # Faction search/replace
    └── GenericGffSearchProvider.cs # Fallback: any GFF tree search/replace

SearchProviderFactory: CreateDefault() returns factory with all registered providers. GetProvider(resourceType) dispatches to the correct implementation.

Radoub.UI Orchestration Layer

Radoub.UI/Services/Search/
├── ModuleSearchService.cs         # Multi-file scan with progress and cancellation
├── BackupService.cs               # Shadow file backup with SHA256 hash verification
├── BatchReplaceService.cs         # Preview → backup → execute → changelog orchestration
├── ToolDispatchService.cs         # ResourceType → tool mapping and --file launch
└── Models/
    ├── FileSearchResult.cs        # Per-file result envelope
    ├── ModuleSearchResults.cs     # Aggregated module results
    ├── ScanProgress.cs            # Progress reporting DTO
    ├── BackupManifest.cs          # Backup record with per-file SHA256 hashes
    └── BatchReplaceModels.cs      # PendingChange, BatchReplacePreview, BatchReplaceResult

See Radoub-UI-Developer for details.

Tool Integration Layer

Each tool provides its own UI integration. Parley's implementation:

Component	Purpose
`DialogSearchService`	Single-file search/replace wrapper around `DlgSearchProvider`
`SearchBar` (UserControl)	Inline search + replace UI (Ctrl+F search, Ctrl+H replace row). Tab order: Search → Replace → Replace → Replace All. No auto-navigate on type — navigation only on F3/Enter/arrows.
`ModuleSearchWindow`	Module-wide search + replace UI with batch replace via `BatchReplaceService`
`KeyboardShortcutManager`	Registers Ctrl+F, Ctrl+H, F3, Shift+F3, Ctrl+Shift+F

See Parley-Developer-Architecture for Parley-specific details.

Data Flow: Single-File Search

sequenceDiagram
    participant U as User
    participant SB as SearchBar
    participant DSS as DialogSearchService
    participant GR as GffReader
    participant DSP as DlgSearchProvider

    U->>SB: Type query (300ms debounce)
    SB->>DSS: Search(filePath, criteria)
    DSS->>GR: Read(filePath)
    GR-->>DSS: GffFile
    DSS->>DSP: Search(gff, criteria)
    DSP->>DSP: Walk entries, replies, starting list
    DSP->>DSP: Match against registered fields
    DSP-->>DSS: SearchMatch[]
    DSS-->>SB: Match count
    SB-->>U: "3 of 12 matches"

Data Flow: Module-Wide Search

sequenceDiagram
    participant U as User
    participant MSW as ModuleSearchWindow
    participant MSS as ModuleSearchService
    participant SPF as SearchProviderFactory
    participant GR as GffReader

    U->>MSW: Enter query, click Search
    MSW->>MSS: ScanModuleAsync(path, criteria, progress, token)
    MSS->>MSS: DiscoverFiles(fileTypeFilter)
    MSS-->>MSW: Progress: "Discovering files"
    loop Each file (Task.Run)
        MSS->>GR: Read(filePath)
        MSS->>SPF: GetProvider(resourceType)
        MSS->>MSS: provider.Search(gff, criteria)
        MSS-->>MSW: Progress: "Searching file.dlg (5/47)"
    end
    MSS-->>MSW: ModuleSearchResults
    MSW->>MSW: Build TreeView (File → Match nodes)
    U->>MSW: Double-click match
    MSW->>MSW: Launch tool instance (--file argument)

Data Flow: Single-File Replace

sequenceDiagram
    participant U as User
    participant SB as SearchBar
    participant DSS as DialogSearchService
    participant GR as GffReader
    participant DSP as DlgSearchProvider
    participant GW as GffWriter

    U->>SB: Click Replace (Ctrl+H row)
    SB->>DSS: ReplaceCurrent(filePath, replacement, criteria)
    DSS->>GR: Read(filePath)
    GR-->>DSS: GffFile
    DSS->>DSP: Replace(gff, [op])
    DSP->>DSP: Navigate to target struct via DlgMatchLocation
    DSP->>DSP: Mutate GFF field value
    DSP-->>DSS: ReplaceResult
    DSS->>GW: Write(gff) → file
    DSS->>DSS: Re-search to update match list
    DSS-->>SB: Updated match count
    SB-->>U: File reloaded, next match highlighted

Data Flow: Module-Wide Replace

sequenceDiagram
    participant U as User
    participant MSW as ModuleSearchWindow
    participant BRS as BatchReplaceService
    participant BKS as BackupService
    participant SPF as SearchProviderFactory
    participant GR as GffReader
    participant GW as GffWriter

    U->>MSW: Click Replace All
    MSW->>BRS: PreviewReplace(fileResults, replacement)
    BRS-->>MSW: BatchReplacePreview (changes grouped by file)
    MSW->>BRS: ExecuteReplaceAsync(preview, moduleName)
    BRS->>BKS: BackupFilesAsync(affectedPaths, moduleName)
    BKS-->>BRS: BackupManifest (with SHA256 hashes)
    loop Each file with changes
        BRS->>GR: Read(filePath)
        BRS->>SPF: GetProvider(resourceType)
        BRS->>BRS: provider.Replace(gff, operations)
        BRS->>GW: Write(gff) → filePath
    end
    BRS-->>MSW: BatchReplaceResult
    MSW-->>U: "Replaced N matches in M files. Backup created."

Replace Safety

Backup before modify: ~/Radoub/Backups/{Module}/{Timestamp}/ with SHA256 per file
Rollback on failure: If any file write fails, BackupService.RestoreAsync() restores all files
ResRef fields excluded: IsReplaceable = false prevents silent reference breakage (#1926)
Reverse offset ordering: Multiple matches in same field applied last-to-first to preserve offsets

Search Field Registry

Central registry maps resource types to searchable fields with categories for UI filtering.

DLG Fields

Category	Fields
Content	Text (LocString entries/replies)
Identity	Speaker, Quest, Comment
Script	ActionScript, ConditionScript
Metadata	Sound, ScriptParams

Other Resource Types

Type	Provider	Fields
UTC/BIC	UtcSearchProvider	FirstName, LastName, Description, Tag, TemplateResRef, Subrace, Deity, Conversation, Comment, 13 event scripts, VarTable, EquipRes (equipped), InventoryRes (backpack)
UTI	UtiSearchProvider	LocalizedName, Description, DescIdentified, Tag, TemplateResRef, Comment
UTM	UtmSearchProvider	LocName, Tag, ResRef, Comment, OnOpenStore, OnStoreClosed, VarTable
UTP	UtpSearchProvider	LocName, Description, Tag, ResRef, Comment, Conversation, 14 event scripts, VarTable, InventoryRes (#1951)
UTD	UtdSearchProvider	LocName, Description, Tag, ResRef, Comment, LinkedTo, 13 event scripts, VarTable
JRL	JrlSearchProvider	Category Name/Tag, Entry Text, Comment (hierarchical location)
ARE	AreSearchProvider	Name, Tag, ResRef, Comments, 4 event scripts
GIT	GitSearchProvider	All string/locstring/resref fields on 8 instance lists (raw GFF)
ITP	ItpSearchProvider	Branch names, category names, blueprint names, blueprint ResRefs (#2001)
FAC	FacSearchProvider	Faction names (#2001)
IFO	(generic fallback)	Module Name/Description, Tag

Search Providers

DlgSearchProvider

Walks the dialog tree structure: StartingList → EntryList → ReplyList. Each node's fields are checked against SearchFieldRegistry DLG definitions. Returns SearchMatch with DlgMatchLocation (node type, index, display path like "Entry #3").

UtcSearchProvider

Searches creature/BIC files. Round-trips GFF → UtcFile for typed field access. Searches 3 LocString fields, 6 string fields, 13 script event fields, VarTable, equipped item ResRefs, and backpack item ResRefs (#1947). Handles both .utc and .bic extensions. Location strings: field name for top-level fields, Equipment > [SlotName] > EquipRes for equipped items (uses EquipmentSlots.GetSlotName()), Backpack > Item [N] > InventoryRes for backpack items.

UtiSearchProvider

Searches item blueprint files. 3 LocString fields (name, description, identified description), 3 string fields (tag, resref, comment). Location is the field name string.

UtmSearchProvider

Searches store/merchant files. 1 LocString (name), 4 string fields, 2 script fields, VarTable. Location is the field name string.

JrlSearchProvider

Walks journal categories and entries. Returns JrlMatchLocation with category index, entry ID, and display path like "Category #0 → Entry #2". Hierarchical structure means entries are always contextualized within their parent category.

GitSearchProvider

Walks 8 instance lists at raw GFF level (no typed model): Creature List, Door List, Encounter List, Placeable List, SoundList, StoreList, TriggerList, WaypointList. Searches all CExoString, CResRef, CExoLocString fields, plus VarTable. Returns GitMatchLocation with instance type, index, tag, and display path like "Creature #0 (LOUIS_ROMAIN)".

UtpSearchProvider

Searches placeable blueprint files. 2 LocString fields (name, description), 6 string fields, 14 script event fields, VarTable, and inventory item ResRefs (#1951). Inventory search iterates ItemList entries, producing locations like Inventory > Item 0 > InventoryRes. InventoryRes fields are non-replaceable (ResRef).

ItpSearchProvider

Searches ITP palette files by walking the parsed palette tree (branches → categories → blueprints). Uses ItpReader.Read(GffFile) directly — no binary round-trip. Returns ItpMatchLocation with hierarchical display paths like "Armor → Medium → King Snake Robe". Searches branch names, category names, blueprint names, and blueprint ResRefs. Blueprint ResRefs are non-replaceable. Replace not yet supported for ITP files (#2001).

FacSearchProvider

Searches FAC faction files. Iterates FactionList entries and searches faction names. Returns FacMatchLocation with faction index and display path like "Faction #2: Commoner". Supports replace by navigating to FactionList[index].FactionName in the GFF struct (#2001).

GenericGffSearchProvider

Fallback provider that recursively walks any GFF struct tree. Matches string-typed fields against criteria. Used for resource types without a specialized provider (e.g., IFO).

Models

SearchCriteria

Property	Type	Purpose
`Pattern`	`string`	Search text or regex
`CaseSensitive`	`bool`	Case-sensitive matching
`WholeWord`	`bool`	Whole word matching
`IsRegex`	`bool`	Interpret pattern as regex
`FieldTypeFilter`	`SearchFieldType[]?`	Restrict to field types (LocString, Text, ResRef, etc.)
`CategoryFilter`	`SearchFieldCategory[]?`	Restrict to field categories (Content, Identity, Script, etc.)
`FileTypeFilter`	`ushort[]?`	Restrict to resource types (applied at discovery time)
`SearchStrRefs`	`bool`	Resolve TLK StrRef values and include in search (#2000). Default false.
`TlkResolver`	`Func<uint, string?>?`	Callback for StrRef resolution. Set by caller (e.g., Marlinspike panel).
`EffectiveTlkResolver`	`Func<uint, string?>?`	Returns `TlkResolver` when `SearchStrRefs` is true, null otherwise. Providers pass this to `SearchLocString`.

SearchMatch

Property	Type	Purpose
`FieldName`	`string`	Field that matched
`MatchedText`	`string`	Matched text content
`Location`	`IMatchLocation`	Provider-specific location context

SearchFieldType

LocString, Text, ResRef, Tag, Script, ScriptParam, Variable

Phased Delivery

Phase	Status	Content
0	Done	Search engine foundation (models, registry, DLG + generic providers)
1	Done	Parley integration (ModuleSearchService, Ctrl+F, Ctrl+Shift+F)
2	Done	Multi-type providers (UTC, UTI, UTM, JRL, GIT)
3	Done	Replace engine, backup, batch replace, tool dispatch, Parley replace UI
4	Done	VarTable replace (#1949), SearchBar tab order (#1950), per-tool search navigation (#1939), AreSearchProvider (#1935), backup cleanup (#1925)
5	Done	UTP inventory search (#1951), ITP/FAC dedicated providers (#2001), StrRef-resolved text search (#2000)
6	Planned	ResRef rename with file rename (#1926)

Usage Guidelines

Adding a New Search Provider

Create provider class implementing IFileSearchProvider in Radoub.Formats/Search/Providers/
Register fields in SearchFieldRegistry via FieldRegistrations
Register provider in SearchProviderFactory for the resource type
ModuleSearchService will automatically use it via SearchProviderFactory

Per-Tool Search Navigation

When the user presses F3/Enter, the SearchBar.NavigateToMatch event fires. Each tool handles this differently based on its UI structure:

Tool	Navigation Behavior	Implementation
Parley	Expands tree path, selects matching dialog node	`FindTreeNodeByReference()` + `ExpandToNode()`
Manifest	Selects matching category/entry in journal tree	Reuses `NavigateToQuest()` with `JrlMatchLocation`
Quartermaster	Switches sidebar panel (Character, Scripts, Advanced, etc.)	Maps `Field.GffPath` → section name, calls `NavigateToSection()`
Fence	Expands collapsed expanders, scrolls to control, focuses it	Maps `Field.GffPath` → named control, uses `BringIntoView()`
Relique	Expands collapsed expanders, scrolls to control, focuses it	Maps `Field.GffPath` → named control, uses `BringIntoView()`

VarTable Replace

SearchProviderBase.ReplaceVarTableField() handles variable name and string value replacement. It parses FullFieldValue ("Name = value" format) to identify the variable, determines whether the match is in the name or value portion, applies the replacement, and writes the full VarTable back via VarTableHelper.WriteVarTable(). Wired into UTC, UTM, UTP, UTD, GIT, and Generic providers via SearchFieldType.Variable case in each provider's Replace() switch.

Adding Search to a New Tool

Create tool-specific search service (wraps provider for single-file use)
Create search UI (can reuse SearchBar pattern or build custom)
Register keyboard shortcuts in tool's KeyboardShortcutManager
For module search, use ModuleSearchService directly — it handles tool dispatch via ToolIdMap

Marlinspike Developer Architecture

Marlinspike Developer: Architecture

Table of Contents

Overview

Component Structure

Layer Architecture

Radoub.Formats Search Layer

Radoub.UI Orchestration Layer

Tool Integration Layer

Data Flow: Single-File Search

Data Flow: Module-Wide Search

Data Flow: Single-File Replace

Data Flow: Module-Wide Replace

Replace Safety

Search Field Registry

DLG Fields

Other Resource Types

Search Providers

DlgSearchProvider

UtcSearchProvider

UtiSearchProvider

UtmSearchProvider

JrlSearchProvider

GitSearchProvider

UtpSearchProvider

ItpSearchProvider

FacSearchProvider

GenericGffSearchProvider

Models

SearchCriteria

SearchMatch

SearchFieldType

Phased Delivery

Usage Guidelines

Adding a New Search Provider

Per-Tool Search Navigation

VarTable Replace

Adding Search to a New Tool

See Also

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!