Skip to content

2D Overlay System

Jump to bottom
hifihedgehog edited this page Mar 3, 2026 · 18 revisions

2D Overlay System

The 2D controller visualization uses PNG overlay images positioned on a Canvas to show controller state. Supports Xbox 360 and DualShock 4 layouts. For custom vJoy controllers, a procedurally-generated schematic view is used instead.

Architecture Overview

overlay_positions.py          (SVG parsing + alpha-match refinement)
        |
        v
ControllerOverlayLayout.cs   (auto-generated C# layout data)
        |
        v
ControllerModel2DView.xaml.cs (2D PNG overlay view)
ControllerSchematicView.xaml.cs (procedural vJoy view)

ControllerOverlayLayout.cs

File: PadForge.App/Models2D/ControllerOverlayLayout.cs

Auto-generated by tools/overlay_positions.py. Do not edit manually.

OverlayElementType

public enum OverlayElementType
{
    Button,
    Trigger,
    StickRing,
    StickClick,
    FaceButtonGroup
}

OverlayElement

public record OverlayElement(
    string ImageFile,
    string TargetName,
    OverlayElementType ElementType,
    double X,
    double Y,
    double Width,
    double Height
);

Xbox360Layout

public static class Xbox360Layout
{
    public const int BaseWidth = 1545;
    public const int BaseHeight = 955;
    public const string BasePath = "2DModels/XBOX360/XB360_base.png";
    public const double StickMaxTravel = 30;
    public static readonly OverlayElement[] Overlays;  // 19 elements
}

Overlay Elements (19 total):

TargetName ElementType Image Position (X,Y) Size (WxH)
ButtonA Button XB360_A_Button.png 1178, 528 127x106
ButtonB Button XB360_B_Button.png 1312, 415 122x115
ButtonX Button XB360_X_Button.png 1058, 423 126x113
ButtonY Button XB360_Y_Button.png 1190, 314 129x118
LeftShoulder Button XB360_LeftBumper_Active.png 138, 134 312x141
RightShoulder Button XB360_RightBumper_Active.png 1125, 131 285x141
LeftTrigger Trigger XB360_LeftTrigger_Active.png 273, 26 143x152
RightTrigger Trigger XB360_RightTrigger_Active.png 1160, 26 143x152
ButtonBack Button XB360_BackButton.png 546, 439 92x65
ButtonStart Button XB360_StartButton.png 914, 440 92x65
ButtonGuide Button XB360_GuideButton.png 689, 414 171x139
LeftThumbRing StickRing XB360_LeftStick.png 192, 439 211x185
RightThumbRing StickRing XB360_RightStick.png 892, 677 211x178
LeftThumbButton StickClick XB360_LeftStick_Click.png 180, 434 233x195
RightThumbButton StickClick XB360_RightStick_Click.png 884, 662 232x191
DPadUp Button XB360_D-PAD_Up.png 484, 592 108x114
DPadDown Button XB360_D-PAD_Down.png 484, 729 108x114
DPadLeft Button XB360_D-PAD_Left.png 383, 664 134x108
DPadRight Button XB360_D-PAD_Right.png 558, 664 134x107

DS4Layout

public static class DS4Layout
{
    public const int BaseWidth = 1466;
    public const int BaseHeight = 783;
    public const string BasePath = "2DModels/DS4/DS4_V2_base.png";
    public const double StickMaxTravel = 25;
    public static readonly OverlayElement[] Overlays;  // 19 elements
}

DS4 reuses DS4_Face_Button.png for all four face buttons (same highlight overlay at different positions). Also reuses DS4_OptionsShare_Button.png for both Back and Start, and DS4_AnalogStick_Click.png for both stick clicks.

PNG Asset Structure

PadForge.App/2DModels/
  XBOX360/
    XB360_base.png                        (1545x955, base controller image)
    Xbox 360 Controller Overlay.png       (composite overlay for refinement)
    XB360_A_Button.png, XB360_B_Button.png, ...
    XB360_LeftBumper_Active.png, XB360_RightBumper_Active.png
    XB360_LeftTrigger_Active.png, XB360_RightTrigger_Active.png
    XB360_LeftStick.png, XB360_RightStick.png
    XB360_LeftStick_Click.png, XB360_RightStick_Click.png
    XB360_D-PAD_Up/Down/Left/Right.png
    XB360_BackButton.png, XB360_StartButton.png, XB360_GuideButton.png
  DS4/
    DS4_V2_base.png                       (1466x783, base controller image)
    DualShock 4 Controller V2 Model Overlay.png  (composite overlay for refinement)
    DS4_Face_Button.png                   (single image for all 4 face buttons)
    DS4_D-PAD_Up/Down/Left/Right.png
    DS4_L1-Active.png, DS4_R1-Active.png
    DS4_L2-Active.png, DS4_R2-Active.png
    DS4_OptionsShare_Button.png           (shared for Back and Start)
    DS4_Home_Button.png
    DS4_V2_LeftAnalogStick.png, DS4_V2_RightAnalogStick.png
    DS4_AnalogStick_Click.png             (shared for both stick clicks)

All PNG files are included as WPF Resource (not EmbeddedResource):

<Resource Include="2DModels\**\*.png" />

Source artwork: Gamepad-Asset-Pack by AL2009man (MIT license).

ControllerModel2DView

File: PadForge.App/Views/ControllerModel2DView.xaml, ControllerModel2DView.xaml.cs

WPF UserControl with a Canvas inside a Viewbox for resolution-independent scaling.

Events

public event EventHandler<string> ControllerElementRecordRequested;

Private State

Field Type Description
_vm PadViewModel Bound ViewModel
_loadedModel string "XBOX360" or "DS4"
_dirty bool Render-frame update flag
_baseImage Image Base controller image at Z=0
_overlayImages Dictionary<string, Image> Target name to overlay Image
_stickTransforms Dictionary<string, TranslateTransform> Target name to stick translation
_triggerClips Dictionary<string, RectangleGeometry> Target name to trigger clip geometry
_elementTypes Dictionary<string, OverlayElementType> Target name to element type
_stickHighlights Dictionary<string, Image> Ring target to stick click overlay (for quadrant highlights)
_stickMaxTravel double Maximum stick overlay travel in pixels
_flashTimer DispatcherTimer Flash animation timer
_flashTarget string Resolved flash target (ring for axes)
_flashRawTarget string Original target before resolution
_flashStickClip Geometry Stored quadrant clip for stick flash
_hoverTarget string Currently hovered target

ViewModel Binding

public void Bind(PadViewModel vm)
public void Unbind()

Bind subscribes to PropertyChanged, hooks CompositionTarget.Rendering, calls EnsureModel(). Unbind stops flash, unhooks rendering, clears VM reference.

BuildCanvas()

private void BuildCanvas(string modelName)

Clears canvas and creates:

  1. Base image at Z=0.
  2. Overlay images at Z=1:
    • StickRing: Always visible, TranslateTransform applied for stick position.
    • Trigger: Always visible, RectangleGeometry clip controls fill level (gas tank effect).
    • Button/StickClick: Hidden until pressed (Visibility.Collapsed).
  3. Hit-test rectangles at Z=10: Transparent rectangles with Cursor = Hand, Tag = TargetName. Handles MouseLeftButtonDown, MouseEnter, MouseLeave. StickClick elements have no hit rect (handled by StickRing center-click detection).
  4. Stick quadrant highlights at Z=5: Created from StickClick overlay images with 40% opacity for quadrant highlighting.

Per-Frame Updates

CompositionTarget.Rendering handler, gated by _dirty:

UpdateButtons() -- Sets overlay Visibility based on PadViewModel button bools (15 buttons):

SetOverlayVisible("ButtonA", _vm.ButtonA);  // etc. for all 15

UpdateTriggers() -- Adjusts RectangleGeometry clip to create fill-level effect:

double clipY = h * (1.0 - v);
clip.Rect = new Rect(0, clipY, w, h - clipY);

UpdateSticks() -- Updates TranslateTransform.X/Y from normalized stick values:

lt.X = lx * _stickMaxTravel;
lt.Y = -ly * _stickMaxTravel;  // Y inverted for screen coords

Quadrant Detection

private static string DetermineAxisFromQuadrant(
    Point pos, double w, double h, string stickTarget)

Used for stick ring clicks:

  • Center (distance < 30% of radius): Returns stick button (LeftThumbButton or RightThumbButton).
  • Dominant X axis: Returns LeftThumbAxisX or LeftThumbAxisXNeg.
  • Dominant Y axis: Returns LeftThumbAxisY or LeftThumbAxisYNeg.

Down = positive Y (screen coordinates, inverted by Step 3 in mapping pipeline).

Hover Highlight

  • Buttons: Shows overlay at 40% opacity on MouseEnter, restores on MouseLeave.
  • Stick rings: Shows quadrant clip with CombinedGeometry:
    • Full ellipse minus center circle (30% radius).
    • Intersected with half-rectangle based on dominant axis direction.
    • Mouse position tracked via StickHitArea_MouseMove.
  • Triggers: Shows full fill image during hover.

Flash Animation (Map All)

private void UpdateFlashTarget(string target)

Resolves target to visual element:

  • Stick axis targets resolve to ring target (LeftThumbAxisX -> LeftThumbRing).
  • Stick button targets also resolve to ring.
  • All others pass through.

Flash timer at 400ms toggles:

  • Stick axes: Quadrant highlight image visibility toggled with pre-computed clip geometry from GetStickQuadrantClip().
  • Buttons: Overlay visibility toggled.
  • Triggers: Full fill shown on flash-on.
  • Stick rings: Opacity toggled between 1.0 and 0.2.
private static Geometry GetStickQuadrantClip(
    string target, double w, double h)

Returns clip geometry for the specified axis direction (half-ellipse minus center).

ControllerSchematicView

File: PadForge.App/Views/ControllerSchematicView.xaml, ControllerSchematicView.xaml.cs

Procedurally generated view for custom vJoy controllers. Displays stick circles, trigger bars, POV compasses, and button grids arranged left-to-right.

Events

public event EventHandler<string> ControllerElementRecordRequested;

Static Brushes

Name Color Usage
AccentBrush #0078D4 Pressed/active elements
FlashBrush #FFA500 Flash animation highlight
HoverBrush #40A0E0 Hover highlight
DimBrush #606060 Inactive borders
BgBrush #2D2D2D Widget backgrounds
LabelBrush #BBBBBB Text labels
DotBrush #FFFFFF Stick position dot

Layout Constants

const double StickSize = 100;
const double TriggerWidth = 24;
const double TriggerHeight = 80;
const double PovSize = 60;
const double ButtonSize = 22;
const double SectionGap = 24;
const double LabelHeight = 18;
const double Padding = 12;
const int ButtonsPerRow = 8;

Widget Structs

private struct StickWidget
{
    public int AxisXIndex, AxisYIndex;
    public Ellipse Dot;
    public Polygon DirectionArrow;
    public Canvas ArrowCanvas;
    public Ellipse OuterCircle;
    public double X, Y;
}

private struct TriggerWidget
{
    public int AxisIndex;
    public Rectangle Background;
    public Rectangle Fill;
    public double X, Y;
}

private struct PovWidget
{
    public int PovIndex;
    public Polygon Arrow;
    public Canvas ArrowCanvas;
    public Ellipse Outer;
    public double CenterX, CenterY;
}

private struct ButtonWidget
{
    public int ButtonIndex;
    public Ellipse Circle;
}

RebuildLayout()

private void RebuildLayout()

Called on bind and when VJoyConfig properties change. Uses VJoySlotConfig.ComputeAxisLayout() for axis index assignment.

Layout flow (left to right):

  1. Sticks -- Circles with crosshairs and position dots. Label: "Stick N".
  2. Triggers -- Vertical bars with fill that grows from bottom. Label: "TN".
  3. POVs -- Circles with direction arrows. Label: "D-Pad" (if 1 POV) or "POV N".
  4. Buttons -- Wrapped grid of numbered circles (8 per row), positioned below the main row.

Canvas size set to fit all widgets for Viewbox scaling.

Widget Creation Methods

private StickWidget CreateStickWidget(int index, int axisXIdx, int axisYIdx, double x, double y)
private TriggerWidget CreateTriggerWidget(int index, int axisIdx, double x, double y)
private PovWidget CreatePovWidget(int index, double x, double y)
private ButtonWidget CreateButtonWidget(int index, double x, double y)

Each method creates WPF shapes, adds them to SchematicCanvas, wires hover (highlight stroke on enter, dim on leave) and click-to-record events.

Click-to-Record Target Names

Widget Target Format Quadrant-Based
Stick VJoyAxis{X} / VJoyAxis{X}Neg Yes (X vs Y, positive vs negative)
Trigger VJoyAxis{N} No
POV VJoyPov{N}Up / Down / Left / Right Yes (4 quadrants)
Button VJoyBtn{N} No

Per-Frame Rendering

CompositionTarget.Rendering handler reads VJoyOutputSnapshot from PadViewModel:

  • Sticks: Maps VJoyRawState.Axes[index] from signed short (-32768..32767) to 0-1 normalized, positions dot.
  • Triggers: Maps axis value to fill height (0-1 of TriggerHeight - 4).
  • POVs: Rotates arrow by Povs[index] / 100.0 degrees. Hides when centered (< 0 or > 36000). Skips update during hover or flash to prevent flickering.
  • Buttons: Accent fill when IsButtonPressed(index), background fill otherwise.

Flash Animation

private void UpdateFlashTarget(string target)
private void ApplyFlashState(bool highlight)

Flash timer at 170ms (~3Hz). Matches target against widgets:

  • Sticks: Highlights outer circle stroke, shows directional arrow rotated to target axis direction.
  • Triggers: Toggles fill color between FlashBrush and AccentBrush.
  • Buttons: Toggles stroke between FlashBrush and DimBrush.
  • POVs: Shows arrow pointing in target direction with FlashBrush fill.

overlay_positions.py

File: tools/overlay_positions.py

Python tool that auto-generates ControllerOverlayLayout.cs from Gamepad-Asset-Pack SVG files.

Dependencies

pip install svgpathtools lxml opencv-python numpy

Process

  1. Parse SVG elements -- Reads labeled elements from Xbox 360 and DS4 SVG theme files using lxml. Computes cumulative SVG transforms to get pixel-space bounding boxes.

  2. Center overlays -- For each element, loads the corresponding PNG overlay image and centers it on the SVG bounding box center.

  3. Alpha-channel refinement -- Uses OpenCV template matching (cv2.matchTemplate with TM_CCOEFF_NORMED) against a full composite overlay image to refine positions within a 40-pixel search radius. Only accepts matches with confidence > 0.3.

  4. Generate C# -- Outputs the ControllerOverlayLayout.cs file with layout constants, overlay element arrays, and stick travel values.

Key Functions

def parse_transform(transform_str) -> np.ndarray        # SVG transform -> 3x3 matrix
def get_cumulative_transform(elem) -> np.ndarray         # Walk ancestors for total transform
def element_bbox(elem) -> tuple                          # Single element bbox (path/ellipse/rect)
def group_bbox(group_elem) -> tuple                      # Combined bbox of group children
def get_element_pixel_bbox(root, label, scale) -> tuple  # Label lookup + pixel conversion
def center_overlay_on_bbox(bbox, overlay_path) -> tuple  # Center overlay image on bbox
def refine_with_composite(composite_path, results, search_radius=40) -> list  # Template matching
def process_xbox360() -> dict                            # Full Xbox 360 pipeline
def process_ds4() -> dict                                # Full DS4 pipeline
def generate_csharp(xbox_data, ds4_data, output_path)    # C# code generation

Usage

python tools/overlay_positions.py

Expects Gamepad-Asset-Pack/Controller Asset Pack/ to be a sibling of the PadForge repository directory.

Clone this wiki locally