Skip to content

2D Overlay System

Jump to bottom
hifihedgehog edited this page May 21, 2026 · 18 revisions

2D Overlay System

PNG overlays on a WPF Canvas show live controller state for Xbox and PlayStation layouts. Custom Extended controllers use a procedurally generated schematic view. Keyboard+Mouse and MIDI types have dedicated preview views.

Architecture Overview

overlay_positions.py          (SVG parsing + alpha-match refinement)
        |
        v
ControllerOverlayLayout.cs   (auto-generated C# layout data)
        |
        v
ControllerModel2DView.xaml.cs (2D PNG overlay view)

ExtendedSlotConfig.ComputeAxisLayout()
        |
        v
ControllerSchematicView.xaml.cs (procedural Extended view)

KeyboardKeyItem.BuildLayout()
        |
        v
KBMPreviewView.xaml.cs         (keyboard + mouse view)

MidiSlotConfig (NoteCount, CcCount, ...)
        |
        v
MidiPreviewView.xaml.cs         (piano + CC slider view)

When Each View Is Used

PadPage.ApplyViewMode() selects the view. Priority: KeyboardMouse > Midi > Custom Extended > standard gamepad (2D/3D toggle).

Condition View Toggle
OutputType == KeyboardMouse KBMPreviewView Hidden
OutputType == Midi MidiPreviewView Hidden
OutputType == Extended ControllerSchematicView Hidden
OutputType == Xbox or PlayStation, Use2DControllerView == true ControllerModel2DView Visible
OutputType == Xbox or PlayStation, Use2DControllerView == false ControllerModelView (3D) Visible

ControllerOverlayLayout.cs

File: PadForge.App/Models2D/ControllerOverlayLayout.cs

Auto-generated by tools/overlay_positions.py. Do not edit manually.

Types

public enum OverlayElementType
{
    Button,         // Show/hide on press
    Trigger,        // Clip-based fill level
    StickRing,      // Translates with stick input
    StickClick,     // Quadrant highlight for hover/flash
    FaceButtonGroup // Reserved (unused)
}

public record OverlayElement(
    string ImageFile,
    string TargetName,
    OverlayElementType ElementType,
    double X,
    double Y,
    double Width,
    double Height
);

Xbox360Layout

public static class Xbox360Layout
{
    public const int BaseWidth = 1545;
    public const int BaseHeight = 955;
    public const string BasePath = "2DModels/XBOX360/XB360_base.png";
    public const double StickMaxTravel = 30;
    public static readonly OverlayElement[] Overlays;  // 19 elements
}

Overlay Elements (19 total):

TargetName ElementType Image Position (X,Y) Size (WxH)
ButtonA Button XB360_A_Button.png 1178, 528 127x106
ButtonB Button XB360_B_Button.png 1312, 415 122x115
ButtonX Button XB360_X_Button.png 1058, 423 126x113
ButtonY Button XB360_Y_Button.png 1190, 314 129x118
LeftShoulder Button XB360_LeftBumper_Active.png 138, 134 312x141
RightShoulder Button XB360_RightBumper_Active.png 1125, 131 285x141
LeftTrigger Trigger XB360_LeftTrigger_Active.png 273, 26 143x152
RightTrigger Trigger XB360_RightTrigger_Active.png 1160, 26 143x152
ButtonBack Button XB360_BackButton.png 546, 439 92x65
ButtonStart Button XB360_StartButton.png 914, 440 92x65
ButtonGuide Button XB360_GuideButton.png 689, 414 171x139
LeftThumbRing StickRing XB360_LeftStick.png 192, 439 211x185
RightThumbRing StickRing XB360_RightStick.png 892, 677 211x178
LeftThumbButton StickClick XB360_LeftStick_Click.png 180, 434 233x195
RightThumbButton StickClick XB360_RightStick_Click.png 884, 662 232x191
DPadUp Button XB360_D-PAD_Up.png 484, 592 108x114
DPadDown Button XB360_D-PAD_Down.png 484, 729 108x114
DPadLeft Button XB360_D-PAD_Left.png 383, 664 134x108
DPadRight Button XB360_D-PAD_Right.png 558, 664 134x107

DS4Layout

public static class DS4Layout
{
    public const int BaseWidth = 1466;
    public const int BaseHeight = 783;
    public const string BasePath = "2DModels/DS4/DS4_V2_base.png";
    public const double StickMaxTravel = 25;
    public static readonly OverlayElement[] Overlays;  // 19 elements
}

DS4 reuses DS4_Face_Button.png for all four face buttons at different positions, DS4_OptionsShare_Button.png for Back and Start, and DS4_AnalogStick_Click.png for both stick clicks.

DualSenseLayout

public static class DualSenseLayout
{
    public const int BaseWidth = 1467;
    public const int BaseHeight = 816;
    public const string BasePath = "2DModels/DualSense/DualSense_base.png";
    public const double StickMaxTravel = 25;
    public static readonly OverlayElement[] Overlays;  // 23 elements
}

DualSense uses dedicated glyphs for the four face buttons (DualSense_Cross.png, Circle, Square, Triangle). Adds LeftTriggerBase and RightTriggerBase elements (the un-pulled trigger artwork sits behind the active fill so the trigger animates as a clip-fill instead of a swap). Touchpad surface is mapped twice: once as a Button (TouchpadClick hit rect, 621x322) and once as a dedicated Touchpad element (499x220 finger-positioning region inset for the active touch area).

XboxOneSLayout

public static class XboxOneSLayout
{
    public const int BaseWidth = 1543;
    public const int BaseHeight = 956;
    public const string BasePath = "2DModels/XBOXONE/XB1_S_base.png";
    public const double StickMaxTravel = 30;
    public static readonly OverlayElement[] Overlays;  // 19 elements
}

Selected for Xbox One, Xbox Elite, and Xbox Adaptive profiles. Trigger elements come as active+base pairs so the trigger pull renders as a clipped fill. No Share button overlay — those profiles do not expose Share.

XboxSeriesXLayout

public static class XboxSeriesXLayout
{
    public const int BaseWidth = 1534;
    public const int BaseHeight = 954;
    public const string BasePath = "2DModels/XBOXSERIES/XBSeries_base.png";
    public const double StickMaxTravel = 30;
    public static readonly OverlayElement[] Overlays;  // 22 elements
}

Selected for Xbox Series profiles. Adds a ButtonShare overlay between View and the dpad — Series profiles are the only Xbox family that exposes Share through HM.

Touchpad Click Highlight

The DS4 touchpad has no PNG overlay in the asset pack, so the TouchpadClick visual is drawn as a WPF Rectangle over the touchpad surface. Color values are sampled from the DS4 face button PNG to keep the highlight consistent with the rest of the layout:

Property Value
Border (Stroke) #24D2F6
Fill #0F7793 at 50% alpha
Stroke thickness 6 px
Corner radius 8 px

The same rectangle doubles as the click hit target for mapping TouchpadClick. Live finger contact dots from Sony Report 0x01 passthrough render on top.

PNG Asset Structure

PadForge.App/2DModels/
  XBOX360/  (21 images)
    XB360_base.png                        (1545x955, base controller image)
    Xbox 360 Controller Overlay.png       (composite overlay for refinement tool)
    XB360_A_Button.png, XB360_B_Button.png, ...
    XB360_LeftBumper_Active.png, XB360_RightBumper_Active.png
    XB360_LeftTrigger.png, XB360_RightTrigger.png         (trigger base)
    XB360_LeftTrigger_Active.png, XB360_RightTrigger_Active.png  (trigger fill)
    XB360_LeftStick.png, XB360_RightStick.png
    XB360_LeftStick_Click.png, XB360_RightStick_Click.png
    XB360_D-PAD_Up/Down/Left/Right.png
    XB360_BackButton.png, XB360_StartButton.png, XB360_GuideButton.png
  XBOXONE/  (19 images)
    XB1_S_base.png                        (1543x956, base controller image)
    Xbox One S Controller Overlay.png     (composite overlay for refinement tool)
    XB1_A_Button.png, XB1_B_Button.png, XB1_X_Button.png, XB1_Y_Button.png
    XB1_LeftBumper_Active.png, XB1_RightBumper_Active.png
    XB1_LeftTrigger.png, XB1_RightTrigger.png             (trigger base)
    XB1_LeftTrigger_Active.png, XB1_RightTrigger_Active.png  (trigger fill)
    XB1_LeftStick.png, XB1_RightStick.png
    XB1_LeftStick_Click.png, XB1_RightStick_Click.png
    XB1_D-PAD_Up/Down/Left/Right.png
    XB1_ViewButton.png, XB1_MenuButton.png, XB1_HomeButton.png
  XBOXSERIES/  (22 images)
    XBSeries_base.png                     (1534x954, base controller image)
    XBSeries_A_Button.png, XBSeries_B_Button.png, XBSeries_X_Button.png, XBSeries_Y_Button.png
    XBSeries_LeftBumper_Active.png, XBSeries_RightBumper_Active.png
    XBSeries_LeftTrigger.png, XBSeries_RightTrigger.png   (trigger base)
    XBSeries_LeftTrigger_Active.png, XBSeries_RightTrigger_Active.png  (trigger fill)
    XBSeries_LeftStick.png, XBSeries_RightStick.png
    XBSeries_LeftStick_Click.png
    XBSeries_D-PAD_Up/Down/Left/Right.png, XBSeries_D-PAD_Center.png
    XBSeries_ViewButton.png, XBSeries_MenuButton.png, XBSeries_HomeButton.png
    XBSeries_ShareButton.png
  DualSense/  (24 images)
    DualSense_base.png                    (1467x816, base controller image)
    DualSense Controller Overlay.png      (composite overlay for refinement tool)
    DualSense_Cross.png, DualSense_Circle.png,
      DualSense_Square.png, DualSense_Triangle.png        (face glyphs)
    DualSense_L1-Active.png, DualSense_R1-Active.png
    DualSense_L2.png, DualSense_R2.png                    (trigger base)
    DualSense_L2-Active.png, DualSense_R2-Active.png      (trigger fill)
    DualSense_LeftAnalogStick.png, DualSense_RightAnalogStick.png
    DualSense_AnalogStick_Click.png                       (shared for both stick clicks)
    DualSense_D-PAD_Up/Down/Left/Right.png
    DualSense_Create_Button.png, DualSense_Option_Button.png, DualSense_Home_Button.png
    DualSense_Mute_Button.png, DualSense_Lightbar.png, DualSense_Gyro-Accel.png
    DualSense_Touchpad_Touch.png, DualSense_Touchpad-Click.png
  DS4/  (16 images)
    DS4_V2_base.png                       (1466x783, base controller image)
    DualShock 4 Controller V2 Model Overlay.png  (composite overlay for refinement tool)
    DS4_Face_Button.png                   (single image for all 4 face buttons)
    DS4_D-PAD_Up/Down/Left/Right.png
    DS4_L1-Active.png, DS4_R1-Active.png
    DS4_L2-Active.png, DS4_R2-Active.png
    DS4_OptionsShare_Button.png           (shared for Back and Start)
    DS4_Home_Button.png
    DS4_V2_LeftAnalogStick.png, DS4_V2_RightAnalogStick.png
    DS4_AnalogStick_Click.png             (shared for both stick clicks)

All PNGs are WPF Resource (not EmbeddedResource), loaded via pack:// URI:

<Resource Include="2DModels\**\*.png" />

Source artwork: Gamepad-Asset-Pack by AL2009man (MIT license).

ControllerModel2DView

File: PadForge.App/Views/ControllerModel2DView.xaml, ControllerModel2DView.xaml.cs

WPF UserControl with a Canvas inside a Viewbox for resolution-independent scaling. About 636 lines of code-behind.

XAML Structure

<Viewbox Stretch="Uniform">
    <Canvas x:Name="ModelCanvas" ClipToBounds="True" />
</Viewbox>

The Viewbox scales the canvas uniformly to fit available space. Canvas dimensions match the layout's BaseWidth/BaseHeight (e.g., 1545x955 for Xbox 360).

Events

public event EventHandler<string> ControllerElementRecordRequested;

Private State

Field Type Description
_vm PadViewModel Bound ViewModel
_loadedModel string One of "XBOX360", "XBOXONE", "XBOXSERIES", "DS4", "DualSense"
_dirty bool Render-frame update flag
_baseImage Image Base controller image at Z=0
_overlayImages Dictionary<string, Image> Target name to overlay Image element
_stickTransforms Dictionary<string, TranslateTransform> Target name to stick translation transform
_triggerClips Dictionary<string, RectangleGeometry> Target name to trigger clip geometry
_elementTypes Dictionary<string, OverlayElementType> Target name to element type
_stickHighlights Dictionary<string, Image> Ring target to stick click overlay image (for quadrant highlights)
_stickMaxTravel double Maximum stick overlay travel in pixels
_flashTimer DispatcherTimer Flash animation timer (400 ms)
_flashTarget string Resolved flash target (ring for axes)
_flashRawTarget string Original target before resolution (e.g., "LeftThumbAxisXNeg")
_flashStickClip Geometry Stored quadrant clip for stick flash
_flashOn bool Current flash toggle state
_hoverTarget string Currently hovered target

ViewModel Binding

public void Bind(PadViewModel vm)
public void Unbind()

Bind subscribes to PropertyChanged, hooks CompositionTarget.Rendering, calls EnsureModel(). Unbind stops flash, unhooks rendering, clears VM reference.

Model Selection

private void EnsureModel()

Resolves the asset folder via HMaestroProfileCatalog.ResolveAssetFolders(ProfileId, OutputType) and dispatches BuildCanvas() against one of five layout classes:

Resolved folder Layout class Profile family
DS4 DS4Layout DualShock 4
DualSense DualSenseLayout DualSense / DualSense Edge
XBOXONE XboxOneSLayout Xbox One, Xbox Elite, Xbox Adaptive
XBOXSERIES XboxSeriesXLayout Xbox Series
anything else Xbox360Layout Xbox 360 fallback

Extended slots route to ControllerSchematicView instead of this view, so this control only ever sees Xbox or PlayStation slots in practice.

Returns immediately if _loadedModel == needed; otherwise calls BuildCanvas().

BuildCanvas()

private void BuildCanvas(string modelName)

Clears the canvas and rebuilds from layout data. Four Z-index layers:

Layer 0. Base Image:

_baseImage = CreateImage(basePath, 0, 0, baseW, baseH);

Layer 1. Overlay Images: For each OverlayElement in the layout:

ElementType Initial State Transform/Clip
StickRing Visible TranslateTransform stored in _stickTransforms
Trigger Visible, full clip RectangleGeometry clip stored in _triggerClips (starts at bottom = empty fill)
Button Collapsed None (toggled by SetOverlayVisible())
StickClick Collapsed None (used as source for quadrant highlights)

All overlay images have IsHitTestVisible = false.clicks pass through to hit-test rectangles.

Layer 10. Hit-Test Rectangles: Transparent Rectangle elements with Cursor = Hand and Tag = TargetName. Events:

  • MouseLeftButtonDown -> HitArea_Click
  • MouseEnter -> HitArea_MouseEnter
  • MouseLeave -> HitArea_MouseLeave
  • MouseMove -> StickHitArea_MouseMove (stick rings only)

StickClick elements have no hit rect.handled by the StickRing's center-click detection.

Layer 5. Stick Quadrant Highlights: Created from StickClick overlay images at 40% opacity. Initially collapsed. Used for hover and flash quadrant display.

Image Loading

private static Image CreateImage(string resourcePath, double x, double y, double w, double h)

Loads a PNG via pack://application:,,,/{resourcePath} URI into a BitmapImage, positioned with Canvas.SetLeft/SetTop.

Per-Frame Updates

CompositionTarget.Rendering handler, gated by _dirty:

UpdateButtons()

Sets overlay visibility for 15 button targets:

SetOverlayVisible("ButtonA", _vm.ButtonA);
SetOverlayVisible("ButtonB", _vm.ButtonB);
// ... 13 more

SetOverlayVisible() skips elements currently being flash-animated or hovered.

UpdateTriggers()

Adjusts RectangleGeometry clip for a fill-from-bottom effect:

double clipY = h * (1.0 - v);  // v = 0: empty (clip at bottom), v = 1: full (clip at top)
clip.Rect = new Rect(0, clipY, w, h - clipY);

UpdateSticks()

Updates TranslateTransform.X/Y from normalized stick values:

lt.X = lx * _stickMaxTravel;
lt.Y = -ly * _stickMaxTravel;  // Y inverted for screen coordinates

Raw short values (-32768–32767) are normalized to -1–1. StickMaxTravel is 30 px for Xbox 360, 25 px for DS4.

Click-to-Record

private void HitArea_Click(object sender, MouseButtonEventArgs e)

For non-stick elements, fires ControllerElementRecordRequested with the Tag value. For stick rings, calls DetermineAxisFromQuadrant():

private static string DetermineAxisFromQuadrant(
    Point pos, double w, double h, string stickTarget)

Determines axis from click position relative to ring center:

  • Center (distance < 30% of radius). LeftThumbButton or RightThumbButton
  • Dominant X (|dx| >= |dy|). LeftThumbAxisX or LeftThumbAxisXNeg
  • Dominant Y. LeftThumbAxisY or LeftThumbAxisYNeg

Down = positive Y (screen coordinates). Step 3's NegateAxis inverts this so screen-down maps to game-down.

Hover Highlight

Buttons: HitArea_MouseEnter shows overlay at 40% opacity. HitArea_MouseLeave sets _dirty = true to restore state on the next render frame.

Triggers: During hover, clip is opened to full image (Rect(0, 0, w, h)).

Stick Rings: StickHitArea_MouseMove tracks mouse position and builds a CombinedGeometry clip on the stick highlight image:

  1. Full ellipse (ring boundary)
  2. Minus center ellipse (30% radius.stick button area)
  3. Intersected with half-rectangle based on dominant axis direction

This produces a quadrant wedge following the mouse:

var quadrant = new CombinedGeometry(GeometryCombineMode.Intersect,
    fullEllipse, new RectangleGeometry(halfRect));
clip = new CombinedGeometry(GeometryCombineMode.Exclude,
    quadrant, centerEllipse);

For center hover (distance < 30%), shows just the center ellipse.

Flash Animation (Map All)

private void UpdateFlashTarget(string target)

Target resolution:

  • Stick axis targets resolve to ring: "LeftThumbAxisX" -> "LeftThumbRing"
  • Stick button targets also resolve to ring: "LeftThumbButton" -> "LeftThumbRing"
  • All others pass through

Flash timer: 400 ms DispatcherTimer toggles _flashOn.

Element Type Flash On Flash Off
Stick axes Quadrant highlight visible (with stored clip) Quadrant highlight collapsed
Stick ring Opacity 1.0 Opacity 0.2
Buttons Overlay visible, full opacity Overlay collapsed
Triggers Clip opened to full fill (restored by StopFlash)

Stick quadrant clip for flash:

private static Geometry GetStickQuadrantClip(string target, double w, double h)

Returns pre-computed clip geometry for the given axis direction:

  • LeftThumbButton / RightThumbButton. Center ellipse (30% radius)
  • AxisX / AxisXNeg. Right or left half-ellipse minus center
  • AxisY / AxisYNeg. Bottom or top half-ellipse minus center

Stored in _flashStickClip and re-applied every tick to guard against clearing by other interactions.

ControllerSchematicView

File: PadForge.App/Views/ControllerSchematicView.xaml, ControllerSchematicView.xaml.cs

Procedurally generated view for custom Extended controllers. Displays stick circles, trigger bars, POV compasses, and button grids. About 794 lines of code-behind.

XAML Structure

<Viewbox Stretch="Uniform">
    <Canvas x:Name="SchematicCanvas" ClipToBounds="True" />
</Viewbox>

Same Viewbox > Canvas pattern as the 2D overlay view. Canvas dimensions computed dynamically to fit all widgets.

Events

public event EventHandler<string> ControllerElementRecordRequested;

Static Brushes

Name Color Usage
AccentBrush #0078D4 Pressed/active elements
FlashBrush #FFA500 Flash animation highlight (orange)
HoverBrush #40A0E0 Hover highlight
DimBrush #606060 Inactive borders
BgBrush #2D2D2D Widget backgrounds
LabelBrush #BBBBBB Text labels
DotBrush #FFFFFF Stick position dot

Layout Constants

const double StickSize = 100;     // Stick circle diameter
const double TriggerWidth = 24;   // Trigger bar width
const double TriggerHeight = 80;  // Trigger bar height
const double PovSize = 60;        // POV circle diameter
const double ButtonSize = 22;     // Button circle diameter
const double SectionGap = 24;     // Horizontal gap between widget sections
const double LabelHeight = 18;    // Space reserved for labels above widgets
const double Padding = 12;        // Canvas edge padding
const int ButtonsPerRow = 8;      // Button grid wrap count

Widget Structs

private struct StickWidget
{
    public int AxisXIndex, AxisYIndex;
    public Ellipse Dot;                    // Position indicator dot
    public Polygon DirectionArrow;         // Flash/hover direction arrow
    public Canvas ArrowCanvas;             // Arrow container for rotation
    public Ellipse OuterCircle;            // Outer boundary circle
    public double X, Y;                    // Canvas position
}

private struct TriggerWidget
{
    public int AxisIndex;
    public Rectangle Background;           // Border rectangle
    public Rectangle Fill;                 // Fill bar (grows from bottom)
    public double X, Y;
}

private struct PovWidget
{
    public int PovIndex;
    public Polygon Arrow;                  // Direction arrow polygon
    public Canvas ArrowCanvas;             // Arrow container for rotation
    public Ellipse Outer;                  // Outer boundary circle
    public double CenterX, CenterY;
}

private struct ButtonWidget
{
    public int ButtonIndex;
    public Ellipse Circle;                 // Button circle
}

ViewModel Binding

public void Bind(PadViewModel vm)
public void Unbind()

Also subscribes to _vm.ExtendedConfig.PropertyChanged so the layout rebuilds when axis/button counts change.

Property change handling:

  • ExtendedOutputSnapshot -> sets _dirty flag
  • OutputType -> calls RebuildLayout()
  • CurrentRecordingTarget -> calls UpdateFlashTarget()
  • Any ExtendedConfig property -> calls RebuildLayout()

RebuildLayout()

private void RebuildLayout()

Clears all widget lists and canvas children, then recreates widgets.

Axis index assignment via ExtendedSlotConfig.ComputeAxisLayout():

  • stickAxisX[] / stickAxisY[]. HID axis indices for each stick pair
  • triggerAxis[]. HID axis indices for standalone triggers

Layout flow (left to right, starting at x = Padding):

+-- Sticks --+-- Triggers --+-- POVs --+
|            |               |          |
| Stick 1    | T1  T2        | D-Pad    |
| [circle]   | [bar][bar]    | [circle] |
|            |               |          |
+---- Buttons (wrapped grid, below main row) ----+
| [1][2][3][4][5][6][7][8]                        |
| [9][10][11]...                                  |
+------------------------------------------------+

Canvas dimensions fit all widgets plus padding, enabling Viewbox auto-scaling.

Widget Creation Methods

CreateStickWidget

private StickWidget CreateStickWidget(int index, int axisXIdx, int axisYIdx, double x, double y)

Creates:

  1. Outer circle (Ellipse). Dim stroke, dark background, hand cursor
  2. Crosshair lines (Line x2). Horizontal and vertical at 50% opacity
  3. Position dot (Ellipse). 10 px accent-colored dot at center
  4. Direction arrow (Polygon inside Canvas). Hidden until flash/hover, rotated via RotateTransform
  5. Label (TextBlock). "Stick N" above the circle

Hover: MouseMove on the outer circle determines quadrant from mouse position, shows direction arrow with HoverBrush, highlights stroke.

Click-to-record quadrant detection:

Condition Target
|dx| > |dy|, dx > 0 ExtendedAxis{axisXIdx} (positive X)
|dx| > |dy|, dx <= 0 ExtendedAxis{axisXIdx}Neg (negative X)
|dy| > |dx|, dy > 0 ExtendedAxis{axisYIdx} (positive Y = down)
|dy| > |dx|, dy <= 0 ExtendedAxis{axisYIdx}Neg (negative Y = up)

CreateTriggerWidget

private TriggerWidget CreateTriggerWidget(int index, int axisIdx, double x, double y)

Creates:

  1. Background (Rectangle). Dim stroke, dark fill, rounded corners, hand cursor
  2. Fill bar (Rectangle). Accent-colored, height 0 initially, grows from bottom
  3. Label. "TN" above the bar

Click-to-record: fires ExtendedAxis{axisIdx}.

CreatePovWidget

private PovWidget CreatePovWidget(int index, double x, double y)

Creates:

  1. Outer circle (Ellipse). Dim stroke, dark background
  2. Arrow (Polygon inside Canvas). Triangular, rotated around POV center, initially hidden
  3. Label. "D-Pad" (1 POV) or "POV N" (multiple)

Hover: direction arrow rotated to mouse quadrant (0/90/180/270 degrees).

Click-to-record: quadrant detection fires ExtendedPov{index}Up, Down, Left, or Right.

CreateButtonWidget

private ButtonWidget CreateButtonWidget(int index, double x, double y)

Creates:

  1. Circle (Ellipse). Dim stroke, dark background, hand cursor
  2. Label (TextBlock). Button number (1-indexed) centered in circle

Click-to-record: fires ExtendedBtn{index}.

Click-to-Record Target Names

Widget Target Format Quadrant-Based
Stick ExtendedAxis{X} / ExtendedAxis{X}Neg Yes (X vs Y, positive vs negative)
Trigger ExtendedAxis{N} No
POV ExtendedPov{N}Up / Down / Left / Right Yes (4 cardinal directions)
Button ExtendedBtn{N} No

Per-Frame Rendering

CompositionTarget.Rendering handler reads ExtendedOutputSnapshot from PadViewModel:

Sticks:

double nx = (raw.Axes[w.AxisXIndex] - (double)short.MinValue) / 65535.0;  // 0–1
double dotX = w.X + nx * (StickSize - 10);

Maps ExtendedRawState.Axes[index] from signed short (-32768–32767) to 0–1 normalized, positioning the dot within the stick circle.

Triggers:

double value = (raw.Axes[w.AxisIndex] - (double)short.MinValue) / 65535.0;  // 0–1
double fillH = Math.Clamp(value, 0, 1) * (TriggerHeight - 4);
w.Fill.Height = fillH;
Canvas.SetTop(w.Fill, w.Y + TriggerHeight - 2 - fillH);  // Grows from bottom

POVs:

int povValue = raw.Povs[w.PovIndex];  // Centidegrees (0–35900) or -1
if (povValue >= 0 && povValue <= 36000)
{
    w.Arrow.Visibility = Visibility.Visible;
    w.ArrowCanvas.RenderTransform = new RotateTransform(povValue / 100.0, ...);
}

POV updates are skipped when the widget is hovered or flash-targeted to prevent flickering from competing rotations.

Buttons:

bool pressed = raw.IsButtonPressed(w.ButtonIndex);
w.Circle.Fill = pressed ? AccentBrush : BgBrush;

Flash Animation

private void UpdateFlashTarget(string target)
private void ApplyFlashState(bool highlight)

400 ms DispatcherTimer, same interval as all other views. Strips "Neg" suffix for matching, then checks each widget list:

Widget Flash On Flash Off
Stick Orange stroke, direction arrow visible, arrow rotated to target axis Dim stroke, arrow hidden
Trigger Fill color = FlashBrush (orange) Fill color = AccentBrush (blue)
Button Stroke = FlashBrush, thicker Stroke = DimBrush, normal
POV Arrow visible, FlashBrush fill, rotated to target direction Arrow visible, AccentBrush fill

POV direction mapping:

string dir = target.Substring($"ExtendedPov{w.PovIndex}".Length);  // "Up", "Down", "Left", "Right"
double angle = dir switch
{
    "Up" => 0,
    "Right" => 90,
    "Down" => 180,
    "Left" => 270,
    _ => 0
};

KBMPreviewView

File: PadForge.App/Views/KBMPreviewView.xaml, KBMPreviewView.xaml.cs

WPF UserControl for Keyboard+Mouse virtual controllers. Displays a full QWERTY keyboard above a mouse graphic. About 502 lines of code-behind.

XAML Structure

<Grid>
    <Grid.RowDefinitions>
        <RowDefinition Height="*"/>       <!-- Keyboard (majority) -->
        <RowDefinition Height="Auto"/>    <!-- Mouse below -->
    </Grid.RowDefinitions>

    <Viewbox Grid.Row="0" Stretch="Uniform">
        <Canvas x:Name="KeyboardCanvas" Width="556" Height="136" ClipToBounds="True"/>
    </Viewbox>

    <Viewbox Grid.Row="1" Stretch="Uniform" HorizontalAlignment="Center" MaxHeight="180">
        <Canvas x:Name="MouseCanvas" Width="160" Height="195" ClipToBounds="True"/>
    </Viewbox>
</Grid>

Keyboard Canvas

BuildKeyboardCanvas() generates Border elements per key from KeyboardKeyItem.BuildLayout() (full ANSI QWERTY with numpad, 556x136 layout units). Each key has:

  • CornerRadius(3) rounded corners
  • KeyNormalBrush background (semi-transparent gray), KeyPressedBrush (accent blue) on press
  • ToolTip with key label
  • MouseLeftButtonDown fires ControllerElementRecordRequested with "KbmKey{VKeyIndex:X2}"
  • Hover: border highlight via HoverBrush

Mouse Canvas

BuildMouseCanvas() draws a mouse graphic using WPF Path geometry:

Element Target Name Description
Mouse body . Rounded Path outline (MouseBodyBrush fill)
LMB KbmMBtn0 Path curving around scroll wheel gap
RMB KbmMBtn1 Mirror of LMB
MMB channel . Background fill between button paths
Scroll wheel pill KbmMBtn2 Rounded Rectangle for middle-click
Scroll up arrow KbmScroll Triangular Polygon on wheel
Scroll down arrow KbmScrollNeg Triangular Polygon on wheel
Movement circle KbmMouseX/Y/Neg Ellipse with direction arrow, quadrant click detection
Side buttons KbmMBtn3, KbmMBtn4 Small Rectangle on left body edge (X1, X2)

Movement circle interaction:

  • MouseMove determines quadrant and shows direction arrow with HoverBrush
  • Click fires KbmMouseX (right), KbmMouseXNeg (left), KbmMouseY (up), or KbmMouseYNeg (down)
  • Dot tracks KbmOutputSnapshot.MouseDeltaX/Y within the circle

Per-Frame Rendering

CompositionTarget.Rendering handler reads KbmOutputSnapshot from PadViewModel:

  • Keyboard keys: kbm.GetKey(vKeyIndex) -> accent/normal brush
  • Mouse buttons: kbm.GetMouseButton(0/1/2) -> accent fill on LMB/RMB/wheel
  • Movement dot: kbm.MouseDeltaX/Y mapped to circle position, accent when non-zero
  • Scroll arrows: kbm.ScrollDelta -> accent fill on up/down arrow

Flash Animation

400 ms DispatcherTimer, same pattern as other views. Target elements get FlashBrush (orange) on tick, restored on stop.

Events

public event EventHandler<string> ControllerElementRecordRequested;

Same click-to-record pattern as all other views. Bind(vm) / Unbind() lifecycle matches 2D/3D/Schematic views.

overlay_positions.py

File: tools/overlay_positions.py

Python tool that generates ControllerOverlayLayout.cs from Gamepad-Asset-Pack SVG files.

Dependencies

pip install svgpathtools lxml opencv-python numpy

Process

  1. Parse SVG. Reads labeled elements from Xbox 360 and DS4 SVG theme files via lxml. Computes cumulative SVG transforms (translate, scale, matrix) for pixel-space bounding boxes.
  2. Center overlays. Loads each PNG overlay and centers it on the SVG bounding box center.
  3. Alpha-channel refinement. OpenCV template matching (cv2.matchTemplate, TM_CCOEFF_NORMED) against the composite overlay image, 40 px search radius, confidence threshold > 0.3.
  4. Generate C#. Outputs ControllerOverlayLayout.cs with layout constants, overlay element arrays, and stick travel values.

Key Functions

def parse_transform(transform_str) -> np.ndarray        # SVG transform -> 3x3 matrix
def get_cumulative_transform(elem) -> np.ndarray         # Walk ancestors for total transform
def element_bbox(elem) -> tuple                          # Single element bbox
def group_bbox(group_elem) -> tuple                      # Combined bbox of group children
def get_element_pixel_bbox(root, label, scale) -> tuple  # Label lookup + pixel conversion
def center_overlay_on_bbox(bbox, overlay_path) -> tuple  # Center overlay on bbox
def refine_with_composite(composite_path, results, search_radius=40) -> list
def process_xbox360() -> dict                            # Xbox 360 pipeline
def process_ds4() -> dict                                # DS4 pipeline
def generate_csharp(xbox_data, ds4_data, output_path)    # C# code generation

Usage

python tools/overlay_positions.py

Expects Gamepad-Asset-Pack/Controller Asset Pack/ as a sibling of the PadForge repository directory.

MidiPreviewView

File: PadForge.App/Views/MidiPreviewView.xaml, MidiPreviewView.xaml.cs

WPF UserControl for MIDI virtual controllers. Displays a piano keyboard for note outputs and vertical CC sliders. About 530 lines of code-behind.

XAML Structure

<Viewbox Stretch="Uniform">
    <Canvas x:Name="MidiCanvas" ClipToBounds="True" />
</Viewbox>

Same Viewbox > Canvas pattern as the Schematic view. Canvas dimensions computed dynamically to fit all widgets.

Events

public event EventHandler<string> ControllerElementRecordRequested;

Static Brushes

Name Color Usage
AccentBrush #0078D4 Active CC fill
AccentDimBrush #005090 (reserved)
DimBrush #606060 Inactive borders, labels
BgBrush #2D2D2D CC bar backgrounds
LabelBrush #BBBBBB Text labels
WhiteKeyBrush #F0F0F0 White piano key fill
WhiteKeyPressedBrush #40A0E0 White key pressed
BlackKeyBrush #202020 Black piano key fill
BlackKeyPressedBrush #0060B0 Black key pressed
KeyBorderBrush #404040 Piano key outlines
HoverBrush #40A0E0 Hover highlight
FlashBrush #FFA500 Flash animation highlight (orange)

Layout Constants

const double WhiteKeyWidth = 28;
const double WhiteKeyHeight = 120;
const double BlackKeyWidth = 18;
const double BlackKeyHeight = 75;
const double CcBarWidth = 20;
const double CcBarHeight = 100;
const double SectionGap = 20;
const double LabelHeight = 16;
const double Padding = 12;

Widget Structs

private struct CcSliderWidget
{
    public int CcIndex;
    public Rectangle Background;
    public Rectangle Fill;
    public double X, Y;
}

private struct PianoKeyWidget
{
    public int NoteIndex;
    public bool IsBlack;
    public Rectangle Rect;
    public Brush NormalBrush;
    public Brush PressedBrush;
}

ViewModel Binding

public void Bind(PadViewModel vm)
public void Unbind()

Subscribes to both _vm.PropertyChanged and _vm.MidiConfig.PropertyChanged. Any MidiConfig property change triggers RebuildLayout().

Property change handling:

  • MidiOutputSnapshot -> sets _dirty flag
  • OutputType -> calls RebuildLayout()
  • CurrentRecordingTarget -> calls UpdateFlashTarget()
  • Any MidiConfig property -> calls RebuildLayout()

RebuildLayout()

private void RebuildLayout()

Layout order:

  1. CC sliders (if mc.CcCount > 0). Section label + one CreateCcSlider() per CC output, horizontal. CC numbers from mc.GetCcNumbers().
  2. Piano keyboard (if mc.NoteCount > 0). Section label + BuildPianoKeys(). Note numbers from mc.GetNoteNumbers().

Canvas sized to fit both sections.

CreateCcSlider

Creates:

  1. Background (Rectangle). Dim stroke, dark fill, rounded corners, hand cursor
  2. Fill bar (Rectangle). Accent-colored, height 0 initially, grows from bottom
  3. Label. CC number below the bar (centered, 9 pt)

Click-to-record: fires MidiCC{index}.

BuildPianoKeys

Two-pass layout:

  1. White keys first. Placed left-to-right, positions stored by MIDI note number.
  2. Black keys on top. Offset by WhiteKeyWidth - BlackKeyWidth/2 from the preceding white key, Z-index 10.

Note labels (e.g., "C4", "F#5") placed below white keys only. Uses a 12-element IsBlackKey[] array and NoteNames[] for display.

Click-to-record: fires MidiNote{noteIndex}.

Per-Frame Rendering

CompositionTarget.Rendering handler reads MidiOutputSnapshot from PadViewModel:

CC sliders:

double value = raw.CcValues[w.CcIndex] / 127.0;  // 0–1
double fillH = Math.Clamp(value, 0, 1) * (CcBarHeight - 4);
w.Fill.Height = fillH;
Canvas.SetTop(w.Fill, w.Y + CcBarHeight - 2 - fillH);  // Grows from bottom

Piano keys:

bool pressed = raw.Notes != null && w.NoteIndex < raw.Notes.Length && raw.Notes[w.NoteIndex];
w.Rect.Fill = pressed ? w.PressedBrush : w.NormalBrush;

Flash-animated keys are skipped during render to avoid overwriting the highlight.

Flash Animation

400 ms DispatcherTimer, same pattern as all other views.

Widget Flash On Flash Off
CC slider Fill color = FlashBrush (orange) Fill color = AccentBrush (blue)
Piano key Key fill = FlashBrush (orange) Key fill = NormalBrush (white or black)

CC flash also matches MidiCC{N}Neg targets (negative direction).

Click-to-Record Target Names

Widget Target Format Quadrant-Based
CC slider MidiCC{N} No
Piano key MidiNote{N} No

How All Views Relate

All five views share the same architecture:

  1. ViewModel interface. Read from PadViewModel, fire ControllerElementRecordRequested with PadSetting target names.
  2. Bind/Unbind lifecycle. PadPage.BindActiveModelView() calls Unbind() on all five, then Bind(vm) on the active one. Only one view processes CompositionTarget.Rendering at a time.
  3. Dirty-flag rendering. CompositionTarget.Rendering gated by _dirty, set by PropertyChanged.
  4. Interactions. Click-to-record, hover highlight, flash animation (Map All). Same target names across views.
  5. Model selection (2D/3D only). EnsureModel() maps OutputType to "DS4" for PlayStation and "XBOX360" for everything else. Extended slots route to ControllerSchematicView instead, so this logic only fires for Xbox and PlayStation slots.

Key differences:

Aspect 3D View 2D View Schematic View KBM View MIDI View
Technology HelixToolkit.WPF Canvas + BitmapImage Canvas + WPF Shapes Canvas + WPF Shapes/Paths Canvas + WPF Shapes
Rotation Left-drag turntable None None None None
Quadrant detection 3D ray-cast + position 2D mouse position 2D mouse position 2D mouse position None
Flash rate 400 ms 400 ms 400 ms 400 ms 400 ms
Output type Xbox, PlayStation Xbox, PlayStation Custom Extended KeyboardMouse MIDI
Assets OBJ meshes (EmbeddedResource) PNG images (Resource) None (procedural) None (procedural) None (procedural)
Config rebuild Model type change Model type change ExtendedConfig change OutputType change MidiConfig change

See Also

  • 3D Model System: ControllerModelView (HelixToolkit 3D alternative to 2D overlay)
  • ViewModels: PadViewModel properties bound by all five preview views
  • XAML Views: PadPage hosts and switches between 2D, 3D, schematic, KBM, and MIDI views
  • Virtual Controllers: Output type determines which preview view is active
  • Engine Library: Gamepad, ExtendedRawState, KbmRawState, MidiRawState snapshot structs
  • Build and Publish: 2D PNG assets (2DModels/) included as WPF Resource items

Clone this wiki locally