Wii U Websocket

Proof-of-concept Wii U Gamepad touchscreen → PC mouse cursor interface

---

So, there's a little program called UsendMii - it promises to let you use your Wii U Gamepad as a PC controller. And to give it some credit, for some button inputs, it works really well, the latency is unnoticeable to me. But wanting to mess around with DS and 3DS emulators, I really wanted to use the Gamepad to get the true "resistive touchscreen" feeling¹ that's hard to come by nowadays. And I'm not the only one!

The program teases this function pretty well: you're given a live preview of the touchscreen input, but as it turns out, all you can do with it is map rectangular zones to simulated keyboard presses, similar to when you're running an emulator on your phone. The general concept for the program is great! But this tiny little detail gets so close, but not quite there, and it ruins the whole thing.

So I made my own.

For those out of the loop, here's what's happening:

• There's a local Websocket server for sending messages between devices
• The Wii U browser loads a webpage from our HTTP server
• We weild the stylus and do stuff on the Gamepad
• A script on that page sends messages to the Websocket server
• Meanwhile, a Python script reads them and controls the PC's mouse

And yeah, it all works without having to hack your Wii U! _{(though don't quote me on that, since I don't really have an unhacked Wii U to try this out on.)}

Sounds cool, can I see this in action?

Sure! Here's a demo with me "drawing" in Paint.NET: (also on YouTube)

VID_20230118_050016.8mbvideo.mp4

Of course, this works with anything on your PC, since it just simulates mouse movements. Feel free to use it with Citra, CEMU or any of your other touch-screen needs.

I wanna try running it myself! How do I set this up?

Warning Again, this is a proof of concept. I spent like 2 hours total on this thing and it's not polished or probably bug-free at all. Writing this readme took me more time than actually making the rest.

Prerequisites

Node.js for the Web­socket server² and Python 3 for moving your mouse.

^{(I tried doing this with AutoHotkey instead of Python at first, but it was more painful than I was willing to endure.)}

I'm not too sure if you need these specific versions, but it's what I have and it works for me.

> node --version
v16.13.1

> python --version
Python 3.10.6

Note I only tested this on Windows 10 x64, so I can't guarantee that the program (or the commands used to install it) are going to work or any other systems.

You also need to have your Wii U and PC running on the same network. ... Hey, since you're already going to the internet settings, why not change the DNS to block Nintendo updates?

Components

Here's the most important parts and what they do:

index.js

The Websocket server, hardcoded to port 8080. Tweak as needed.

wiiu.html

The "sender client" that you load up on the Wii U, again with hardcoded things you have to edit.

mousey.py

The "receiver client" that does the mouse moving stuff, with more hardcoded things.
I named it this way to stop Python complaining about "circular dependencies".

Installation

Note Do each step its own separate terminal / command prompt.

1. Set up the Websocket server

npm install express http ws
node .

2. Run the Python script

pip install mouse
python mousey.py

3. Figure out your local IP address (Windows)

ipconfig /all | findstr "IPv4"

Warning Change the code in wiiu.html (line 16) to point to your PC.

If the above command shows several different addresses, drop the filter part and just ipconfig /all and look for the right network adapter name.

4. Open the Wii U Browser

Navigate to the place where you're hosting your edited wiiu.html, making sure not to forget to specifically write the http:// too. (https where applicable) – When I entered my IP without the protocol, I got redirected to a search engine.

Note The web server is not included, use your favorite solution for serving static html files!³

5. Double-check the Websocket connection

Your Wii U should now display a mostly-blank page with two checkboxes on it, and if you look at the Node.js terminal that's got index.js running on it, you should hopefully see that both Python and Wii U connected to it successfully:

Server started on port 8080
Client connected
Client connected

If not, please make sure that you didn't forget to change any of the hard-coded IP addresses and ports to something more suitable for you!

6. Lament the defunct pen-trail feature

Sure enough, I wanted to make it easier to draw with this thing, so I added a canvas to the Wii U's side of things, where a temporary trail would follow your stylus, letting you orientate yourself in your scribbles before they slowly faded out.

Unfortunately, the Wii U doesn't quite handle the browser mouse events like expected. But if you open the html file in your PC browser of choice, and (hold)­drag your mouse on the canvas _{(in the upper left, remember – it's the 480p gamepad resolution)}, it works well!

7. Realize you would've been better off using a DS4 controller

That's right! It has both a touchscreen and motion controls, and as a big plus, it's an actual usable controller.

Usage / Options

You can find two checkboxes on the Wii U page:

• Drag mouse

Enable this to simulate pressing the left mouse button. Off by default, so drawing on the Gamepad screen just moves your stylus, but doesn't click anything.

• Smooth mouse

Poor attempt at mouse stabilization. It's based on functionality in mouse (the Python package), and is on by default, but tweak the duration yourself in mousey.py for best results.

Please note that – while a smart idea – using external stabilization programs like SilkyShark doesn't work, as the two programs end up fighting for control over the mouse, and you might end up with something undesirable that looks like the Beziers screensaver from Windows XP. _{Too lazy to boot up the Wii U again just to make a visual example.} You might have better luck using built-in stabilizers in art programs like FireAlpaca, as those don't tend to directly affect your mouse!

Note If you have trouble clicking the checkboxes, there's a bunch of ways to scale them up, like adjusting the font size with CSS, smaller <meta ... viewport width and height, or changing user-scalable to yes here! Watch out for double-click triggering zooming, though.

FAQ (Frequently anticipated questions)

Can I use this for CEMU?

I guess! I haven't tried it, but it should work! Don't forget to tick the Drag Mouse checkbox. Now here's my question: Why do you want to emulate a Wii U and use a real Wii U to control it?

How do I get my PC screen to show up on the gamepad?

No idea. I've heard people mention libDRC a lot when it comes to this stuff. Other people mention something called Moonlight, which seems to be a way to trick Nvidia software to stream gameplay to your Gamepad as if it was a Shield.

Why does the HTML look so awful and outdated?

I tried to play it extra safe to make sure it works on Wii U's 11 year old browser. Plus, I copied part of it from UsendMii where it was hiding deep inside RCDATA\DATA_INDEXWIIU.

How can I draw better with this thing?

Use some Wii U software that's intended for drawing, like the Miiverse posting part of Splatoon. Oh wait--

This sucks. I could've done better.

I agree! Also, not a question. This isn't meant to be a serious solution, thooouugh if you do wanna improve this bodgy mess, I'll be reading through issues and PRs at least throughout 2023.

Useful link: Official developer reference for the Wii U browser's extended Gamepad capabilities

Let's discuss on Discord!

Footnotes

1. Of course, I acknowledge that it would've been much easier to use a laptop/phone touchscreen (these are capacitive), or maybe even a graphic tablet if you're feeling fancy, but this is much jankier and much cooler. ↩

2. Feel free to use your own thing for the Websocket stuff, all this does is repeat (or "echo") every received message to all connected clients. ↩

3. My go-to is the (now-obsolete) Live Server extension for VS Code, though Caddy server might also work. Since what I used is an extension, there's no easy way for me to include it here, and adding a specific HTML server to the Express stuff in index.js would've taken too much effort. Sorry! ↩