Control and automate Nintendo Switch from a browser. A tool developed with React and Arduino.
PhantomHand is a tool that allows you to control and automate your Nintendo Switch from your browser. If you just want to know how to use it quickly, please see the Getting Started Guide.
Briefly, this is a web application for the NintendoSwitch macro controller.
Specifically, it will have the following functionality: with React and Arduino.
- Operate the Switch in the browser GUI
- Save a series of operations as a command (macro function)
- Recall and reproduce saved commands (macro replay function)
- Share the command with your followers via Twitter
The differences from the macro tools created by the great pioneers are as follows.
- Works with Google Chrome on Windows / Mac / Linux
- Macro commands can be created while watching the Switch screen
- Can be connected via Bluetooth
- Can be used for Smash Bros (most important!)
You can save commands entered in the browser GUI as macros.
If you connect a pro-controller to your PC, you can also create macros by operating the pro-controller.
Recall saved macros with the browser GUI. Unlimited number of macros can be saved, and the macro command to be called can be changed with a single click. Repeating playback is also supported.
- Arduino Leonard (Atmega32u4)
- BLE Serial Device (CC2840R2F) or USB Serial Device (FT232)
- Jumper wire
Please see the Getting Started Guide for installation instructions.
The Arduino Leonardo microcontroller will allow you to control the Switch remotely.The Arduino Leonardo contains the ATmega 32U4 chip.
The Arduino Leonardo is powered by the ATmega 32U4 chip, which has the unique feature of working as a USB device.
This means that the Arduino Leonardo can be configured to behave as a controller when connected to the Switch.
The logic for acting as a controller has been made into a library by a great pioneer under the MIT license.
https://github.com/lefmarna/NintendoSwitchControlLibrary
PhantomHand uses the NintendoSwitchControlLibrary to control the Switch from a web browser.
The GUI was developed in React with the possibility of making it a native application with Electron or ReactNative in the future.
Serial communication was adopted to link the browser and Arduino Leonardo. controller operation signals are sent to the serial communication module connected to the Arduino via wireless (WebBluetooth) or wired (WebUsb).
The controller operation signals are sent to the Arduino as an 8-bit binary data array. The length of the array is 2, and its contents are the identifiers of the controller's buttons and sticks (0 - 21) and the state of the buttons and sticks (0 or 1, 0 - 255).
In order to realize the macro function, in addition to the binary sent to the Arduino, the time data of "how many seconds after the start will that command fire" is added and saved as JSON.
type command = {
t: number, // Time
s: Uint8Array[] // Signal => [buttonNumber, buttonState]
}[]The browser GUI is not suitable for creating commands for games with complex operations such as Smash Bros.Therefore, a function to acquire the status of the pro-controller was added using the JavaScript GamePad API.
When you connect the pro-controller to your PC via Bluetooth or USB, you can play the game as if it were directly connected to the Switch while still going through JavaScript and Arduino.
https://developer.mozilla.org/ja/docs/Web/API/Gamepad_API/Using_the_Gamepad_API
https://www.npmjs.com/package/react-gamepad
Use Rison to embed JSON in the query parameter of a URL. When that URL is shared, anyone who accesses it can emulate that command on Switch by simply pressing the play button. It will be interesting to see if this catches on.
To avoid reaching Twitter's character limit with overly long URLs, a URL shortening service called Bitly was used. (It's a free service, so you may reach the limit soon.
https://app.bitly.com/Bl9miVlf2zr/bitlinks/3AUZ7PB
https://www.npmjs.com/package/rison
Here's the React project deployed on Firebase.
You can experience the demo by connecting to an Arduino with a program written on it.
For more information on how to build an Arduino, please refer to the Getting Started Guide.
I've been testing it myself for about a week, and I've made surprisingly good progress in practicing Smash Bros. I'm sure that Pokémon and RPG enthusiasts will be happy with it. If there seems to be a reasonable demand for it, I'm interested in commercializing it.
However, I have almost no knowledge of hardware mass production, so it will take time.
If you are a game peripheral manufacturer interested in mass production, please send me a direct message.
Here is a list of articles and other Switch automation ideas that I found useful. If you're interested in Switch automation after reading this article, I hope you'll explore the various ways to automate your Switch!
This method uses a Python library called Joycontrol, which sends the controller signal via Bluetooth, so there is no need to attach a device to the Switch. It's revolutionary.
https://github.com/mart1nro/joycontrol
ESP-32 is a microcontroller module with built-in WiFi and Bluetooth.
https://github.com/mizuyoukanao/UARTSwitchCon
There are several Arduino microcontrollers (Atmega32u4) with the ESP8266 WiFi module, which can be used to turn itself into a server, or connected to WiFi to turn it into an IoT device.
https://ja.aliexpress.com/i/32839674193.html
If you connect this to the Switch, you can control it from outside the house. .... It seems like it could be used for a lot of interesting things. I think this idea has the most potential for development.