Discord: https://discord.gg/5Yv4kqjAbm

This Android application written in Kotlin is designed to fully automate a run of Uma Musume Pretty Derby by offering a comprehensive set of options to customize event rewards, stat prioritization, race scheduling, automatic skill point buying, and more. Featuring a robust modern frontend built on React Native (Expo) and an extensive computer-vision driven backend, this app aims to solve the issue of spending too much hands-on time completing runs into something you can set and forget.

This project is purely for educational purposes to learn about Android automation and computer vision - basically a fun way to practice coding skills. Any usage is at your own risk. No one will be responsible for anything that happens to you or your own account except for yourself.

• Android Device or Emulator (Nougat 7.0+)
  • Hard requirement for Android phones is 1080p and 240 DPI or 450 DPI (for Samsung phones). If your device do not meet these, you can try the Basic Template Matching Test in the Settings under the Debug Tests section to determine the best scale to use in the Custom Scale for Template Matching setting. If not, then you can also try the To set the phone's resolution to 1080p (faster and more accurate) section to forcibly set the display resolution and DPI of your Android phone. Note that may come with the side-effect of your device UI being scrunched in or zoomed out.
  • Tested emulators are Bluestacks 5 (Pie 64-bit, but other versions should work) and MuMu. The following setup is required:
    • Portrait Mode needs to be forced on always.
    • Bluestacks itself needs to be updated to the latest version to avoid Uma Musume crashing.
    • In the Bluestacks Settings > Phone, the predefined profile needs to be set to a modern high-end phone like the Samsung Galaxy S22.
    • Setup for both BlueStacks and MuMu:
      • 6 CPU Cores
      • 6GB Memory
      • 1080 x 1920 (width x height)
      • 240 DPI (This is important)
      • Disable the equivalent of "Keep alive in background" in the emulator settings to prevent the overlay button from not showing up.
      • Specific to MuMu: the App running emulator setting also needs to be disabled for the same issue.

• Completes a full run from start/midway to its conclusion.
• Supports multiple scenarios including URA Finale, Unity Cup, and those in the future to come.
• Advanced OpenCV template matching for real-time gamestate awareness.
• YOLOv8-powered real-time stat gain detection for improved accuracy.
• Tesseract OCR integrated with rapid fuzzy string matching.
• Modern user interface built using React Native, Typescript and Expo for full configurability.
• Remote Log Viewer to monitor real-time automation progress from any browser on the same network.
• Screen recording for debugging to easily capture and review issues.
• Import/export settings as JSON, alongside customizable skill point buying plans and training configurations.
• Smart racing plan that dynamically schedules extra races based on current stats and fan requirements.
• Training Event customization per event for fine-grained control over choices.
• Load and manage profiles for the Training Settings to easily swap between different builds.
• A multitude of settings to configure including setting preferred stat targets per distance.
• Optional Ask the Docs on-device chatbot. Answers questions about the app grounded in README.md, HOW_IT_WORKS.md, in-app option descriptions, and the Kotlin source code via MiniLM embeddings + a downloaded GGUF model running fully offline through llama.rn. Hidden behind an opt-in toggle on the LLM Settings page; once enabled the page reveals model download, active-model selection (also switchable on the fly from the chat page itself), and generation tuning (max output tokens, per-citation context cap, model context window).

1. Download the latest .apk file from the Releases section on the right of this page and install it on your Android device.
2. Open the application. Upon launching, navigate through the user-friendly frontend to select your desired scenario (URA Finale, Unity Cup, etc.) and configure your training priorities, races, and other settings.
3. You can review your loaded settings and configurations directly on the Home page.
4. Tap the Start button. If this is the first time, you will be prompted to grant Overlay permissions and enable the Accessibility service.

5. Once enabled, tapping Start will request MediaProjection access (select Entire screen if prompted). A floating overlay button will appear that you can drag around the screen.
6. Follow the guidance overlay when you drag the overlay button for the places on the screen to safely leave the button at to avoid covering important UI elements.

7. Navigate to the main training menu in Uma Musume (where Rest, Train, Buy Skills, Races, etc. are visible).

8. Tap the overlay button to start automation.

1. Install Android Studio and create any new project or open an existing one in order for the Logcat console to appear at the bottom.
2. Connect your Android device to your computer:
   • USB Connection: Enable Developer Options and USB Debugging on your device, then connect via USB cable.
   • Wireless Connection: In Developer Options, enable Wireless debugging and pair your device using the pairing code or QR code.
   • Bluestacks or other emulators: In the emulator settings, there is usually an option to allow ADB wireless connection on 127.0.0.1:5555. Enabling that option should be enough, but if Android Studio still does not see it, you can open up a terminal like cmd and type adb connect 127.0.0.1:5555 and it should say connected to 127.0.0.1:5555.

3. In Android Studio's Logcat console at the bottom of the window, select your connected device from the device dropdown menu.
4. Filter the logs by typing package:com.steve1316.uma_android_automation [UAA] or just [UAA] in the search box to see only the logs from this app.
5. Run the app - you'll now see all of its logs appear in real-time as it runs.

1. Install the aShell You app. This allows you to run adb commands locally on your Android device, but requires Shizuku.

2. Install Shizuku, then start it by following these instructions.

3. With Shizuku started, you can then use aShell You to send the following adb commands:

   • Change resolution to 1080p: wm size 1080x1920 && wm density 240
   • Revert to original: wm size reset && wm density reset

   You can also bookmark the commands for your own convenience.

Alternatively, you can do the same on a computer if you cannot get the above to work out.

1. Install adb. You will also to add the file path to the folder to PATH via the Environment Variable setting under View advanced system settings so that the terminal will know what the adb command should do. You may need to restart your computer to have your terminal pick up the changes.
2. Open up a new terminal anywhere (cmd, Powershell, etc).
3. Plug in your Android device via USB. If all goes well, then executing adb devices will show your connected device when Settings > Developer options > USB Debugging is enabled. There may be a popup on your Android device beforehand asking you to give permission to connect to ADB. Wirelessly connecting to ADB is also available via the Android Settings > Developer options > Wireless debugging
4. Execute the following commands individually to forcibly set your display resolution to 1080p and DPI to 240:
   • Change resolution to 1080p: adb shell wm size 1080x1920 and adb shell wm density 240
   • Revert changes: adb shell wm size reset and adb shell wm density reset

This project is separated into a React Native frontend configured via Expo and an extensive Kotlin/OpenCV backend.

1. Download and extract the repository.
2. Download OpenCV for Android (v4.12.0) from https://opencv.org/releases/. Create /android/opencv and copy the extracted /OpenCV-android-sdk/sdk/ contents into it.
3. The project uses a YOLOv8 model for stat gain detection. Ensure the best.onnx model file is present in the android/app/src/main/assets/ directory.

4. The project utilizes Expo. Run yarn install from the root directory to install frontend dependencies.
5. The dev environment is ready. Run yarn start or npx expo start to run the Metro HTTP server.
6. To ensure code consistency, developers should format and lint the codebase using the following commands:
   • yarn format: Formats both TypeScript/TSX files (via Prettier) and Kotlin files (via Ktlint).
   • yarn format:tsx: Formats only TypeScript and TSX files using Prettier.
   • yarn format:kt: Formats only Kotlin files using Ktlint (following settings in android/.editorconfig).
7. To test Android builds, execute yarn android to compile and install the application directly on your device. Use yarn build for release APK generation.

1. eng.traineddata from tessdata
2. MediaProjection - Used to obtain full screenshots
3. AccessibilityService - Used to dispatch gestures like tapping and scrolling
4. OpenCV Android - Used to template match
5. Tesseract4Android - For performing OCR on the screen
6. string-similarity - For comparing string similarities during text detection
7. React Native - Used as the frontend
8. Expo - Modern modular frontend
9. SQLite - Local database via expo-sqlite
10. Ktor - For the Remote Log Viewer
11. YOLOv8 - Object detection
12. ONNX Runtime - Lightweight engine for executing the YOLOv8 model
13. llama.rn - On-device GGUF LLM inference for the Ask the Docs chatbot
14. sentence-transformers/all-MiniLM-L6-v2 - Embedding model powering retrieval over the docs and Kotlin source
15. Qwen 2.5 Instruct GGUF - Default chat model presets for the Ask the Docs chatbot
16. react-native-marked - Renders the chatbot's Markdown answers and citations