Android automations without ADB - directly on emulators
CyAndroEmu is a powerful Python library designed for advanced Android automation. For performance reasons, it’s primarily written in C++ and Cython (Line count - C++: 20229 lines / Python: 2960 lines / Cython: 9397 lines), providing multiple backends for interacting with UI elements, handling input events, and efficiently extracting screen data.
Unlike any other Android automation library, CyAndroEmu is designed to run directly and independently on the emulator or rooted device without relying on ADB. This architecture offers significant performance and reliability advantages.
-
No Broken Connections: Since it operates independently of ADB, you’ll never face issues with unstable or dropped connections.
-
Blazing Fast Automation: No need for slow ADB screenshot transmissions—screen data is processed directly on the device.
-
Unlimited Device Scalability: Run as many emulators as your hardware can handle without hitting ADB’s device limits.
-
Natural Mouse Movements: Supports smooth, human-like mouse movements, making it perfect for bypassing CAPTCHA and anti-bot detection systems.
-
Fingerprint Avoidance: Eliminate the risk of device fingerprinting. Curious about how easy it is to fingerprint your device? 👉 Check it here
-
Direct Botting Mode with Magisk/KernelSU: With Magisk or KernelSU, you can start your device and go straight into botting mode—no manual intervention needed.
-
Easy Device-Wide Proxying: Effortlessly proxy all network traffic using tools like Proxifier or SocksDroid for enhanced privacy and IP rotation.
-
Low-Level Performance: Thanks to its C++ core, CyAndroEmu achieves unparalleled speed and efficiency, even in resource-intensive environments.
-
Multi-Backend Support: Choose from various parsers and input methods tailored to different Android environments (real devices, emulators, custom ROMs).
-
Seamless Emulator Integration: Designed to work flawlessly with popular emulators like BlissOS, BlueStacks, VMware, and more.
-
Root-Friendly: Full support for rooted devices, enabling deep system-level automation capabilities.
-
Magisk / KernelSU Plugins: Start your Python automation automatically when the emulator or device boots—no human interaction needed.
1. Download and install your favorite Android emulator, and ensure it has root access. CyAndroEmu should also work on any rooted smartphone (not tested yet). A non-root version may be available in the future. CyAndroEmu has been tested with the following:
- BlueStacks 5
- BlissOs 14, 15, 16 Go (on ProxMox, Virtual Box - Windows and VMWare - Windows)
- LDPlayer
- MeMuPlay
- MuMuPlayer
- GenyMotion
- Nox Player
- Android Studio - Magisk Patched PlayStore API 30 - Android 11
Although neither Magisk nor KernelSU is required, using them can significantly simplify tasks such as running Python scripts at Android startup and automatically adding the Termux folder to the PATH when accessing the ADB shell . They also assist in hiding root access and automating actions - writing your own Plugins is really simple. Here are some of my tutorials in Portuguese/English that teach you how to install them:
| How to install Python on an emulator | |
|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
2. Install Termux and TermuxBoot - optional
3. Use the following command to install the required packages (Python, Pandas, NumPy and some other stuff):
-
Open a ADB shell, and Termux on your emulator and copy and paste this command to the ADB shell to save some typing:
- necessary packages:
input text 'yes | pkg up;pkg install -y openssh;pkg install -y git;pkg install -y openssl;pkg install -y python;pkg install -y python-numpy;pkg install -y python-pip;pkg install -y clang;pkg install -y cmake;pkg install -y coreutils;pkg install -y termux-tools;pkg install -y root-repo;pkg install -y cpulimit;pkg install -y tur-repo;pkg install -y python-pandas;pkg install -y tesseract;pkg install -y imagemagick; pkg install -y imagemagick-static;yes | python -m pip install ptpython;yes | python -m pip install cyandroemu;yes | python -m pip install regex;yes | python -m pip install ptpython;yes | python -m pip install setuptools;yes | python -m pip install Cython;yes | python -m pip install requests';input keyevent KEYCODE_ENTER;input keyevent KEYCODE_ENTER
- recommended packages:
input text 'yes | pkg up;yes | pkg install -y openssh;yes | pkg install -y ripgrep;yes | pkg rei libexpat;pkg install -y git;pkg install -y openssl;pkg install -y python;pkg install -y cloneit;pkg install -y python-lxml;pkg install -y python-numpy;pkg install -y python-pillow;pkg install -y python-pip;pkg install -y python-scipy;pkg install -y vim-python;pkg install -y clang;pkg install -y cmake;pkg install -y coreutils;pkg install -y termux-exec;pkg install -y termux-tools;pkg install -y root-repo;pkg install -y matplotlib;pkg install -y cpulimit;pkg install -y tur-repo;pkg install -y python-pandas;pkg install -y tmux;pkg install -y opencv-python;pkg install -y boost;pkg install -y boost-headers;pkg install -y boost-static;pkg install -y tesseract; pkg install -y tor; pkg install -y torsocks; pkg install -y fzf; pkg install -y imagemagick; pkg install -y imagemagick-static;pkg install -y python-scikit-image; pkg install -y python-polars; pkg install -y nodejs-20; pkg install -y gcc-14; pkg install -y ffmpeg7;yes | python -m pip install ptpython;yes | python -m pip install cyandroemu;yes | python -m pip install regex;yes | python -m pip install ptpython;yes | python -m pip install setuptools;yes | python -m pip install Cython;yes | python -m pip install requests';input keyevent KEYCODE_ENTER;input keyevent KEYCODE_ENTER
4. Accessing Python from the ADB shell. Choose one of the 3 solutions:
-
Recommended: Install the Magisk/KernelSU plugin to automatically add the Termux bin folder to the PATH
-
Add
export PATH="$PATH:/data/data/com.termux/files/usr/bin"to/system/etc/mkshrc(don't forget to set the correct owner/group ... afterwards) -
Manually type
export PATH="$PATH:/data/data/com.termux/files/usr/bin"each time you enter the ADB su shell.
Cyandroemu will be compiled the first time you import it! Since this process is very CPU- and memory-intensive, I recommend setting the memory to 8 GB of RAM. Later on, you can reduce it. The SO files can be copied to other instances of your chosen emulator, but be careful: the CPU must be the same, as the C++ files are compiled with the flags -march=native and -mtune=native to ensure the highest possible performance."
Cyandroemu is primarily written in C++ (6 out of 7 parsers, modified sendevent, etc.) and Cython (1 parser and the Python interface), but all automation is handled using the Python class TermuxAutomation.
TermuxAutomation:
parsers: dict,
parser_timeout: int = 30,
add_input_tap: bool = True,
add_sendevent_mouse_click: bool = True,
add_sendevent_tap: bool = True,
add_mouse_action: bool = True,
mouse_device: str = "/dev/input/event4",
touch_device: str = "/dev/input/event5",
keyboard_device: str = "/dev/input/event1",
touch_device_max: int = 32767,
mouse_device_max: int = 65535,
input_cmd_tap: str = "/system/bin/input tap",
input_cmd_text: str = "/system/bin/input text",
sendevent_path: str = "/system/bin/sendevent",
screen_height: int = 768,
screen_width: int = 1024,
sh_exe: str = "/bin/sh",
su_exe: str = "su",
uiautomator_dump_path: str = "/sdcard/window_dump.xml",
uiautomator_cpu_limit: int = 5,
system_bin_path: str = "/system/bin/",
path_screencap: str = "screencap",
path_exe_tesseract: str = "/data/data/com.termux/files/usr/bin/tesseract",
path_exe_imagemagick: str = "/data/data/com.termux/files/usr/bin/magick",
tesseract_args: str = '"-l por+eng --oem 3"',
imagemagick_args: object = None,
tesseract_path_outpic: str = "/sdcard/screenshot.ppm",
uiautomator2_download_link1: str = "https://github.com/hansalemaos/uiautomator2tocsv/raw/refs/heads/main/app-uiautomator-test.apk",
uiautomator2_download_link2: str = "https://github.com/hansalemaos/uiautomator2tocsv/raw/refs/heads/main/app-uiautomator.apk",
uiautomator2_save_path1: str = "/sdcard/app-uiautomator-test.apk",
uiautomator2_save_path2: str = "/sdcard/app-uiautomator.apk",
lcp_deque_size: int = 40960,
lcp_print_stdout: bool = False,
lcp_print_stderr: bool = False,
kwargs: object = None,
adb_keyboard_link="https://github.com/hansalemaos/ADBKeyBoard/raw/refs/heads/master/ADBKeyboard.apk",
adb_keyboard_save_path="/sdcard/ADBKeyboard.apk",
valid_adb_keyboards=(
"com.android.adbkeyboard/.AdbIME",
"com.github.uiautomator/.AdbKeyboard",
),The heart of the library, containing all the C++ applications required to run the automation. To obtain the dictionary, use the following command. The first time you run it, it will take about 5 minutes as the C++ files will be downloaded and compiled using GCC.
parsers = TermuxAutomation.load_parsers_or_download_and_compile("g++")The timeout (in seconds) for subprocess.run.
Wether to add the option to input tap any elment.
Whether to add the option to perform mouse clicks on any element using sendevent.
Whether to add the option to tap any element using sendevent.
Whether to add the option to perform mouse actions on elements (e.g., move to, right-click, left-click, middle-click, etc.) using a modified version of sendevent.
The mouse/keyboard/touch devices and their corresponding device max can be identified using
getevent -lpor by using the static method find_suitable_devices_for_input_events:
df_devices = TermuxAutomation.find_suitable_devices_for_input_events() # returns a pandas DataFrameThese are the standard input commands. In most cases, there’s no need to modify them.
Screen height and screen width, can be identified using
wm sizeor by using the static method get_resolution_of_screen
screen_width, screen_height = TermuxAutomation.get_resolution_of_screen()Caution
MAKE SURE TO PROVIDE THE CORRECT VALUES HERE AND DO NOT ROTATE THE SCREEN DURING AUTOMATION! Many parsers rely on this information to function correctly.
The path to the shell executable, in most cases, there’s no need to modify it.
The command used to enter su (superuser) mode. In most cases, there’s no need to modify it.
Dump path for uiautomator. In most cases, there’s no need to modify it.
The CPU usage limit for the UIAutomator + cpulimit backend.
The path where system executables are stored. In most cases, there’s no need to modify it.
The command used for taking screenshots. In most cases, there’s no need to modify it.
The path to the Tesseract executable. In most cases, there’s no need to modify it.
The path to the ImageMagick executable. Used for tesseract preprocessing. In most cases, there’s no need to modify it.
Default Tesseract arguments. The default languages are Portuguese (por) and English (eng), with fully automatic page segmentation but no OSD (default).
Use tesseract --help-extra to view all available options. Words are grouped using pandas
Note: Be aware that changing the output format may cause the parser to crash, as it requires the output to be in hOCR format.
Pass None if you don’t want to preprocess the screenshot before passing it to Tesseract. If you choose to preprocess, you can provide ImageMagick command-line arguments to adjust the image (e.g., converting to grayscale, increasing contrast) to improve OCR accuracy.
The C++ Tesseract parser captures a screenshot using screencap, converts the image to a PPM format, and saves the converted output to disk before invoking Tesseract. In most cases, there’s no need to modify this path.
uiautomator2_download_link1: str = "https://github.com/hansalemaos/uiautomator2tocsv/raw/refs/heads/main/app-uiautomator-test.apk"
uiautomator2_download_link2: str = "https://github.com/hansalemaos/uiautomator2tocsv/raw/refs/heads/main/app-uiautomator.apk"
Backups APKs from the Uiautomator2 project.
You can also use my modified version which shows all hidden elements (good for web scraping)
https://github.com/hansalemaos/uiautomator2tocsv/raw/refs/heads/main/app-uiautomator-test_with_hidden_elements.apk
https://github.com/hansalemaos/uiautomator2tocsv/raw/refs/heads/main/app-uiautomator_with_hidden_elements.apk
Download path for the APKs from the Uiautomator2 project. In most cases, there’s no need to modify them.
The ring buffer size of the C++ vector used by the Logcat Parser backend. In most cases, there’s no need to modify it.
Debug print options for the Logcat Parser backend. These should only be enabled if you encounter issues with the Logcat Parser backend.
kwargs for subprocess.run In most cases, there’s no need to modify it.
adb_keyboard_link: str = "https://github.com/hansalemaos/ADBKeyBoard/raw/refs/heads/master/ADBKeyboard.apk"
Backup APK from the ADBKeyboard project. It allows sending Unicode characters but may not work on all devices. In most cases, there’s no need to modify it.
The download path for the APK from the ADBKeyboard project project. In most cases, there’s no need to modify it.
valid_adb_keyboards: tuple = ("com.android.adbkeyboard/.AdbIME", "com.github.uiautomator/.AdbKeyboard",)
Possible package names for ADBKeyboard.
- The first name is the original from the ADBKeyboard project project.
- The second name is from the Uiautomator2 project project.
In most cases, there’s no need to modify them.
from cyandroemu import TermuxAutomation, clean_zombies
import pandas as pd
from time import sleep as timesleep
import atexit
# Ensure background parsers and shells are terminated on exit
atexit.register(clean_zombies)
# Load parsers (first time may take ~5 mins due to download and compilation)
parsers = TermuxAutomation.load_parsers_or_download_and_compile("g++")
# Detect input devices dynamically (returns a pandas DataFrame)
df_devices = TermuxAutomation.find_suitable_devices_for_input_events()
print(df_devices)
screen_width, screen_height = TermuxAutomation.get_resolution_of_screen()
# On VMware and BlissOS, device order may change after a reboot, so hardcoding is avoided.
# If device order doesn't change on your emulator, you can hardcode the path, e.g.:
# mouse_device = "/dev/input/event4"
te = TermuxAutomation(
parsers=parsers,
mouse_device=df_devices.loc[
(df_devices["max"].str.len() == 2) & (df_devices["type"] == "mouse")
]["path"].iloc[0],
keyboard_device=df_devices.loc[
(df_devices["keys_found"]) & (df_devices["type"] == "keyboard")
]["path"].iloc[0],
mouse_device_max=df_devices.loc[
(df_devices["max"].str.len() == 2) & (df_devices["type"] == "mouse")
]["max"].iloc[0][0],
screen_height=screen_height,
screen_width=screen_width,
)
# Clean up any leftover background processes (e.g., uiautomator2 server) from earlier automations.
# atexit should handle this, but "Doppelt genaeht haelt besser :)" (German proverb - double-stitched holds better).
te.kill_zombies()Most automation tools implement their own logic for finding elements, which often looks like this:
self.find(text="Some text").children[0].click()
That looks beautiful and very Pythonic, but most of the time, it’s just unreliable — mainly when the attributes and positions of objects vary, or when the objects aren’t loaded on the screen yet.
Instead of bothering you with my own "great" ideas, I decided to rely on battle-tested solutions: pandas DataFrames.
With DataFrames, you can leverage powerful features like:
- Fast sorting algorithms
- All kinds of comparisons
- Regular expressions (regex)
- Custom filter functions
If you know how to work with pandas DataFrames, you basically already know everything you need to automate whatever you want. If not, you can learn everything from the official documentation: https://pandas.pydata.org/docs/ That’s where I got 95% of my pandas knowledge from.
The default __repr__ and __str__ functions in pandas are not ideal for displaying DataFrames, which is why they’ve been monkey-patched to ensure the entire DataFrame is pretty-printed.
df = te.get_df_uiautomator2(with_screenshot=True)
print(df)Now, when you print the DataFrame, you’ll see the complete content neatly formatted, making it easier to analyze UI elements. For a line-break-free experience, I recommend using the VTM terminal
In the example above, the elements are very easy to locate. However, this is not always the case. You might want to see a screenshot of each element to write your script faster. In this case, you can use the following command on the device:
df.bb_save_screenshots_as_png("/sdcard/uiautomator2screenshots")Then, pull the screenshots using ADB to review them on your computer:
adb -s 192.168.115.131:5555 pull /sdcard/uiautomator2screenshotsThe screenshot filenames correspond to the location of the elements in the DataFrame. For example, if you want to locate the third element (index 2), simply run:
df.iloc[2]This will extract the element’s information, allowing you to build your pandas query from there.
| Advantages ✅ | Disadvantages ❌ |
|---|---|
| Very fast parsing | Blocks other parsers like the Logcat parser and traditional UIAutomator |
| Never fails due to excessive screen movement (unlike UIAutomator) | Requires APKs to be installed |
| Includes both element text and description | The server needs to run in the background |
| Great for web scraping and interacting with web elements | Automation might be detected when used with installed apps |
| Runs on any emulator | Useless with specific engines like the Unreal Engine |
from cyandroemu import TermuxAutomation, clean_zombies
import pandas as pd
from time import sleep as timesleep
import atexit
# Ensure all background processes are terminated on exit
atexit.register(clean_zombies)
# Load parsers (first time may take ~5 mins due to download/compilation)
parsers = TermuxAutomation.load_parsers_or_download_and_compile("g++")
# Detect input devices dynamically (Necessary on BlissOS 16 Go with VMWare)
df_devices = TermuxAutomation.find_suitable_devices_for_input_events()
screen_width, screen_height = TermuxAutomation.get_resolution_of_screen()
# Initialize TermuxAutomation with dynamic device paths
te = TermuxAutomation(
parsers=parsers,
mouse_device=df_devices.loc[
(df_devices["max"].str.len() == 2) & (df_devices["type"] == "mouse")
]["path"].iloc[0],
keyboard_device=df_devices.loc[
(df_devices["keys_found"]) & (df_devices["type"] == "keyboard")
]["path"].iloc[0],
mouse_device_max=df_devices.loc[
(df_devices["max"].str.len() == 2) & (df_devices["type"] == "mouse")
]["max"].iloc[0][0],
screen_height=screen_height,
screen_width=screen_width,
)
# Kill any leftover background processes
te.kill_zombies()
# You can install the APK using:
# te.download_and_install_uiautomator2_apks()
# Ensure elements are loaded on the screen instead of relying solely on sleep
df = pd.DataFrame()
while df.empty:
# First DataFrame load may take ~5 seconds (UIAutomator2 server startup)
# For production scripts, set with_screenshot=False for better performance
# You can also pass these kwargs to temporarily change the default paramenters passed to the class constructor (valid for all backends, but not recommended)
# If you pass anything else than None, it will override the default parameters once.
# add_input_tap=None,
# add_sendevent_mouse_click=None,
# add_sendevent_tap=None,
# add_mouse_action=None,
# x_column=None,
# y_column=None,
# mouse_device=None,
# touch_device=None,
# touch_device_max=None,
# mouse_device_max=None,
# input_cmd=None,
# sendevent_path=None,
# screen_height=None,
# screen_width=None,
# mouse_action_exe=None,
# sh_exe=None,
# su_exe=None,
# kwargs=None,
# timeout=None,
df = te.get_df_uiautomator2(with_screenshot=True)
# Filter elements to target the TikTok app
df = df.loc[
(df.aa_text == "TikTok")
& (df.aa_content_desc == "TikTok")
& (df.aa_clickable == 1)
]
# Click the first matching element using input tap
# Optional: Add offset like df.aa_input_tap.iloc[0](20, 40)
df.aa_input_tap.iloc[0]()
# Clear DataFrame for the next loop
df = pd.DataFrame()
# optional sleep
timesleep(2)
# Repeating the same process for other UI elements
while df.empty:
df = te.get_df_uiautomator2(with_screenshot=True)
df = df.loc[
(df.aa_text == "Already have an account? Log in")
& (df.aa_clazz == "android.widget.Button")
& (df.aa_content_desc == "")
]
df.aa_input_tap.iloc[0]()
df = pd.DataFrame()
timesleep(2)
while df.empty:
df = te.get_df_uiautomator2(with_screenshot=True)
df = df.loc[
(df.aa_text == "Use phone / email / username")
& (df.aa_clazz == "android.widget.TextView")
& (df.aa_content_desc == "")
]
df.aa_input_tap.iloc[0]()
df = pd.DataFrame()
timesleep(2)
while df.empty:
df = te.get_df_uiautomator2(with_screenshot=True)
df = df.loc[
(df.aa_text == "Email / Username")
& (df.aa_clazz == "android.widget.TextView")
& (df.aa_content_desc == "")
]
df.aa_input_tap.iloc[0]()
df = pd.DataFrame()
timesleep(2)
# Send text input naturally using a lightweight Cython class
# In this example, __call__ is used directly to enter the email once
te.get_cmd_send_text_natural("myemailtest@gmail.com")()| index | aa_index | aa_indent | aa_text | aa_resource_id | aa_clazz | aa_package | aa_content_desc | aa_checkable | aa_checked | aa_clickable | aa_enabled | aa_focusable | aa_focused | aa_scrollable | aa_long_clickable | aa_password | aa_selected | aa_visible_to_user | aa_bounds | aa_drawing_order | aa_hint | aa_display_id | aa_line_index | aa_children | aa_parents | aa_start_x | aa_start_y | aa_end_x | aa_end_y | aa_center_x | aa_center_y | aa_width | aa_height | aa_area | aa_w_h_relation | aa_input_tap | aa_sendevent_mouse_click | aa_sendevent_tap | aa_mouse_action | aa_screenshot |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0 | 4 | com.android.systemui:id/status | android.widget.FrameLayout | com.android.systemui | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | [0,0][1024,24] | 1 | 0 | 1 | () | () | 0 | 0 | 1024 | 24 | 512 | 12 | 1024 | 24 | 24576 | 42.666668 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 1 | 0 | 6 | 8:18 | com.android.systemui:id/clock | android.widget.TextView | com.android.systemui | 8:18 PM | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | [16,0][45,24] | 2 | 0 | 2 | () | () | 16 | 0 | 45 | 24 | 30 | 12 | 29 | 24 | 696 | 1.208333 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 2 | 1 | 6 | android.widget.ImageView | com.android.systemui | Termux notification: 1 session | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | [45,0][69,24] | 4 | 0 | 3 | () | () | 45 | 0 | 69 | 24 | 57 | 12 | 24 | 24 | 576 | 1.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 3 | 2 | 6 | android.widget.ImageView | com.android.systemui | Settings notification: NECVMWa | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | [69,0][93,24] | 5 | 0 | 4 | () | () | 69 | 0 | 93 | 24 | 81 | 12 | 24 | 24 | 576 | 1.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 4 | 3 | 6 | android.widget.ImageView | com.android.systemui | Ethernet. | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | [959,1][978,23] | 12 | 0 | 5 | () | () | 959 | 1 | 978 | 23 | 968 | 12 | 19 | 22 | 418 | 0.863636 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 5 | 4 | 6 | com.android.systemui:id/batter | android.widget.LinearLayout | com.android.systemui | Battery percentage unknown. | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | [984,0][996,24] | 16 | 0 | 6 | () | () | 984 | 0 | 996 | 24 | 990 | 12 | 12 | 24 | 288 | 0.5 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 6 | 0 | 2 | android.widget.FrameLayout | com.zhiliaoapp.musically | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | [0,0][1024,768] | 0 | 0 | 7 | () | () | 0 | 0 | 1024 | 768 | 512 | 384 | 1024 | 768 | 786432 | 1.333333 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||
| 7 | 0 | 4 | com.zhiliaoapp.musically:id/d4 | android.view.ViewGroup | com.zhiliaoapp.musically | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | [0,24][1024,736] | 1 | 0 | 8 | () | () | 0 | 24 | 1024 | 736 | 512 | 380 | 1024 | 712 | 729088 | 1.438202 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 8 | 0 | 6 | com.zhiliaoapp.musically:id/r7 | android.widget.ImageView | com.zhiliaoapp.musically | Report a problem | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | [4,26][52,74] | 3 | 0 | 9 | () | () | 4 | 26 | 52 | 74 | 28 | 50 | 48 | 48 | 2304 | 1.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 9 | 2 | 6 | Skip | com.zhiliaoapp.musically:id/p6 | android.widget.Button | com.zhiliaoapp.musically | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | [980,38][1008,56] | 5 | 0 | 10 | () | () | 980 | 38 | 1008 | 56 | 994 | 47 | 28 | 18 | 504 | 1.555556 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 10 | 3 | 6 | com.zhiliaoapp.musically:id/tt | androidx.viewpager.widget.View | com.zhiliaoapp.musically | 0 | 0 | 0 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 1 | [0,77][1024,736] | 2 | 0 | 11 | () | () | 0 | 77 | 1024 | 736 | 512 | 406 | 1024 | 659 | 674816 | 1.553869 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 11 | 0 | 8 | com.zhiliaoapp.musically:id/c7 | androidx.recyclerview.widget.R | com.zhiliaoapp.musically | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | [267,77][758,396] | 1 | 0 | 12 | () | () | 267 | 77 | 758 | 396 | 512 | 236 | 491 | 319 | 156629 | 1.539185 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 12 | 0 | 10 | android.widget.LinearLayout | com.zhiliaoapp.musically | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | [267,82][758,228] | 1 | 0 | 13 | () | () | 267 | 82 | 758 | 228 | 512 | 155 | 491 | 146 | 71686 | 3.363014 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||
| 13 | 0 | 12 | Sign up for TikTok | com.zhiliaoapp.musically:id/ti | android.widget.TextView | com.zhiliaoapp.musically | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | [267,122][758,150] | 1 | 0 | 14 | () | () | 267 | 122 | 758 | 150 | 512 | 136 | 491 | 28 | 13748 | 17.535715 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 14 | 1 | 12 | Create a profile, follow other | com.zhiliaoapp.musically:id/oe | android.widget.TextView | com.zhiliaoapp.musically | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | [267,162][758,200] | 2 | 0 | 15 | () | () | 267 | 162 | 758 | 200 | 512 | 181 | 491 | 38 | 18658 | 12.921053 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 15 | 1 | 10 | com.zhiliaoapp.musically:id/cz | android.widget.Button | com.zhiliaoapp.musically | Use phone or email | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | [267,228][758,272] | 2 | 0 | 16 | () | () | 267 | 228 | 758 | 272 | 512 | 250 | 491 | 44 | 21604 | 11.159091 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 16 | 0 | 12 | Use phone or email | com.zhiliaoapp.musically:id/c7 | android.widget.TextView | com.zhiliaoapp.musically | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | [454,241][593,260] | 1 | 0 | 17 | () | () | 454 | 241 | 593 | 260 | 523 | 250 | 139 | 19 | 2641 | 7.31579 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 17 | 2 | 10 | com.zhiliaoapp.musically:id/cz | android.widget.Button | com.zhiliaoapp.musically | Continue with Facebook | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | [267,284][758,328] | 3 | 0 | 18 | () | () | 267 | 284 | 758 | 328 | 512 | 306 | 491 | 44 | 21604 | 11.159091 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 18 | 0 | 12 | Continue with Facebook | com.zhiliaoapp.musically:id/c7 | android.widget.TextView | com.zhiliaoapp.musically | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | [437,297][611,316] | 1 | 0 | 19 | () | () | 437 | 297 | 611 | 316 | 524 | 306 | 174 | 19 | 3306 | 9.157895 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 19 | 3 | 10 | com.zhiliaoapp.musically:id/cz | android.widget.Button | com.zhiliaoapp.musically | Continue with Google | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | [267,340][758,384] | 4 | 0 | 20 | () | () | 267 | 340 | 758 | 384 | 512 | 362 | 491 | 44 | 21604 | 11.159091 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 20 | 0 | 12 | Continue with Google | com.zhiliaoapp.musically:id/c7 | android.widget.TextView | com.zhiliaoapp.musically | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | [445,353][602,371] | 1 | 0 | 21 | () | () | 445 | 353 | 602 | 371 | 523 | 362 | 157 | 18 | 2826 | 8.722222 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 21 | 1 | 8 | By continuing with an account | com.zhiliaoapp.musically:id/rw | android.widget.TextView | com.zhiliaoapp.musically | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | [32,628][992,675] | 3 | 0 | 22 | () | () | 32 | 628 | 992 | 675 | 512 | 651 | 960 | 47 | 45120 | 20.425531 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 22 | 2 | 8 | Already have an account? Log i | com.zhiliaoapp.musically:id/s5 | android.widget.Button | com.zhiliaoapp.musically | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | [0,676][1024,736] | 7 | 0 | 23 | () | () | 0 | 676 | 1024 | 736 | 512 | 706 | 1024 | 60 | 61440 | 17.066668 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[248 248 248] [248 248 248 |
| Advantages ✅ | Disadvantages ❌ |
|---|---|
| Very fast parsing with extremely low CPU usage | Requires interaction with elements or changes in element status |
| Detects elements that are not even visible on the screen | Requires APKs to be installed |
| Includes both element text and description | Needs to run as an Accessibility Service |
| Great for web scraping and interacting with web elements | Automation might be detected when used with installed apps |
| Doesn’t block the classic UIAutomator | Ineffective with specific engines like Unreal Engine |
| Automatically captures any element changes—perfect for crypto, sports betting | Best results achieved on emulators with a mouse device (e.g., BlissOS, BlueStacks) |
# Imports and instance creation omitted for brevity
df = pd.DataFrame()
# Loop to find the TikTok element and click it
while df.empty:
# Moves the mouse horizontally and vertically to provoke interaction with all elements on the screen
# Works on BlueStacks 5 and BlissOS with VMware, and probably some other emulators
df = te.get_df_lcp_with_mouse_movement()
df = df.loc[
(df.aa_Text == "TikTok")
& (df.aa_ContentDescription == "TikTok")
& (df.aa_ClassName == "android.widget.TextView")
]
# Clear the DataFrame for the next interaction
df.aa_input_tap.iloc[0]()
timesleep(2)
df = pd.DataFrame()
while df.empty:
df = te.get_df_lcp_with_mouse_movement()
df = df.loc[
(df.aa_Text == "Already have an account? Log in")
& (df.aa_ContentDescription == "null")
& (df.aa_ClassName == "android.widget.Button")
]
df.aa_input_tap.iloc[0]()
timesleep(2)
df = pd.DataFrame()
while df.empty:
df = te.get_df_lcp_with_mouse_movement()
df = df.loc[
(df.aa_Text == "null")
& (df.aa_ContentDescription == "Use phone / email / username")
& (df.aa_ClassName == "android.widget.Button")
]
df.aa_input_tap.iloc[0]()
timesleep(2)
df = pd.DataFrame()
while df.empty:
df = te.get_df_lcp_with_mouse_movement()
df = df.loc[
(df.aa_Text == "null")
& (df.aa_ContentDescription == "Email / Username")
& (df.aa_ClassName == "android.widget.LinearLayout")
]
df.aa_input_tap.iloc[0]()
timesleep(2)
te.get_cmd_send_text_natural("myemailtest@gmail.com")()| index | aa_Text | aa_ContentDescription | aa_StateDescription | aa_ClassName | aa_PackageName | aa_Error | aa_AccessNodeInfo | aa_WindowId | aa_WindowChanges | aa_WindowChangeTypes | aa_VirtualDescendantId | aa_ViewIdResName | aa_UniqueId | aa_TraversalBefore | aa_TraversalAfter | aa_TooltipText | aa_TimeStamp | aa_TimeNow | aa_SpeechStateChangeTypes | aa_SourceWindowId | aa_SourceNodeId | aa_SourceDisplayId | aa_Source | aa_Sealed | aa_Records | aa_ParentNodeId | aa_ParcelableData | aa_MovementGranularities | aa_HashCode | aa_EventType | aa_Actions | aa_ContentChangeTypes | aa_ConnectionId | aa_ChildAccessibilityIds | aa_BooleanProperties | aa_BeforeText | aa_Active | aa_AccessibilityViewId | aa_AccessibilityTool | aa_BoundsInScreen | aa_BoundsInParent | aa_UnixTimeText | aa_start_x_real | aa_start_y_real | aa_end_x_real | aa_end_y_real | aa_start_x | aa_start_y | aa_end_x | aa_end_y | aa_center_x | aa_center_y | aa_width | aa_height | aa_w_h_relation | aa_area | aa_parent_start_x_real | aa_parent_start_y_real | aa_parent_end_x_real | aa_parent_end_y_real | aa_parent_start_x | aa_parent_start_y | aa_parent_end_x | aa_parent_end_y | aa_parent_center_x | aa_parent_center_y | aa_parent_width | aa_parent_height | aa_parent_w_h_relation | aa_parent_area | aa_UnixTime | aa_distance_from_start | aa_size | aa_Visible | aa_Password | aa_Selected | aa_Scrollable | aa_LongClickable | aa_Loggable | aa_IsTextSelectable | **aa_ImportantForAccessibility | aa_Enabled | aa_Empty | aa_ContextClickable | aa_ContentInvalid | aa_FullScreen | aa_Focused | aa_Focusable | aa_AccessibilityFocused | **aa_AccessibilityDataSensitiv | aa_Clickable | aa_Checked | aa_Checkable | aa_input_tap | aa_sendevent_mouse_click | aa_sendevent_tap | aa_mouse_action | aa_screenshot |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | null | null | null | android.widget.LinearLayout | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_S | Rect(267, 82 - 758, 228) | Rect(0, 0 - 491, 146) | 02-06 20:16:04.651 | 267 | 82 | 758 | 228 | 267 | 82 | 758 | 228 | 512 | 155 | 491 | 146 | 3.363013698630137 | 71686 | 0 | 0 | 491 | 146 | 0 | 0 | 491 | 146 | 245 | 73.0 | 491.0 | 146.0 | 3.363013698630137 | 71686.0 | 1738883764651.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||||||
| 1 | Sign up for TikTok | null | null | android.widget.TextView | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_S | Rect(267, 122 - 758, 150) | Rect(0, 0 - 491, 28) | 02-06 20:16:04.651 | 267 | 122 | 758 | 150 | 267 | 122 | 758 | 150 | 512 | 136 | 491 | 28 | 17.535714285714285 | 13748 | 0 | 0 | 491 | 28 | 0 | 0 | 491 | 28 | 245 | 14.0 | 491.0 | 28.0 | 17.535714285714285 | 13748.0 | 1738883764651.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||||||
| 2 | null | null | null | androidx.recyclerview.widget.R | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_F | Rect(267, 77 - 758, 396) | Rect(0, 0 - 491, 319) | 02-06 20:16:04.651 | 267 | 77 | 758 | 396 | 267 | 77 | 758 | 396 | 512 | 236 | 491 | 319 | 1.5391849529780564 | 156629 | 0 | 0 | 491 | 319 | 0 | 0 | 491 | 319 | 245 | 159.0 | 491.0 | 319.0 | 1.5391849529780564 | 156629.0 | 1738883764651.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||||||
| 3 | Sign up for TikTok | null | null | android.widget.TextView | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_S | Rect(52, 24 - 201, 76) | Rect(0, 0 - 149, 52) | 02-06 20:16:04.651 | 52 | 24 | 201 | 76 | 52 | 24 | 201 | 76 | 126 | 50 | 149 | 52 | 2.8653846153846154 | 7748 | 0 | 0 | 149 | 52 | 0 | 0 | 149 | 52 | 74 | 26.0 | 149.0 | 52.0 | 2.8653846153846154 | 7748.0 | 1738883764651.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||||||
| 4 | Skip | null | null | android.widget.Button | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_F | Rect(980, 38 - 1008, 56) | Rect(0, 0 - 28, 18) | 02-06 20:16:04.651 | 980 | 38 | 1008 | 56 | 980 | 38 | 1008 | 56 | 994 | 47 | 28 | 18 | 1.5555555555555556 | 504 | 0 | 0 | 28 | 18 | 0 | 0 | 28 | 18 | 14 | 9.0 | 28.0 | 18.0 | 1.5555555555555556 | 504.0 | 1738883764651.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||||||
| 5 | null | Report a problem | null | android.widget.ImageView | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_F | Rect(4, 26 - 52, 74) | Rect(0, 0 - 48, 48) | 02-06 20:16:04.651 | 4 | 26 | 52 | 74 | 4 | 26 | 52 | 74 | 28 | 50 | 48 | 48 | 1.0 | 2304 | 0 | 0 | 48 | 48 | 0 | 0 | 48 | 48 | 24 | 24.0 | 48.0 | 48.0 | 1.0 | 2304.0 | 1738883764651.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||||||
| 6 | null | null | null | androidx.viewpager.widget.View | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_F | Rect(0, 77 - 1024, 736) | Rect(0, 0 - 1024, 659) | 02-06 20:16:04.651 | 0 | 77 | 1024 | 736 | 0 | 77 | 1024 | 736 | 512 | 406 | 1024 | 659 | 1.5538694992412747 | 674816 | 0 | 0 | 1024 | 659 | 0 | 0 | 1024 | 659 | 512 | 329.0 | 1024.0 | 659.0 | 1.5538694992412747 | 674816.0 | 1738883764651.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||||||
| 7 | null | null | null | android.view.ViewGroup | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_S | Rect(0, 24 - 1024, 736) | Rect(0, 0 - 1024, 712) | 02-06 20:16:04.623 | 0 | 24 | 1024 | 736 | 0 | 24 | 1024 | 736 | 512 | 380 | 1024 | 712 | 1.4382022471910112 | 729088 | 0 | 0 | 1024 | 712 | 0 | 0 | 1024 | 712 | 512 | 356.0 | 1024.0 | 712.0 | 1.4382022471910112 | 729088.0 | 1738883764623.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||||||
| 8 | Continue with Google | null | null | android.widget.TextView | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_S | Rect(1469, 352 - 1024, 370) | Rect(0, 0 - 157, 18) | 02-06 20:16:04.622 | 1469 | 352 | 1024 | 370 | 1469 | 352 | 1024 | 370 | 1246 | 361 | -445 | 18 | -24.72222222222222 | -8010 | 0 | 0 | 157 | 18 | 0 | 0 | 157 | 18 | 78 | 9.0 | 157.0 | 18.0 | 8.722222222222221 | 2826.0 | 1738883764622.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [] | ||||||||||||||||||||||||||||
| 9 | By continuing with an account | null | null | android.widget.TextView | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_F | Rect(32, 628 - 992, 675) | Rect(0, 0 - 960, 47) | 02-06 20:16:04.621 | 32 | 628 | 992 | 675 | 32 | 628 | 992 | 675 | 512 | 651 | 960 | 47 | 20.425531914893615 | 45120 | 0 | 0 | 960 | 47 | 0 | 0 | 960 | 47 | 480 | 23.0 | 960.0 | 47.0 | 20.425531914893615 | 45120.0 | 1738883764621.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||||||
| 10 | Continue with Google | null | null | android.widget.TextView | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_S | Rect(445, 353 - 602, 371) | Rect(0, 0 - 157, 18) | 02-06 20:16:04.621 | 445 | 353 | 602 | 371 | 445 | 353 | 602 | 371 | 523 | 362 | 157 | 18 | 8.722222222222221 | 2826 | 0 | 0 | 157 | 18 | 0 | 0 | 157 | 18 | 78 | 9.0 | 157.0 | 18.0 | 8.722222222222221 | 2826.0 | 1738883764621.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||||||
| 11 | Manage your account, check not | null | null | android.widget.TextView | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_S | Rect(1291, 162 - 1024, 199) | Rect(0, 0 - 491, 37) | 02-06 20:16:04.621 | 1291 | 162 | 1024 | 199 | 1291 | 162 | 1024 | 199 | 1157 | 180 | -267 | 37 | -7.216216216216216 | -9879 | 0 | 0 | 491 | 37 | 0 | 0 | 491 | 37 | 245 | 18.0 | 491.0 | 37.0 | 13.27027027027027 | 18167.0 | 1738883764621.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [] | ||||||||||||||||||||||||||||
| 12 | Log in to TikTok | null | null | android.widget.TextView | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_S | Rect(1291, 122 - 1024, 150) | Rect(0, 0 - 491, 28) | 02-06 20:16:04.621 | 1291 | 122 | 1024 | 150 | 1291 | 122 | 1024 | 150 | 1157 | 136 | -267 | 28 | -9.535714285714286 | -7476 | 0 | 0 | 491 | 28 | 0 | 0 | 491 | 28 | 245 | 14.0 | 491.0 | 28.0 | 17.535714285714285 | 13748.0 | 1738883764621.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [] | ||||||||||||||||||||||||||||
| 13 | null | null | null | android.widget.LinearLayout | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_S | Rect(1291, 82 - 1024, 227) | Rect(0, 0 - 491, 145) | 02-06 20:16:04.621 | 1291 | 82 | 1024 | 227 | 1291 | 82 | 1024 | 227 | 1157 | 154 | -267 | 145 | -1.8413793103448275 | -38715 | 0 | 0 | 491 | 145 | 0 | 0 | 491 | 145 | 245 | 72.0 | 491.0 | 145.0 | 3.386206896551724 | 71195.0 | 1738883764621.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [] | ||||||||||||||||||||||||||||
| 14 | null | null | null | androidx.recyclerview.widget.R | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_F | Rect(1291, 77 - 1024, 451) | Rect(0, 0 - 491, 374) | 02-06 20:16:04.621 | 1291 | 77 | 1024 | 451 | 1291 | 77 | 1024 | 451 | 1157 | 264 | -267 | 374 | -0.713903743315508 | -99858 | 0 | 0 | 491 | 374 | 0 | 0 | 491 | 374 | 245 | 187.0 | 491.0 | 374.0 | 1.3128342245989304 | 183634.0 | 1738883764621.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [] | ||||||||||||||||||||||||||||
| 15 | Already have an account? Log i | null | null | android.widget.Button | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_F | Rect(0, 676 - 1024, 736) | Rect(0, 0 - 1024, 60) | 02-06 20:16:04.621 | 0 | 676 | 1024 | 736 | 0 | 676 | 1024 | 736 | 512 | 706 | 1024 | 60 | 17.066666666666666 | 61440 | 0 | 0 | 1024 | 60 | 0 | 0 | 1024 | 60 | 512 | 30.0 | 1024.0 | 60.0 | 17.066666666666666 | 61440.0 | 1738883764621.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[248 248 248] [248 248 248 | ||||||||||||||||||||||||||||
| 16 | Create a profile, follow other | null | null | android.widget.TextView | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_S | Rect(267, 162 - 758, 200) | Rect(0, 0 - 491, 38) | 02-06 20:16:04.621 | 267 | 162 | 758 | 200 | 267 | 162 | 758 | 200 | 512 | 181 | 491 | 38 | 12.921052631578949 | 18658 | 0 | 0 | 491 | 38 | 0 | 0 | 491 | 38 | 245 | 19.0 | 491.0 | 38.0 | 12.921052631578949 | 18658.0 | 1738883764621.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||||||
| 17 | null | Use phone or email | null | android.widget.Button | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_F | Rect(267, 228 - 758, 272) | Rect(0, 0 - 491, 44) | 02-06 20:16:04.621 | 267 | 228 | 758 | 272 | 267 | 228 | 758 | 272 | 512 | 250 | 491 | 44 | 11.159090909090908 | 21604 | 0 | 0 | 491 | 44 | 0 | 0 | 491 | 44 | 245 | 22.0 | 491.0 | 44.0 | 11.159090909090908 | 21604.0 | 1738883764621.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||||||
| 18 | Use phone or email | null | null | android.widget.TextView | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_S | Rect(454, 241 - 593, 260) | Rect(0, 0 - 139, 19) | 02-06 20:16:04.621 | 454 | 241 | 593 | 260 | 454 | 241 | 593 | 260 | 523 | 250 | 139 | 19 | 7.315789473684211 | 2641 | 0 | 0 | 139 | 19 | 0 | 0 | 139 | 19 | 69 | 9.0 | 139.0 | 19.0 | 7.315789473684211 | 2641.0 | 1738883764621.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||||||
| 19 | null | Continue with Facebook | null | android.widget.Button | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_F | Rect(267, 284 - 758, 328) | Rect(0, 0 - 491, 44) | 02-06 20:16:04.621 | 267 | 284 | 758 | 328 | 267 | 284 | 758 | 328 | 512 | 306 | 491 | 44 | 11.159090909090908 | 21604 | 0 | 0 | 491 | 44 | 0 | 0 | 491 | 44 | 245 | 22.0 | 491.0 | 44.0 | 11.159090909090908 | 21604.0 | 1738883764621.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||||||
| 20 | Continue with Facebook | null | null | android.widget.TextView | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_S | Rect(437, 297 - 611, 316) | Rect(0, 0 - 174, 19) | 02-06 20:16:04.621 | 437 | 297 | 611 | 316 | 437 | 297 | 611 | 316 | 524 | 306 | 174 | 19 | 9.157894736842104 | 3306 | 0 | 0 | 174 | 19 | 0 | 0 | 174 | 19 | 87 | 9.0 | 174.0 | 19.0 | 9.157894736842104 | 3306.0 | 1738883764621.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||||||
| 21 | null | Continue with Google | null | android.widget.Button | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_F | Rect(267, 340 - 758, 384) | Rect(0, 0 - 491, 44) | 02-06 20:16:04.621 | 267 | 340 | 758 | 384 | 267 | 340 | 758 | 384 | 512 | 362 | 491 | 44 | 11.159090909090908 | 21604 | 0 | 0 | 491 | 44 | 0 | 0 | 491 | 44 | 245 | 22.0 | 491.0 | 44.0 | 11.159090909090908 | 21604.0 | 1738883764621.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||||||
| 22 | null | Continue with Instagram | null | android.widget.Button | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_F | Rect(1291, 395 - 1024, 439) | Rect(0, 0 - 491, 44) | 02-06 20:16:04.620 | 1291 | 395 | 1024 | 439 | 1291 | 395 | 1024 | 439 | 1157 | 417 | -267 | 44 | -6.068181818181818 | -11748 | 0 | 0 | 491 | 44 | 0 | 0 | 491 | 44 | 245 | 22.0 | 491.0 | 44.0 | 11.159090909090908 | 21604.0 | 1738883764620.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [] | ||||||||||||||||||||||||||||
| 23 | Use phone / email / username | null | null | android.widget.TextView | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_S | Rect(1440, 240 - 1024, 259) | Rect(0, 0 - 216, 19) | 02-06 20:16:04.620 | 1440 | 240 | 1024 | 259 | 1440 | 240 | 1024 | 259 | 1232 | 249 | -416 | 19 | -21.894736842105264 | -7904 | 0 | 0 | 216 | 19 | 0 | 0 | 216 | 19 | 108 | 9.0 | 216.0 | 19.0 | 11.36842105263158 | 4104.0 | 1738883764620.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [] | ||||||||||||||||||||||||||||
| 24 | Continue with Instagram | null | null | android.widget.TextView | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_S | Rect(1459, 408 - 1024, 426) | Rect(0, 0 - 177, 18) | 02-06 20:16:04.620 | 1459 | 408 | 1024 | 426 | 1459 | 408 | 1024 | 426 | 1241 | 417 | -435 | 18 | -24.166666666666668 | -7830 | 0 | 0 | 177 | 18 | 0 | 0 | 177 | 18 | 88 | 9.0 | 177.0 | 18.0 | 9.833333333333334 | 3186.0 | 1738883764620.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [] | ||||||||||||||||||||||||||||
| 25 | null | null | null | android.widget.FrameLayout | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_S | Rect(0, 0 - 1024, 768) | Rect(0, 0 - 1024, 768) | 02-06 20:16:04.620 | 0 | 0 | 1024 | 768 | 0 | 0 | 1024 | 768 | 512 | 384 | 1024 | 768 | 1.3333333333333333 | 786432 | 0 | 0 | 1024 | 768 | 0 | 0 | 1024 | 768 | 512 | 384.0 | 1024.0 | 768.0 | 1.3333333333333333 | 786432.0 | 1738883764620.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||||||
| 26 | Don’t have an account? Sign up | null | null | android.widget.Button | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_F | Rect(1024, 676 - 1024, 736) | Rect(0, 0 - 1024, 60) | 02-06 20:16:04.620 | 1024 | 676 | 1024 | 736 | 1024 | 676 | 1024 | 736 | 1024 | 706 | 0 | 60 | 0.0 | 0 | 0 | 0 | 1024 | 60 | 0 | 0 | 1024 | 60 | 512 | 30.0 | 1024.0 | 60.0 | 17.066666666666666 | 61440.0 | 1738883764620.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [] | ||||||||||||||||||||||||||||
| 27 | By continuing with an account | null | null | android.widget.TextView | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_F | Rect(1056, 628 - 1024, 675) | Rect(0, 0 - 960, 47) | 02-06 20:16:04.620 | 1056 | 628 | 1024 | 675 | 1056 | 628 | 1024 | 675 | 1040 | 651 | -32 | 47 | -0.6808510638297872 | -1504 | 0 | 0 | 960 | 47 | 0 | 0 | 960 | 47 | 480 | 23.0 | 960.0 | 47.0 | 20.425531914893615 | 45120.0 | 1738883764620.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [] | ||||||||||||||||||||||||||||
| 28 | null | Continue with Google | null | android.widget.Button | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_F | Rect(1291, 339 - 1024, 383) | Rect(0, 0 - 491, 44) | 02-06 20:16:04.620 | 1291 | 339 | 1024 | 383 | 1291 | 339 | 1024 | 383 | 1157 | 361 | -267 | 44 | -6.068181818181818 | -11748 | 0 | 0 | 491 | 44 | 0 | 0 | 491 | 44 | 245 | 22.0 | 491.0 | 44.0 | 11.159090909090908 | 21604.0 | 1738883764620.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [] | ||||||||||||||||||||||||||||
| 29 | null | Continue with Facebook | null | android.widget.Button | com.zhiliaoapp.musically | null | android.view.accessibility.Acc | null | null | null | [AccessibilityAction: ACTION_F | Rect(1291, 283 - 1024, 327) | Rect(0, 0 - 491, 44) | 02-06 20:16:04.620 | 1291 | 283 | 1024 | 327 | 1291 | 283 | 1024 | 327 | 1157 | 305 | -267 | 44 | -6.068181818181818 | -11748 | 0 | 0 | 491 | 44 | 0 | 0 | 491 | 44 | 245 | 22.0 | 491.0 | 44.0 | 11.159090909090908 | 21604.0 | 1738883764620.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [] |
| Advantages ✅ | Disadvantages ❌ |
|---|---|
| Detects (in theory) all text on the screen | Higher CPU usage compared to other parsers |
| Works with Unreal Engine and Roblox games | May produce inaccurate results (mostly resolved by fast C++-implemented Levenshtein distance calculation) |
| Can categorize elements (image, text, line, etc.) | Requires additional downloads |
| Undetectable | Needs a screenshot to function |
| Can be used alongside any other parser | — |
# Imports and instance creation omitted for brevity
df = pd.DataFrame()
# Loop to find the "TikTok" element and click it
while df.empty:
# word_group_limit: groups words within 40 pixels
df = te.get_df_tesseract_fast(with_screenshot=True, word_group_limit=40)
# OCR is often not 100% accurate, so we use double Damerau-Levenshtein distance
# (right-to-left and left-to-right) to improve matching accuracy.
# - First argument: list of strings to locate
# - Second argument: column where the text exists in the DataFrame
# - Third argument: new column name for matched results
# There are 3 methods to get the DataFrame:
# The first time you call one of these methods, the required models will be downloaded automatically:
# - te.get_df_tesseract_fast -> uses models from "https://github.com/tesseract-ocr/tessdata_fast"
# - te.get_df_tesseract -> uses models from "https://github.com/tesseract-ocr/tessdata"
# - te.get_df_tesseract_best -> uses models from "https://github.com/tesseract-ocr/tessdata_best"
df = df.d_fm_damerau_levenshtein_distance_2ways(
["TikTok"], "aa_text", "aa_searchfortext"
).sort_values(by=["damerau_levenshtein_distance_2ways_match"], ascending=False)
# Filter: keeping matches with a Damerau-Levenshtein distance < 2.1
df = df.loc[df["damerau_levenshtein_distance_2ways_match"] < 2.1]
# Perform the tap on the best match
df.aa_input_tap.iloc[0]()
timesleep(2)
# Clear DataFrame for the next interaction
df = pd.DataFrame()
# Loop to find the "Already have an account? Log in" button and click it
while df.empty:
df = te.get_df_tesseract_fast(with_screenshot=True, word_group_limit=40)
df = df.d_fm_damerau_levenshtein_distance_2ways(
["Already have an account? Log in"], "aa_text", "aa_searchfortext"
).sort_values(by=["damerau_levenshtein_distance_2ways_match"], ascending=False)
df = df.loc[df["damerau_levenshtein_distance_2ways_match"] < 2.1]
df.aa_input_tap.iloc[0]()
timesleep(2)
df = pd.DataFrame()
# Loop to find the "Use phone / email / username" button and click it
while df.empty:
df = te.get_df_tesseract_fast(with_screenshot=True, word_group_limit=40)
df = df.d_fm_damerau_levenshtein_distance_2ways(
["Use phone / email / username"], "aa_text", "aa_searchfortext"
).sort_values(by=["damerau_levenshtein_distance_2ways_match"], ascending=False)
df = df.loc[df["damerau_levenshtein_distance_2ways_match"] < 2.1]
df.aa_input_tap.iloc[0]()
timesleep(2)
df = pd.DataFrame()
# Loop to find the "Log in" button and click it
while df.empty:
df = te.get_df_tesseract_fast(with_screenshot=True, word_group_limit=40)
df = df.d_fm_damerau_levenshtein_distance_2ways(
["Log in"],
"aa_text",
"aa_searchfortext", # Tesseract fails to get "Email / Username" text
).sort_values(by=["damerau_levenshtein_distance_2ways_match"], ascending=False)
df = df.loc[df["damerau_levenshtein_distance_2ways_match"] < 2.1]
# Clicking with an offset (90, 50) as Tesseract may miss the light gray "Email / Username" text
df.aa_input_tap.iloc[0](90, 50)
timesleep(2)
# Sending the email input naturally
te.get_cmd_send_text_natural("myemailtest@gmail.com")()| index | aa_text | aa_text_group | aa_word | aa_title | aa_id | aa_lang | aa_clazz | aa_tag | aa_bbox | aa_baseline | aa_poly | aa_x_bboxes | aa_x_font | aa_x_fsize | aa_x_size | aa_x_ascenders | aa_x_descenders | aa_x_wconf | aa_x_confs | aa_x_mpconf | aa_line_conf | aa_char_conf | aa_ppageno | aa_block_num | aa_par_num | aa_line_num | aa_word_num | aa_image | aa_scan_res | aa_rotate | aa_x_line_bboxes | aa_x_line_confs | aa_x_text | aa_line_index | aa_children | aa_parents | aa_start_x | aa_start_y | aa_end_x | aa_end_y | aa_center_x | aa_center_y | aa_width | aa_height | aa_area | aa_w_h_relation | aa_input_tap | aa_sendevent_mouse_click | aa_sendevent_tap | aa_mouse_action | aa_screenshot |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 21 | -1 | bbox 413 126 612 149baseline | line_1_5 | ocr_line | 413 126 612 149 | 0 -5 | 23 | 5 | 5 | 34 | () | () | 413 | 126 | 612 | 149 | 512 | 137 | 199 | 23 | 4577 | 8.652174 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||
| 22 | Sign up for TikTok | 6 | Sign | bbox 413 126 461 149x_wconf 9 | word_1_7 | ocrx_word | 413 126 461 149 | 24 | 96 | 35 | () | () | 413 | 126 | 461 | 149 | 437 | 137 | 48 | 23 | 1104 | 2.086957 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||
| 23 | Sign up for TikTok | 6 | up | bbox 468 131 495 149x_wconf 9 | word_1_8 | ocrx_word | 468 131 495 149 | 24 | 94 | 36 | () | () | 468 | 131 | 495 | 149 | 481 | 140 | 27 | 18 | 486 | 1.5 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[206 206 206] [147 147 147 | ||||||||||||||||||||||||
| 24 | Sign up for TikTok | 6 | for | bbox 502 127 532 144x_wconf 9 | word_1_9 | ocrx_word | 502 127 532 144 | 24 | 93 | 37 | () | () | 502 | 127 | 532 | 144 | 517 | 135 | 30 | 17 | 510 | 1.764706 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||
| 25 | Sign up for TikTok | 6 | TikTok | bbox 539 126 612 144x_wconf 9 | word_1_10 | ocrx_word | 539 126 612 144 | 24 | 92 | 38 | () | () | 539 | 126 | 612 | 144 | 575 | 135 | 73 | 18 | 1314 | 4.055556 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||
| 26 | -1 | bbox 500 158 742 165 | block_1_6 | ocr_photo | 500 158 742 165 | 42 | () | () | 500 | 158 | 742 | 165 | 621 | 161 | 242 | 7 | 1694 | 34.571429 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||||||
| 27 | -1 | bbox 486 169 500 177 | block_1_7 | ocr_carea | 486 169 500 177 | 43 | () | () | 486 | 169 | 500 | 177 | 493 | 173 | 14 | 8 | 112 | 1.75 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[251 251 251] [191 191 191 | ||||||||||||||||||||||||||||
| 28 | -1 | bbox 486 169 500 177 | par_1_6 | eng | ocr_par | 486 169 500 177 | 44 | () | () | 486 | 169 | 500 | 177 | 493 | 173 | 14 | 8 | 112 | 1.75 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[251 251 251] [191 191 191 | |||||||||||||||||||||||||||
| 29 | -1 | bbox 486 169 500 177baseline | line_1_6 | ocr_line | 486 169 500 177 | 0 -6 | 20 | 5 | 5 | 45 | () | () | 486 | 169 | 500 | 177 | 493 | 173 | 14 | 8 | 112 | 1.75 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[251 251 251] [191 191 191 | ||||||||||||||||||||||||
| 30 | ac file, follow othe | 7 | ac | bbox 486 169 500 177x_wconf 8 | word_1_11 | ocrx_word | 486 169 500 177 | 21 | 87 | 46 | () | () | 486 | 169 | 500 | 177 | 493 | 173 | 14 | 8 | 112 | 1.75 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[251 251 251] [191 191 191 | ||||||||||||||||||||||||
| 31 | -1 | bbox 364 166 527 196 | block_1_8 | ocr_carea | 364 166 527 196 | 50 | () | () | 364 | 166 | 527 | 196 | 445 | 181 | 163 | 30 | 4890 | 5.433333 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [249 249 249 | ||||||||||||||||||||||||||||
| 32 | -1 | bbox 364 166 527 196 | par_1_7 | eng | ocr_par | 364 166 527 196 | 51 | () | () | 364 | 166 | 527 | 196 | 445 | 181 | 163 | 30 | 4890 | 5.433333 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [249 249 249 | |||||||||||||||||||||||||||
| 33 | -1 | bbox 364 166 474 177baseline | line_1_7 | ocr_line | 364 166 474 177 | 0 0 | 20 | 5 | 5 | 52 | () | () | 364 | 166 | 474 | 177 | 419 | 171 | 110 | 11 | 1210 | 10.0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [249 249 249 | ||||||||||||||||||||||||
| 34 | ac file, follow othe | 7 | file, | bbox 364 169 384 177x_wconf 8 | word_1_12 | ocrx_word | 364 169 384 177 | 21 | 89 | 53 | () | () | 364 | 169 | 384 | 177 | 374 | 173 | 20 | 8 | 160 | 2.5 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[149 149 149] [133 133 133 | ||||||||||||||||||||||||
| 35 | ac file, follow othe | 7 | follow | bbox 396 166 436 177x_wconf 9 | word_1_13 | ocrx_word | 396 166 436 177 | 21 | 96 | 54 | () | () | 396 | 166 | 436 | 177 | 416 | 171 | 40 | 11 | 440 | 3.636364 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [215 215 215 | ||||||||||||||||||||||||
| 36 | ac file, follow othe | 7 | othe | bbox 444 166 474 177x_wconf 9 | word_1_14 | ocrx_word | 444 166 474 177 | 21 | 96 | 55 | () | () | 444 | 166 | 474 | 177 | 459 | 171 | 30 | 11 | 330 | 2.727273 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||||||||||||||||||||||||
| 37 | -1 | bbox 494 188 527 196baseline | line_1_8 | ocr_line | 494 188 527 196 | 0 0 | 20 | 5 | 5 | 57 | () | () | 494 | 188 | 527 | 196 | 510 | 192 | 33 | 8 | 264 | 4.125 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[166 166 166] [219 219 219 |
| Advantages ✅ | Disadvantages ❌ |
|---|---|
| The fastest of all parsers | Results don’t include the element’s text |
| Very low CPU usage | Sometimes misses elements that UIAutomator-based parsers can detect |
| Can be used alongside any other parser | — |
| Provides useful additional information that other parsers don’t have | — |
| Can find elements that UIAutomator-based parsers can’t detect | — |
| Undetectable | — |
| Highly reliable—never fails | — |
# Imports and instance creation omitted for brevity
df = pd.DataFrame()
# Loop to find and interact with the first target element
while df.empty:
# Many elements in the DataFrame are repeated intentionally to observe parent/child relationships.
# However, with_screenshot=True remains efficient because it crops the numpy arrays only once per element.
df = te.get_df_fragments(with_screenshot=True)
df = df.loc[
(
df.aa_classname
== "com.android.launcher3.views.DoubleShadowBubbleTextView"
)
& (df.aa_rel_width_height > 1)
& (df.aa_clickable == "C")
& (df.aa_drawn == "D")
& (df.aa_enabled == "E")
& (df.aa_focusable == "F")
& (df.aa_long_clickable == "L")
& (df.aa_visibility == "V")
& (df.aa_is_child)
& (df.aa_screenshot.str.len() > 0) # It's a numpy array, but this works as expected
].sort_values(by=["aa_start_y"])
df.aa_input_tap.iloc[0]()
df = pd.DataFrame()
timesleep(10)
while df.empty:
df = te.get_df_fragments(with_screenshot=True)
df = df.loc[
(df.aa_classname == "com.bytedance.tux.input.TuxTextView")
& (df.aa_rel_width_height > 15)
& (df.aa_clickable == "C")
& (df.aa_enabled == "E")
& (df.aa_focusable == "F")
& (df.aa_visibility == "V")
& (df.aa_drawn == "D")
& (df.aa_is_child)
& (df.aa_long_clickable == ".")
& (df.aa_screenshot.str.len() > 0)
].sort_values(by=["aa_start_x", "aa_start_y_relative"])
df.aa_input_tap.iloc[0]()
df = pd.DataFrame()
timesleep(3)
while df.empty:
df = te.get_df_fragments(with_screenshot=True)
df = df.loc[
(df.aa_classname == "com.bytedance.tux.input.TuxTextView")
& (df.aa_rel_width_height > 6)
& (df.aa_rel_width_height < 9)
& (df.aa_enabled == "E")
& (df.aa_visibility == "V")
& (df.aa_drawn == "D")
& (df.aa_clickable == ".")
& (df.aa_focusable == ".")
& (df.aa_is_child)
& (df.aa_long_clickable == ".")
& (df.aa_screenshot.str.len() > 0)
].sort_values(by=["aa_start_y"])
df.aa_input_tap.iloc[0]()
df = pd.DataFrame()
timesleep(3)
while df.empty:
df = te.get_df_fragments(with_screenshot=True)
df = df.loc[
(df.aa_classname == "X.iti")
& (df.aa_rel_width_height > 3)
& (df.aa_rel_width_height < 6)
& (df.aa_enabled == "E")
& (df.aa_visibility == "V")
& (df.aa_drawn == ".")
& (df.aa_clickable == "C")
& (df.aa_focusable == "F")
& (~df.aa_is_child)
& (df.aa_long_clickable == ".")
& (df.aa_screenshot.str.len() > 0)
].sort_values(by=["aa_start_x"], ascending=[False])
df.aa_input_tap.iloc[0]()
timesleep(2)
# Send the email input naturally
te.get_cmd_send_text_natural("myemailtest@gmail.com")()
| index | aa_my_id | aa_my_group_id | aa_my_element_id | aa_my_direct_parent_id | aa_my_parent_ids | aa_original_string | aa_center_x | aa_center_y | aa_area | aa_start_x | aa_start_y | aa_end_x | aa_end_y | aa_height | aa_width | aa_is_sqare | aa_rel_width_height | aa_hashcode_int | aa_mid_int | aa_spaces | aa_classname | aa_element_id | aa_hashcode | aa_mid | aa_start_x_relative | aa_end_x_relative | aa_start_y_relative | aa_end_y_relative | aa_clickable | aa_context_clickable | aa_drawn | aa_enabled | aa_focusable | aa_long_clickable | aa_pflag_activated | aa_pflag_dirty_mask | aa_pflag_focused | aa_pflag_hovered | aa_pflag_invalidated | aa_pflag_is_root_namespace | aa_pflag_prepressed | aa_pflag_selected | aa_scrollbars_horizontal | aa_scrollbars_vertical | aa_visibility | aa_is_child | aa_input_tap | aa_sendevent_mouse_click | aa_sendevent_tap | aa_mouse_action | aa_screenshot |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 92 | 0 | 0 | 0 | android.view.View{7181534 V.ED | 512 | 752 | 32768 | 0 | 736 | 1024 | 768 | 32 | 1024 | 0 | 32.0 | 119018804 | 16908336 | 8 | android.view.View | android:id/navigationBarBackgr | 7181534 | 1020030 | 0 | 1024 | 736 | 768 | . | . | D | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 1 | 91 | 1 | 0 | 0 | android.view.View{2b0fec9 V.ED | 512 | 12 | 24576 | 0 | 0 | 1024 | 24 | 24 | 1024 | 0 | 42.666668 | 45153993 | 16908335 | 8 | android.view.View | android:id/statusBarBackground | 2b0fec9 | 102002f | 0 | 1024 | 0 | 24 | . | . | D | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 2 | 0 | 2 | 0 | 0 | android.widget.LinearLayout{2b | 512 | 368 | 753664 | 0 | 0 | 1024 | 736 | 736 | 1024 | 0 | 1.3913044 | 45420503 | 0 | 8 | android.widget.LinearLayout | 2b50fd7 | 0 | 1024 | 0 | 736 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 3 | 2 | 2 | 1 | 0 | 0, | android.widget.FrameLayout{7 | 512 | 368 | 753664 | 0 | 0 | 1024 | 736 | 736 | 1024 | 0 | 1.3913044 | 131089238 | 16908290 | 10 | android.widget.FrameLayout | android:id/content | 7d04356 | 1020002 | 0 | 1024 | 0 | 736 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 |
| 4 | 3 | 2 | 2 | 2 | 0,2, | X.jTV{c982971 V.E...... .. | 512 | 368 | 753664 | 0 | 0 | 1024 | 736 | 736 | 1024 | 0 | 1.3913044 | 211298673 | 0 | 12 | X.jTV | c982971 | 0 | 1024 | 0 | 736 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 5 | 7 | 2 | 3 | 3 | 0,2,3, | android.widget.FrameLayo | 512 | 368 | 753664 | 0 | 0 | 1024 | 736 | 736 | 1024 | 0 | 1.3913044 | 53721880 | 2131435664 | 14 | android.widget.FrameLayout | app:id/frl | 333bb18 | 7f0b2090 | 0 | 1024 | 0 | 736 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 |
| 6 | 8 | 2 | 4 | 7 | 0,2,3,7, | X.isZ{f77efb V.E...... | 512 | 380 | 729088 | 0 | 24 | 1024 | 736 | 712 | 1024 | 0 | 1.4382023 | 16219899 | 2131432044 | 16 | X.isZ | app:id/d4r | f77efb | 7f0b126c | 0 | 1024 | 24 | 736 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 |
| 7 | 9 | 2 | 5 | 8 | 0,2,3,7,8, | android.widget.Relat | 512 | 380 | 729088 | 0 | 24 | 1024 | 736 | 712 | 1024 | 0 | 1.4382023 | 82542831 | 2131446746 | 18 | android.widget.RelativeLayout | app:id/nv5 | 4eb80ef | 7f0b4bda | 0 | 1024 | 0 | 712 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 |
| 8 | 90 | 2 | 6 | 9 | 0,2,3,7,8,9 | com.bytedance.tux. | 994 | 47 | 504 | 980 | 38 | 1008 | 56 | 18 | 28 | 0 | 1.5555556 | 194303692 | 2131448536 | 20 | com.bytedance.tux.input.TuxTex | app:id/p6i | b94d6cc | 7f0b52d8 | 980 | 1008 | 14 | 32 | C | . | D | E | F | . | . | . | . | . | . | . | . | . | . | . | V | True | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 |
| 9 | 0 | 3 | 0 | 0 | android.widget.LinearLayout{2b | 512 | 368 | 753664 | 0 | 0 | 1024 | 736 | 736 | 1024 | 0 | 1.3913044 | 45420503 | 0 | 8 | android.widget.LinearLayout | 2b50fd7 | 0 | 1024 | 0 | 736 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 10 | 2 | 3 | 1 | 0 | 0, | android.widget.FrameLayout{7 | 512 | 368 | 753664 | 0 | 0 | 1024 | 736 | 736 | 1024 | 0 | 1.3913044 | 131089238 | 16908290 | 10 | android.widget.FrameLayout | android:id/content | 7d04356 | 1020002 | 0 | 1024 | 0 | 736 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 |
| 11 | 3 | 3 | 2 | 2 | 0,2, | X.jTV{c982971 V.E...... .. | 512 | 368 | 753664 | 0 | 0 | 1024 | 736 | 736 | 1024 | 0 | 1.3913044 | 211298673 | 0 | 12 | X.jTV | c982971 | 0 | 1024 | 0 | 736 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 12 | 7 | 3 | 3 | 3 | 0,2,3, | android.widget.FrameLayo | 512 | 368 | 753664 | 0 | 0 | 1024 | 736 | 736 | 1024 | 0 | 1.3913044 | 53721880 | 2131435664 | 14 | android.widget.FrameLayout | app:id/frl | 333bb18 | 7f0b2090 | 0 | 1024 | 0 | 736 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 |
| 13 | 8 | 3 | 4 | 7 | 0,2,3,7, | X.isZ{f77efb V.E...... | 512 | 380 | 729088 | 0 | 24 | 1024 | 736 | 712 | 1024 | 0 | 1.4382023 | 16219899 | 2131432044 | 16 | X.isZ | app:id/d4r | f77efb | 7f0b126c | 0 | 1024 | 24 | 736 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 |
| 14 | 9 | 3 | 5 | 8 | 0,2,3,7,8, | android.widget.Relat | 512 | 380 | 729088 | 0 | 24 | 1024 | 736 | 712 | 1024 | 0 | 1.4382023 | 82542831 | 2131446746 | 18 | android.widget.RelativeLayout | app:id/nv5 | 4eb80ef | 7f0b4bda | 0 | 1024 | 0 | 712 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 |
| 15 | 89 | 3 | 6 | 9 | 0,2,3,7,8,9 | com.bytedance.tux. | 0 | 24 | 0 | 0 | 24 | 0 | 24 | 0 | 0 | 1 | 0.0 | 134974938 | 2131451338 | 20 | com.bytedance.tux.icon.TuxIcon | app:id/r8_ | 80b8dda | 7f0b5dca | 0 | 0 | 0 | 0 | . | . | D | E | . | . | . | . | . | . | I | . | . | . | . | . | G | True | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [] |
| 16 | 0 | 4 | 0 | 0 | android.widget.LinearLayout{2b | 512 | 368 | 753664 | 0 | 0 | 1024 | 736 | 736 | 1024 | 0 | 1.3913044 | 45420503 | 0 | 8 | android.widget.LinearLayout | 2b50fd7 | 0 | 1024 | 0 | 736 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 17 | 2 | 4 | 1 | 0 | 0, | android.widget.FrameLayout{7 | 512 | 368 | 753664 | 0 | 0 | 1024 | 736 | 736 | 1024 | 0 | 1.3913044 | 131089238 | 16908290 | 10 | android.widget.FrameLayout | android:id/content | 7d04356 | 1020002 | 0 | 1024 | 0 | 736 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 |
| 18 | 3 | 4 | 2 | 2 | 0,2, | X.jTV{c982971 V.E...... .. | 512 | 368 | 753664 | 0 | 0 | 1024 | 736 | 736 | 1024 | 0 | 1.3913044 | 211298673 | 0 | 12 | X.jTV | c982971 | 0 | 1024 | 0 | 736 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 19 | 7 | 4 | 3 | 3 | 0,2,3, | android.widget.FrameLayo | 512 | 368 | 753664 | 0 | 0 | 1024 | 736 | 736 | 1024 | 0 | 1.3913044 | 53721880 | 2131435664 | 14 | android.widget.FrameLayout | app:id/frl | 333bb18 | 7f0b2090 | 0 | 1024 | 0 | 736 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 |
| 20 | 8 | 4 | 4 | 7 | 0,2,3,7, | X.isZ{f77efb V.E...... | 512 | 380 | 729088 | 0 | 24 | 1024 | 736 | 712 | 1024 | 0 | 1.4382023 | 16219899 | 2131432044 | 16 | X.isZ | app:id/d4r | f77efb | 7f0b126c | 0 | 1024 | 24 | 736 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 |
| 21 | 9 | 4 | 5 | 8 | 0,2,3,7,8, | android.widget.Relat | 512 | 380 | 729088 | 0 | 24 | 1024 | 736 | 712 | 1024 | 0 | 1.4382023 | 82542831 | 2131446746 | 18 | android.widget.RelativeLayout | app:id/nv5 | 4eb80ef | 7f0b4bda | 0 | 1024 | 0 | 712 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 |
| 22 | 88 | 4 | 6 | 9 | 0,2,3,7,8,9 | com.bytedance.tux. | 28 | 50 | 2304 | 4 | 26 | 52 | 74 | 48 | 48 | 1 | 1.0 | 13215999 | 2131451307 | 20 | com.bytedance.tux.icon.TuxIcon | app:id/r7f | c9a8ff | 7f0b5dab | 4 | 52 | 2 | 50 | C | . | D | E | F | . | . | . | . | . | . | . | . | . | . | . | V | True | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 |
| 23 | 0 | 5 | 0 | 0 | android.widget.LinearLayout{2b | 512 | 368 | 753664 | 0 | 0 | 1024 | 736 | 736 | 1024 | 0 | 1.3913044 | 45420503 | 0 | 8 | android.widget.LinearLayout | 2b50fd7 | 0 | 1024 | 0 | 736 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 24 | 2 | 5 | 1 | 0 | 0, | android.widget.FrameLayout{7 | 512 | 368 | 753664 | 0 | 0 | 1024 | 736 | 736 | 1024 | 0 | 1.3913044 | 131089238 | 16908290 | 10 | android.widget.FrameLayout | android:id/content | 7d04356 | 1020002 | 0 | 1024 | 0 | 736 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 |
| 25 | 3 | 5 | 2 | 2 | 0,2, | X.jTV{c982971 V.E...... .. | 512 | 368 | 753664 | 0 | 0 | 1024 | 736 | 736 | 1024 | 0 | 1.3913044 | 211298673 | 0 | 12 | X.jTV | c982971 | 0 | 1024 | 0 | 736 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 26 | 7 | 5 | 3 | 3 | 0,2,3, | android.widget.FrameLayo | 512 | 368 | 753664 | 0 | 0 | 1024 | 736 | 736 | 1024 | 0 | 1.3913044 | 53721880 | 2131435664 | 14 | android.widget.FrameLayout | app:id/frl | 333bb18 | 7f0b2090 | 0 | 1024 | 0 | 736 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 |
| 27 | 8 | 5 | 4 | 7 | 0,2,3,7, | X.isZ{f77efb V.E...... | 512 | 380 | 729088 | 0 | 24 | 1024 | 736 | 712 | 1024 | 0 | 1.4382023 | 16219899 | 2131432044 | 16 | X.isZ | app:id/d4r | f77efb | 7f0b126c | 0 | 1024 | 24 | 736 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 |
| 28 | 9 | 5 | 5 | 8 | 0,2,3,7,8, | android.widget.Relat | 512 | 380 | 729088 | 0 | 24 | 1024 | 736 | 712 | 1024 | 0 | 1.4382023 | 82542831 | 2131446746 | 18 | android.widget.RelativeLayout | app:id/nv5 | 4eb80ef | 7f0b4bda | 0 | 1024 | 0 | 712 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 |
| 29 | 12 | 5 | 6 | 9 | 0,2,3,7,8,9, | android.widget.Lin | 512 | 406 | 674816 | 0 | 77 | 1024 | 736 | 659 | 1024 | 0 | 1.5538695 | 219162762 | 2131448421 | 20 | android.widget.LinearLayout | app:id/p3e | d10288a | 7f0b5265 | 0 | 1024 | 53 | 712 | . | . | . | E | . | . | . | . | . | . | . | . | . | . | . | . | V | False | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 |
| Advantages ✅ | Disadvantages ❌ |
|---|---|
| Includes both element text and description | Higher CPU usage compared to UIAutomator2 |
| Undetectable | May fail (idle state error) if there’s too much activity on the screen |
| Can be used alongside any other parser (except UIAutomator2) | — |
# Imports and instance creation omitted for brevity
df = pd.DataFrame()
while df.empty:
df = te.get_df_uiautomator_classic(with_screenshot=True)
df = df.loc[
(df.aa_text == "TikTok")
& (df.aa_content_desc == "TikTok")
& (df.aa_clickable == 1)
]
df.aa_input_tap.iloc[0]()
df = pd.DataFrame()
timesleep(10)
while df.empty:
df = te.get_df_uiautomator_classic(with_screenshot=True)
df = df.loc[
(df.aa_text == "Already have an account? Log in")
& (df.aa_clazz == "android.widget.Button")
& (df.aa_content_desc == "")
]
df.aa_input_tap.iloc[0]()
df = pd.DataFrame()
timesleep(3)
while df.empty:
df = te.get_df_uiautomator_classic(with_screenshot=True)
df = df.loc[
(df.aa_text == "Use phone / email / username")
& (df.aa_clazz == "android.widget.TextView")
& (df.aa_content_desc == "")
]
df.aa_input_tap.iloc[0]()
df = pd.DataFrame()
timesleep(3)
while df.empty:
df = te.get_df_uiautomator_classic(with_screenshot=True)
df = df.loc[
(df.aa_text == "Email / Username")
& (df.aa_clazz == "android.widget.TextView")
& (df.aa_content_desc == "")
]
df.aa_input_tap.iloc[0]()
df = pd.DataFrame()
timesleep(2)
te.get_cmd_send_text_natural("myemailtest@gmail.com")()| index | aa_bounds | aa_text | aa_package | aa_resource_id | aa_clazz | aa_content_desc | aa_center_x | aa_center_y | aa_area | aa_width | aa_height | aa_start_x | aa_start_y | aa_end_x | aa_end_y | aa_is_square | aa_w_h_relation | aa_checkable | aa_checked | aa_clickable | aa_enabled | aa_focusable | aa_focused | aa_index | aa_long_clickable | aa_password | aa_scrollable | aa_selected | aa_naf | aa_input_tap | aa_sendevent_mouse_click | aa_sendevent_tap | aa_mouse_action | aa_screenshot |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | [0,0][1024,768] | com.zhiliaoapp.musically | android.widget.FrameLayout | 512 | 384 | 786432 | 1024 | 768 | 0 | 0 | 1024 | 768 | 0 | 1.3333334 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 1 | [0,0][1024,736] | com.zhiliaoapp.musically | android.widget.LinearLayout | 512 | 368 | 753664 | 1024 | 736 | 0 | 0 | 1024 | 736 | 0 | 1.3913044 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 2 | [0,0][1024,736] | com.zhiliaoapp.musically | android:id/content | android.widget.FrameLayout | 512 | 368 | 753664 | 1024 | 736 | 0 | 0 | 1024 | 736 | 0 | 1.3913044 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 3 | [0,0][1024,736] | com.zhiliaoapp.musically | android.widget.FrameLayout | 512 | 368 | 753664 | 1024 | 736 | 0 | 0 | 1024 | 736 | 0 | 1.3913044 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 4 | [0,0][1024,736] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/tq | android.widget.LinearLayout | 512 | 368 | 753664 | 1024 | 736 | 0 | 0 | 1024 | 736 | 0 | 1.3913044 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 5 | [0,0][1024,736] | com.zhiliaoapp.musically | android.widget.FrameLayout | 512 | 368 | 753664 | 1024 | 736 | 0 | 0 | 1024 | 736 | 0 | 1.3913044 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 6 | [0,0][1024,736] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/fr | android.widget.FrameLayout | 512 | 368 | 753664 | 1024 | 736 | 0 | 0 | 1024 | 736 | 0 | 1.3913044 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 7 | [0,24][1024,736] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/d4 | android.view.ViewGroup | 512 | 380 | 729088 | 1024 | 712 | 0 | 24 | 1024 | 736 | 0 | 1.4382023 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 8 | [0,24][1024,736] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/nv | android.widget.RelativeLayout | 512 | 380 | 729088 | 1024 | 712 | 0 | 24 | 1024 | 736 | 0 | 1.4382023 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 9 | [4,26][52,74] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/r7 | android.widget.ImageView | Report a problem | 28 | 50 | 2304 | 48 | 48 | 4 | 26 | 52 | 74 | 1 | 1.0 | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 10 | [980,38][1008,56] | Skip | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/p6 | android.widget.Button | 994 | 47 | 504 | 28 | 18 | 980 | 38 | 1008 | 56 | 0 | 1.5555556 | 0 | 0 | 1 | 1 | 1 | 0 | 2 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 11 | [0,77][1024,736] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/p3 | android.widget.LinearLayout | 512 | 406 | 674816 | 1024 | 659 | 0 | 77 | 1024 | 736 | 0 | 1.5538695 | 0 | 0 | 0 | 1 | 0 | 0 | 4 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 12 | [0,77][1024,736] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/tt | androidx.viewpager.widget.View | 512 | 406 | 674816 | 1024 | 659 | 0 | 77 | 1024 | 736 | 0 | 1.5538695 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 13 | [0,77][1024,736] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/p3 | android.view.ViewGroup | 512 | 406 | 674816 | 1024 | 659 | 0 | 77 | 1024 | 736 | 0 | 1.5538695 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 14 | [30,77][994,628] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/c7 | android.view.ViewGroup | 512 | 352 | 531164 | 964 | 551 | 30 | 77 | 994 | 628 | 0 | 1.7495463 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 15 | [267,77][758,396] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/c7 | androidx.recyclerview.widget.R | 512 | 236 | 156629 | 491 | 319 | 267 | 77 | 758 | 396 | 0 | 1.5391849 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 16 | [267,82][758,228] | com.zhiliaoapp.musically | android.widget.LinearLayout | 512 | 155 | 71686 | 491 | 146 | 267 | 82 | 758 | 228 | 0 | 3.3630137 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 17 | [267,122][758,150] | Sign up for TikTok | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/ti | android.widget.TextView | 512 | 136 | 13748 | 491 | 28 | 267 | 122 | 758 | 150 | 0 | 17.535715 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 18 | [267,162][758,200] | Create a profile, follow other | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/oe | android.widget.TextView | 512 | 181 | 18658 | 491 | 38 | 267 | 162 | 758 | 200 | 0 | 12.921053 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 19 | [267,228][758,272] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/cz | android.widget.Button | Use phone or email | 512 | 250 | 21604 | 491 | 44 | 267 | 228 | 758 | 272 | 0 | 11.159091 | 0 | 0 | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 20 | [279,238][303,262] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/gs | android.widget.ImageView | 291 | 250 | 576 | 24 | 24 | 279 | 238 | 303 | 262 | 1 | 1.0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 21 | [454,241][593,260] | Use phone or email | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/c7 | android.widget.TextView | 523 | 250 | 2641 | 139 | 19 | 454 | 241 | 593 | 260 | 0 | 7.3157897 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 22 | [267,284][758,328] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/cz | android.widget.Button | Continue with Facebook | 512 | 306 | 21604 | 491 | 44 | 267 | 284 | 758 | 328 | 0 | 11.159091 | 0 | 0 | 1 | 1 | 1 | 0 | 2 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 23 | [279,294][303,318] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/gs | android.widget.ImageView | 291 | 306 | 576 | 24 | 24 | 279 | 294 | 303 | 318 | 1 | 1.0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 24 | [437,297][611,316] | Continue with Facebook | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/c7 | android.widget.TextView | 524 | 306 | 3306 | 174 | 19 | 437 | 297 | 611 | 316 | 0 | 9.157895 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 25 | [267,340][758,384] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/cz | android.widget.Button | Continue with Google | 512 | 362 | 21604 | 491 | 44 | 267 | 340 | 758 | 384 | 0 | 11.159091 | 0 | 0 | 1 | 1 | 1 | 0 | 3 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 26 | [279,350][303,374] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/gs | android.widget.ImageView | 291 | 362 | 576 | 24 | 24 | 279 | 350 | 303 | 374 | 1 | 1.0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 27 | [445,353][602,371] | Continue with Google | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/c7 | android.widget.TextView | 523 | 362 | 2826 | 157 | 18 | 445 | 353 | 602 | 371 | 0 | 8.722222 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 28 | [32,628][992,675] | By continuing with an account | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/rw | android.widget.TextView | 512 | 651 | 45120 | 960 | 47 | 32 | 628 | 992 | 675 | 0 | 20.425531 | 0 | 0 | 0 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 29 | [0,675][1024,676] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/dx | android.view.View | 512 | 675 | 1024 | 1024 | 1 | 0 | 675 | 1024 | 676 | 0 | 1024.0 | 0 | 0 | 0 | 1 | 0 | 0 | 2 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[228 228 229] [228 228 229 |
| Advantages ✅ | Disadvantages ❌ |
|---|---|
| Includes both element text and description | Higher CPU usage compared to UIAutomator2 |
| Undetectable | May fail (null root node returned by UiTestAutomationBridge) if the CPU usage of the top window is limited too much |
| Can be used alongside any other parser (except UIAutomator2) | — |
| No idle state errors when used correctly | — |
# Imports and instance creation omitted for brevity
df = pd.DataFrame()
while df.empty:
# If cpu_limit is not passed, the value from the TermuxAutomation instance is used
df = te.get_df_uiautomator_with_cpu_limit(with_screenshot=True)
df = df.loc[
(df.aa_text == "TikTok")
& (df.aa_content_desc == "TikTok")
& (df.aa_clickable == 1)
]
df.aa_input_tap.iloc[0]()
df = pd.DataFrame()
timesleep(10)
while df.empty:
df = te.get_df_uiautomator_with_cpu_limit(with_screenshot=True, cpu_limit=14)
df = df.loc[
(df.aa_text == "Already have an account? Log in")
& (df.aa_clazz == "android.widget.Button")
& (df.aa_content_desc == "")
]
df.aa_input_tap.iloc[0]()
df = pd.DataFrame()
timesleep(3)
while df.empty:
df = te.get_df_uiautomator_with_cpu_limit(with_screenshot=True, cpu_limit=13)
df = df.loc[
(df.aa_text == "Use phone / email / username")
& (df.aa_clazz == "android.widget.TextView")
& (df.aa_content_desc == "")
]
df.aa_input_tap.iloc[0]()
df = pd.DataFrame()
timesleep(3)
while df.empty:
df = te.get_df_uiautomator_with_cpu_limit(with_screenshot=True, cpu_limit=16)
df = df.loc[
(df.aa_text == "Email / Username")
& (df.aa_clazz == "android.widget.TextView")
& (df.aa_content_desc == "")
]
df.aa_input_tap.iloc[0]()
df = pd.DataFrame()
timesleep(2)
te.get_cmd_send_text_natural("myemailtest@gmail.com")()| index | aa_bounds | aa_text | aa_package | aa_resource_id | aa_clazz | aa_content_desc | aa_center_x | aa_center_y | aa_area | aa_width | aa_height | aa_start_x | aa_start_y | aa_end_x | aa_end_y | aa_is_square | aa_w_h_relation | aa_checkable | aa_checked | aa_clickable | aa_enabled | aa_focusable | aa_focused | aa_index | aa_long_clickable | aa_password | aa_scrollable | aa_selected | aa_naf | aa_input_tap | aa_sendevent_mouse_click | aa_sendevent_tap | aa_mouse_action | aa_screenshot |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | [0,0][1024,768] | com.zhiliaoapp.musically | android.widget.FrameLayout | 512 | 384 | 786432 | 1024 | 768 | 0 | 0 | 1024 | 768 | 0 | 1.3333334 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 1 | [0,0][1024,736] | com.zhiliaoapp.musically | android.widget.LinearLayout | 512 | 368 | 753664 | 1024 | 736 | 0 | 0 | 1024 | 736 | 0 | 1.3913044 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 2 | [0,0][1024,736] | com.zhiliaoapp.musically | android:id/content | android.widget.FrameLayout | 512 | 368 | 753664 | 1024 | 736 | 0 | 0 | 1024 | 736 | 0 | 1.3913044 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 3 | [0,0][1024,736] | com.zhiliaoapp.musically | android.widget.FrameLayout | 512 | 368 | 753664 | 1024 | 736 | 0 | 0 | 1024 | 736 | 0 | 1.3913044 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 4 | [0,0][1024,736] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/tq | android.widget.LinearLayout | 512 | 368 | 753664 | 1024 | 736 | 0 | 0 | 1024 | 736 | 0 | 1.3913044 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 5 | [0,0][1024,736] | com.zhiliaoapp.musically | android.widget.FrameLayout | 512 | 368 | 753664 | 1024 | 736 | 0 | 0 | 1024 | 736 | 0 | 1.3913044 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 6 | [0,0][1024,736] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/fr | android.widget.FrameLayout | 512 | 368 | 753664 | 1024 | 736 | 0 | 0 | 1024 | 736 | 0 | 1.3913044 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 7 | [0,24][1024,736] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/d4 | android.view.ViewGroup | 512 | 380 | 729088 | 1024 | 712 | 0 | 24 | 1024 | 736 | 0 | 1.4382023 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 8 | [0,24][1024,736] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/nv | android.widget.RelativeLayout | 512 | 380 | 729088 | 1024 | 712 | 0 | 24 | 1024 | 736 | 0 | 1.4382023 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 9 | [4,26][52,74] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/r7 | android.widget.ImageView | Report a problem | 28 | 50 | 2304 | 48 | 48 | 4 | 26 | 52 | 74 | 1 | 1.0 | 0 | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 10 | [980,38][1008,56] | Skip | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/p6 | android.widget.Button | 994 | 47 | 504 | 28 | 18 | 980 | 38 | 1008 | 56 | 0 | 1.5555556 | 0 | 0 | 1 | 1 | 1 | 0 | 2 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 11 | [0,77][1024,736] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/p3 | android.widget.LinearLayout | 512 | 406 | 674816 | 1024 | 659 | 0 | 77 | 1024 | 736 | 0 | 1.5538695 | 0 | 0 | 0 | 1 | 0 | 0 | 4 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 12 | [0,77][1024,736] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/tt | androidx.viewpager.widget.View | 512 | 406 | 674816 | 1024 | 659 | 0 | 77 | 1024 | 736 | 0 | 1.5538695 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 13 | [0,77][1024,736] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/p3 | android.view.ViewGroup | 512 | 406 | 674816 | 1024 | 659 | 0 | 77 | 1024 | 736 | 0 | 1.5538695 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 14 | [30,77][994,628] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/c7 | android.view.ViewGroup | 512 | 352 | 531164 | 964 | 551 | 30 | 77 | 994 | 628 | 0 | 1.7495463 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 15 | [267,77][758,396] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/c7 | androidx.recyclerview.widget.R | 512 | 236 | 156629 | 491 | 319 | 267 | 77 | 758 | 396 | 0 | 1.5391849 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 16 | [267,82][758,228] | com.zhiliaoapp.musically | android.widget.LinearLayout | 512 | 155 | 71686 | 491 | 146 | 267 | 82 | 758 | 228 | 0 | 3.3630137 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |||
| 17 | [267,122][758,150] | Sign up for TikTok | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/ti | android.widget.TextView | 512 | 136 | 13748 | 491 | 28 | 267 | 122 | 758 | 150 | 0 | 17.535715 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 18 | [267,162][758,200] | Create a profile, follow other | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/oe | android.widget.TextView | 512 | 181 | 18658 | 491 | 38 | 267 | 162 | 758 | 200 | 0 | 12.921053 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 19 | [267,228][758,272] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/cz | android.widget.Button | Use phone or email | 512 | 250 | 21604 | 491 | 44 | 267 | 228 | 758 | 272 | 0 | 11.159091 | 0 | 0 | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 20 | [279,238][303,262] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/gs | android.widget.ImageView | 291 | 250 | 576 | 24 | 24 | 279 | 238 | 303 | 262 | 1 | 1.0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 21 | [454,241][593,260] | Use phone or email | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/c7 | android.widget.TextView | 523 | 250 | 2641 | 139 | 19 | 454 | 241 | 593 | 260 | 0 | 7.3157897 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 22 | [267,284][758,328] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/cz | android.widget.Button | Continue with Facebook | 512 | 306 | 21604 | 491 | 44 | 267 | 284 | 758 | 328 | 0 | 11.159091 | 0 | 0 | 1 | 1 | 1 | 0 | 2 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 23 | [279,294][303,318] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/gs | android.widget.ImageView | 291 | 306 | 576 | 24 | 24 | 279 | 294 | 303 | 318 | 1 | 1.0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 24 | [437,297][611,316] | Continue with Facebook | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/c7 | android.widget.TextView | 524 | 306 | 3306 | 174 | 19 | 437 | 297 | 611 | 316 | 0 | 9.157895 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 25 | [267,340][758,384] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/cz | android.widget.Button | Continue with Google | 512 | 362 | 21604 | 491 | 44 | 267 | 340 | 758 | 384 | 0 | 11.159091 | 0 | 0 | 1 | 1 | 1 | 0 | 3 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 26 | [279,350][303,374] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/gs | android.widget.ImageView | 291 | 362 | 576 | 24 | 24 | 279 | 350 | 303 | 374 | 1 | 1.0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | ||
| 27 | [445,353][602,371] | Continue with Google | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/c7 | android.widget.TextView | 523 | 362 | 2826 | 157 | 18 | 445 | 353 | 602 | 371 | 0 | 8.722222 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 28 | [32,628][992,675] | By continuing with an account | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/rw | android.widget.TextView | 512 | 651 | 45120 | 960 | 47 | 32 | 628 | 992 | 675 | 0 | 20.425531 | 0 | 0 | 0 | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[255 255 255] [255 255 255 | |
| 29 | [0,675][1024,676] | com.zhiliaoapp.musically | com.zhiliaoapp.musically:id/dx | android.view.View | 512 | 675 | 1024 | 1024 | 1 | 0 | 675 | 1024 | 676 | 0 | 1024.0 | 0 | 0 | 0 | 1 | 0 | 0 | 2 | 0 | 0 | 0 | 0 | 0 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, durat | (offset_x=0, offset_y=0, actio | [[[228 228 229] [228 228 229 |
| Advantages ✅ | Disadvantages ❌ |
|---|---|
| Very fast parser | Results don’t include the element’s text |
| Very low CPU usage | Doesn’t work everywhere (e.g., the start page of the launcher) |
| Can be used alongside any other parser | — |
| Provides lots of additional information that other parsers don’t have | — |
| Can find elements that UIAutomator-based parsers can’t detect | — |
| Undetectable | — |
# We need to open a shell and start the app using the am command because
# the parser doesn’t work on the launcher’s start page
myshell = te.open_shell()
myshell.sh_force_open_app("com.zhiliaoapp.musically", 3)
timesleep(10)
df = pd.DataFrame()
while df.empty:
df = te.get_df_window_dump(with_screenshot=True)
# Provides lots of additional information no other parser has
# (pivot points, font size, font weight, etc.)
df = df.loc[
(df["aa_text_scaledTextSize"] == 12) & (df.aa_end_x <= te.screen_width)
]
df.aa_input_tap.iloc[0]()
df = pd.DataFrame()
timesleep(2)
while df.empty:
df = te.get_df_window_dump(with_screenshot=True)
df = df.loc[
(df["aa_text_scaledTextSize"] == 14) & (df.aa_end_x <= te.screen_width)
].sort_values(by=["aa_start_y"])
df.aa_input_tap.iloc[0]()
df = pd.DataFrame()
timesleep(2)
while df.empty:
df = te.get_df_window_dump(with_screenshot=True)
df = df.loc[
(df["aa_accessibility_labelFor"] == -1)
& (df["aa_topMargin"] == 0)
& (df["aa_layout_gravity"] == -1)
& (df["aa_layout_left"] > 0)
& (df["aa_layout_left"] < 20)
].sort_values(by=["aa_start_x"], ascending=[False])
df.aa_input_tap.iloc[0]()
df = pd.DataFrame()
timesleep(2)
te.get_cmd_send_text_natural("myemailtest@gmail.com")()| index | aa_window_left | aa_window_top | aa_meta___name__ | aa_meta___hash__ | aa_id | aa_misc_transformation.alpha | aa_misc_transitionName | aa_layout_left | aa_layout_right | aa_layout_top | aa_layout_bottom | aa_layout_width | aa_layout_height | aa_layout_layoutDirection | aa_layout_layoutRtl | aa_layout_hasTransientState | aa_layout_baseline | aa_layoutParams | aa_y | aa_horizontalWeight | aa_verticalWeight | aa_type | aa_flags | aa_scrolling_scrollX | aa_scrolling_scrollY | aa_padding_paddingLeft | aa_padding_paddingRight | aa_padding_paddingTop | aa_padding_paddingBottom | aa_padding_userPaddingRight | aa_padding_userPaddingLeft | aa_padding_userPaddingBottom | aa_padding_userPaddingStart | aa_padding_userPaddingEnd | aa_measurement_minHeight | aa_measurement_minWidth | aa_measurement_measuredWidth | aa_measurement_measuredHeight | aa_drawing_elevation | aa_drawing_translationX | aa_drawing_translationY | aa_drawing_translationZ | aa_drawing_rotation | aa_drawing_rotationX | aa_drawing_rotationY | aa_drawing_scaleX | aa_drawing_scaleY | aa_drawing_pivotX | aa_drawing_pivotY | aa_drawing_clipBounds | aa_drawing_opaque | aa_drawing_alpha | aa_drawing_transitionAlpha | aa_drawing_shadow | aa_drawing_solidColor | aa_drawing_layerType | aa_drawing_willNotDraw | aa_drawing_hardwareAccelerated | aa_drawing_willNotCacheDrawing | aa_drawing_drawingCacheEnabled | aa_drawing_overlappingRendering | aa_drawing_outlineAmbientShadowColor | aa_drawing_outlineSpotShadowColor | aa_focus_hasFocus | aa_focus_isFocused | aa_focus_focusable | aa_focus_isFocusable | aa_focus_isFocusableInTouchMode | aa_misc_clickable | aa_misc_pressed | aa_misc_selected | aa_misc_touchMode | aa_misc_hovered | aa_misc_activated | aa_misc_visibility | aa_misc_fitsSystemWindows | aa_misc_filterTouchesWhenObscured | aa_misc_enabled | aa_misc_soundEffectsEnabled | aa_misc_hapticFeedbackEnabled | aa_theme | aa_meta___attrCount__ | aa_misc_scrollBarStyle | aa_text_textDirection | aa_text_textAlignment | aa_accessibility_labelFor | aa_accessibility_importantForAccessibility | aa_focus_descendantFocusability | aa_drawing_clipChildren | aa_drawing_clipToPadding | aa_drawing_childrenDrawingOrderEnabled | aa_drawing_persistentDrawingCache | aa_meta___childCount__ | aa_CHILD_META | aa_aa_hashcode_int | aa_bottomMargin | aa_endMargin | aa_height | aa_layout_alignWithParent | aa_layout_baselineAligned | aa_layout_baselineAlignedChildIndex | aa_layout_gravity | aa_layout_totalLength | aa_layout_useLargestChild | aa_layout_weight | aa_leftMargin | aa_measurement_baselineChildTop | aa_measurement_gravity | aa_measurement_measureAllChildren | aa_measurement_orientation | aa_measurement_totalLength | aa_meta___child__0 | aa_meta___child__1 | aa_meta___child__2 | aa_meta___child__3 | aa_meta___child__4 | aa_meta___child__5 | aa_meta___child__6 | aa_meta___child__7 | aa_meta___child__8 | aa_padding_foregroundPaddingBottom | aa_padding_foregroundPaddingLeft | aa_padding_foregroundPaddingRight | aa_padding_foregroundPaddingTop | aa_rightMargin | aa_startMargin | aa_text_curTextColor | aa_text_ellipsize | aa_text_gravity | aa_text_scaledTextSize | aa_text_selectionEnd | aa_text_selectionStart | aa_text_textSize | aa_text_typefaceStyle | aa_topMargin | aa_start_x | aa_start_y | aa_end_x | aa_end_y | aa_center_x | aa_center_y | aa_window_name | aa_hashcode_hex | aa_input_tap | aa_sendevent_mouse_click | aa_sendevent_tap | aa_mouse_action | aa_screenshot |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0 | 0.0 | com.android.internal.policy.DecorView | 192145870.0 | NO_ID | 0.0 | 0 | 1024 | 0 | 768 | 1024 | 768 | 0 | False | False | -1 | (3, 'android.view.WindowManager$LayoutParams', 4, | 0.0 | 0.0 | 0.0 | 1.0 | -2122252030.0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -2147483648 | -2147483648 | 0 | 0 | 1024 | 768 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 512.0 | 384.0 | False | 1.0 | 1.0 | False | 0 | 0 | True | True | False | False | True | -16777216 | -16777216 | False | False | 16 | False | False | False | False | False | True | False | False | 0 | False | False | True | True | True | (3, 'android.content.res.Resources$Theme', 4, 5365 | 0 | 0 | 1 | 1 | -1 | 1 | 262144 | True | True | False | 2 | 3 | None | nan | nan | nan | None | None | None | nan | nan | nan | None | nan | nan | nan | nan | None | nan | nan | None | None | None | None | None | None | None | None | None | nan | nan | nan | nan | nan | nan | nan | None | nan | nan | nan | nan | nan | nan | nan | -1 | -1 | -1 | -1 | -1 | -1 | a57b656 com.zhiliaoapp.musically/com.ss.android.ug | -1 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, duration=0.0) | (offset_x=0, offset_y=0, duration=0.0) | (offset_x=0, offset_y=0, action=0, event_multiply= | [] | ||
| 1 | None | nan | None | nan | None | nan | 0 | 1024 | 0 | 736 | 1024 | 736 | 0 | False | False | -1 | (3, 'android.widget.FrameLayout$LayoutParams', 4, | nan | nan | nan | nan | nan | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -2147483648 | -2147483648 | 0 | 0 | 1024 | 736 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 512.0 | 368.0 | False | 1.0 | 1.0 | False | 0 | 0 | True | True | False | False | True | -16777216 | -16777216 | False | False | 16 | False | False | False | False | False | True | False | False | 0 | True | False | True | True | True | (3, 'android.content.res.Resources$Theme', 4, 1907 | 0 | 0 | 1 | 1 | -1 | 0 | 131072 | True | True | False | 2 | 2 | ('meta:__child__0', 3, 'android.widget.LinearLayou | 45420503.0 | 32.0 | -2147483648.0 | None | None | None | nan | nan | nan | None | nan | nan | nan | nan | None | nan | nan | None | None | None | None | None | None | None | None | None | nan | nan | nan | nan | 0.0 | -2147483648.0 | nan | None | nan | nan | nan | nan | nan | nan | 0.0 | 0 | 0 | 1024 | 736 | 512 | 368 | a57b656 com.zhiliaoapp.musically/com.ss.android.ug | 2b50fd7 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, duration=0.0) | (offset_x=0, offset_y=0, duration=0.0) | (offset_x=0, offset_y=0, action=0, event_multiply= | [[[255 255 255] [255 255 255] [255 255 255] | ||
| 2 | None | nan | None | nan | None | nan | 0 | 1024 | 0 | 736 | 1024 | 736 | 0 | False | False | -1 | (3, 'android.widget.LinearLayout$LayoutParams', 4, | nan | nan | nan | nan | nan | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -2147483648 | -2147483648 | 0 | 0 | 1024 | 736 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 512.0 | 368.0 | False | 1.0 | 1.0 | False | 0 | 0 | True | True | False | False | True | -16777216 | -16777216 | False | False | 16 | False | False | False | False | False | True | False | False | 0 | False | False | True | True | True | (3, 'android.content.res.Resources$Theme', 4, 1907 | 0 | 0 | 1 | 1 | -1 | 0 | 131072 | True | True | False | 2 | 1 | ('meta:__child__0', 3, 'android.view.ViewStub', 4, | 99812585.0 | 0.0 | -2147483648.0 | None | None | None | nan | -1.0 | nan | None | 0.0 | nan | nan | nan | None | nan | nan | None | (3, 'android.widget.FrameLayout', 4, 131089238, 5, | None | None | None | None | None | None | None | nan | nan | nan | nan | 0.0 | -2147483648.0 | nan | None | nan | nan | nan | nan | nan | nan | 0.0 | 0 | 0 | 0 | 0 | 0 | 0 | a57b656 com.zhiliaoapp.musically/com.ss.android.ug | 5f304e9 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, duration=0.0) | (offset_x=0, offset_y=0, duration=0.0) | (offset_x=0, offset_y=0, action=0, event_multiply= | [] | ||
| 3 | None | nan | None | nan | None | nan | 0 | 1024 | 0 | 736 | 1024 | 736 | 0 | False | False | -1 | (3, 'android.widget.FrameLayout$LayoutParams', 4, | nan | nan | nan | nan | nan | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -2147483648 | -2147483648 | 0 | 0 | 1024 | 736 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1.0 | 512.0 | 368.0 | False | 1.0 | 1.0 | False | 0 | 0 | True | True | False | False | True | -16777216 | -16777216 | False | False | 16 | False | False | False | False | False | True | False | False | 0 | False | False | True | True | True | (3, 'android.content.res.Resources$Theme', 4, 1907 | 0 | 0 | 1 | 1 | -1 | 0 | 131072 | True | True | False | 2 | 2 | ('meta:__child__0', 3, 'X.jTV', 4, 211298673, 5, ' | 211298673.0 | 0.0 | -2147483648.0 | None | None | None | nan | nan | nan | None | nan | nan | nan | nan | None | nan | nan | None | None | None | None | None | None | None | None | None | nan | nan | nan | nan | 0.0 | -2147483648.0 | nan | None | nan | nan | nan | nan | nan | nan | 0.0 | 0 | 0 | 1024 | 736 | 512 | 368 | a57b656 com.zhiliaoapp.musically/com.ss.android.ug | c982971 | (offset_x=0, offset_y=0) | (offset_x=0, offset_y=0, duration=0.0) | (offset_x=0, offset_y=0, duration=0.0) | (offset_x=0, offset_y=0, action=0, event_multiply= | [[[255 255 255] [255 255 255] [255 255 255] |
If you decide to add a screenshot to your DataFrame, you can perform a very fast color search (uses C++ vectors and structs for the results internally).
# Pass RGB colors and the name of the column
df.bb_search_for_colors([[255, 255, 255], [0, 0, 0]], result_column="white_and_black")
# An new column will be added
df.white_and_black
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
iloc | index | white_and_black |
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0 | 0 | [{'x': 140, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 141, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 142, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 143, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 144, 'y': 65, 'count': 20 |
1 | 1 | [{'x': 140, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 141, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 142, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 143, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 144, 'y': 65, 'count': 20 |
2 | 2 | [{'x': 140, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 141, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 142, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 143, 'y': 65, 'count': 20148, 'r': 255, 'g': 255, 'b': 255}, {'x': 144, 'y': 65, 'count': 20 |
3 | 3 | [{'x': 128, 'y': 100, 'count': 4811, 'r': 255, 'g': 255, 'b': 255}, {'x': 129, 'y': 100, 'count': 4811, 'r': 255, 'g': 255, 'b': 255}, {'x': 130, 'y': 100, 'count': 4811, 'r': 255, 'g': 255, 'b': 255}, {'x': 131, 'y': 100, 'count': 4811, 'r': 255, 'g': 255, 'b': 255}, {'x': 132, 'y': 100, 'count': 4 |
4 | 4 | [{'x': 128, 'y': 100, 'count': 665, 'r': 255, 'g': 255, 'b': 255}, {'x': 129, 'y': 100, 'count': 665, 'r': 255, 'g': 255, 'b': 255}, {'x': 130, 'y': 100, 'count': 665, 'r': 255, 'g': 255, 'b': 255}, {'x': 131, 'y': 100, 'count': 665, 'r': 255, 'g': 255, 'b': 255}, {'x': 132, 'y': 100, 'count': 665, |
5 | 5 | [{'x': 248, 'y': 100, 'count': 1717, 'r': 255, 'g': 255, 'b': 255}, {'x': 249, 'y': 100, 'count': 1717, 'r': 255, 'g': 255, 'b': 255}, {'x': 250, 'y': 100, 'count': 1717, 'r': 255, 'g': 255, 'b': 255}, {'x': 251, 'y': 100, 'count': 1717, 'r': 255, 'g': 255, 'b': 255}, {'x': 252, 'y': 100, 'count': 1 |
6 | 6 | [{'x': 470, 'y': 100, 'count': 830, 'r': 255, 'g': 255, 'b': 255}, {'x': 471, 'y': 100, 'count': 830, 'r': 255, 'g': 255, 'b': 255}, {'x': 472, 'y': 100, 'count': 830, 'r': 255, 'g': 255, 'b': 255}, {'x': 473, 'y': 100, 'count': 830, 'r': 255, 'g': 255, 'b': 255}, {'x': 474, 'y': 100, 'count': 830, |
7 | 7 | [{'x': 117, 'y': 412, 'count': 311, 'r': 255, 'g': 255, 'b': 255}, {'x': 118, 'y': 412, 'count': 311, 'r': 255, 'g': 255, 'b': 255}, {'x': 119, 'y': 412, 'count': 311, 'r': 255, 'g': 255, 'b': 255}, {'x': 118, 'y': 413, 'count': 311, 'r': 255, 'g': 255, 'b': 255}, {'x': 119, 'y': 413, 'count': 311, |
8 | 8 | [{'x': 339, 'y': 412, 'count': 402, 'r': 255, 'g': 255, 'b': 255}, {'x': 340, 'y': 412, 'count': 402, 'r': 255, 'g': 255, 'b': 255}, {'x': 341, 'y': 412, 'count': 402, 'r': 255, 'g': 255, 'b': 255}, {'x': 340, 'y': 413, 'count': 402, 'r': 255, 'g': 255, 'b': 255}, {'x': 341, 'y': 413, 'count': 402, |
9 | 9 | [{'x': 568, 'y': 497, 'count': 62, 'r': 255, 'g': 255, 'b': 255}, {'x': 569, 'y': 497, 'count': 62, 'r': 255, 'g': 255, 'b': 255}, {'x': 580, 'y': 497, 'count': 62, 'r': 255, 'g': 255, 'b': 255}, {'x': 589, 'y': 497, 'count': 62, 'r': 255, 'g': 255, 'b': 255}, {'x': 597, 'y': 497, 'count': 62, 'r': |
10 | 10 | [{'x': 151, 'y': 729, 'count': 197, 'r': 255, 'g': 255, 'b': 255}, {'x': 152, 'y': 729, 'count': 197, 'r': 255, 'g': 255, 'b': 255}, {'x': 85, 'y': 730, 'count': 197, 'r': 255, 'g': 255, 'b': 255}, {'x': 86, 'y': 730, 'count': 197, 'r': 255, 'g': 255, 'b': 255}, {'x': 149, 'y': 730, 'count': 197, 'r |
11 | 11 | [{'x': 362, 'y': 638, 'count': 627, 'r': 255, 'g': 255, 'b': 255}, {'x': 363, 'y': 638, 'count': 627, 'r': 255, 'g': 255, 'b': 255}, {'x': 364, 'y': 638, 'count': 627, 'r': 255, 'g': 255, 'b': 255}, {'x': 365, 'y': 638, 'count': 627, 'r': 255, 'g': 255, 'b': 255}, {'x': 366, 'y': 638, 'count': 627, |
12 | 12 | [{'x': 140, 'y': 65, 'count': 17610, 'r': 255, 'g': 255, 'b': 255}, {'x': 141, 'y': 65, 'count': 17610, 'r': 255, 'g': 255, 'b': 255}, {'x': 142, 'y': 65, 'count': 17610, 'r': 255, 'g': 255, 'b': 255}, {'x': 143, 'y': 65, 'count': 17610, 'r': 255, 'g': 255, 'b': 255}, {'x': 144, 'y': 65, 'count': 17 |
13 | 13 | [{'x': 136, 'y': 77, 'count': 100, 'r': 255, 'g': 255, 'b': 255}, {'x': 141, 'y': 77, 'count': 100, 'r': 255, 'g': 255, 'b': 255}, {'x': 142, 'y': 77, 'count': 100, 'r': 255, 'g': 255, 'b': 255}, {'x': 143, 'y': 77, 'count': 100, 'r': 255, 'g': 255, 'b': 255}, {'x': 144, 'y': 77, 'count': 100, 'r': |
14 | 14 | [{'x': 150, 'y': 65, 'count': 15852, 'r': 255, 'g': 255, 'b': 255}, {'x': 151, 'y': 65, 'count': 15852, 'r': 255, 'g': 255, 'b': 255}, {'x': 152, 'y': 65, 'count': 15852, 'r': 255, 'g': 255, 'b': 255}, {'x': 153, 'y': 65, 'count': 15852, 'r': 255, 'g': 255, 'b': 255}, {'x': 154, 'y': 65, 'count': 15 |
15 | 15 | [{'x': 568, 'y': 77, 'count': 177, 'r': 255, 'g': 255, 'b': 255}, {'x': 569, 'y': 77, 'count': 177, 'r': 255, 'g': 255, 'b': 255}, {'x': 570, 'y': 77, 'count': 177, 'r': 255, 'g': 255, 'b': 255}, {'x': 571, 'y': 77, 'count': 177, 'r': 255, 'g': 255, 'b': 255}, {'x': 572, 'y': 77, 'count': 177, 'r': |
16 | 16 | [] |
17 | 17 | [] |
18 | 18 | [] |
19 | 19 | [{'x': 344, 'y': 1129, 'count': 250, 'r': 0, 'g': 0, 'b': 0}, {'x': 333, 'y': 1130, 'count': 250, 'r': 0, 'g': 0, 'b': 0}, {'x': 334, 'y': 1130, 'count': 250, 'r': 0, 'g': 0, 'b': 0}, {'x': 331, 'y': 1131, 'count': 250, 'r': 0, 'g': 0, 'b': 0}, {'x': 334, 'y': 1131, 'count': 250, 'r': 0, 'g': 0, 'b' |
20 | 20 | [] |
21 | 21 | [{'x': 599, 'y': 1147, 'count': 1, 'r': 255, 'g': 255, 'b': 255}] |
22 | 22 | []HDBSCAN of the color search results is also possible. It should not be done with a large number of colors. Even though it is a fast C++ implementation, it can take quite some time on an emulator.
# Use your favorite method to get the elements
df = te.get_df_uiautomator2(with_screenshot=True)
# Search for colors
df.bb_search_for_colors([[0, 0, 0]], result_column="black_color")
# Clustering the color search results from the first element found
clustered = df["black_color"][:1].s_cluster_values(min_points=50, min_cluster_size=50)
# Displaying the first 20 results
clustered[:20]
-----------------------------------------------------------------------------------------------------------------------
iloc | index | original_data | label | membership_probability | outlier_score | outlier_id | dfiloc |
-----------------------------------------------------------------------------------------------------------------------
0 | 0 | [106.0, 388.0] | 2 | 0.05719095841793657 | 0.0 | 424 | 0 |
1 | 1 | [107.0, 388.0] | 2 | 0.10247253214424938 | 0.0 | 425 | 0 |
2 | 2 | [108.0, 388.0] | 2 | 0.10247253214424938 | 0.0 | 426 | 0 |
3 | 3 | [109.0, 388.0] | 2 | 0.10247253214424938 | 0.0 | 427 | 0 |
4 | 4 | [110.0, 388.0] | 2 | 0.10247253214424938 | 0.0 | 428 | 0 |
5 | 5 | [111.0, 388.0] | 2 | 0.1501634144012026 | 0.0 | 429 | 0 |
6 | 6 | [112.0, 388.0] | 2 | 0.1501634144012026 | 0.0 | 430 | 0 |
7 | 7 | [113.0, 388.0] | 2 | 0.1501634144012026 | 0.0 | 431 | 0 |
8 | 8 | [114.0, 388.0] | 2 | 0.1501634144012026 | 0.0 | 432 | 0 |
9 | 9 | [115.0, 388.0] | 2 | 0.1501634144012026 | 0.0 | 433 | 0 |
10 | 10 | [116.0, 388.0] | 2 | 0.1501634144012026 | 0.0 | 434 | 0 |
11 | 11 | [117.0, 388.0] | 2 | 0.1501634144012026 | 0.0 | 435 | 0 |
12 | 12 | [118.0, 388.0] | 2 | 0.1501634144012026 | 0.0 | 436 | 0 |
13 | 13 | [119.0, 388.0] | 2 | 0.1501634144012026 | 0.0 | 437 | 0 |
14 | 14 | [120.0, 388.0] | 2 | 0.1501634144012026 | 0.0 | 438 | 0 |
15 | 15 | [121.0, 388.0] | 2 | 0.1501634144012026 | 0.0 | 439 | 0 |
16 | 16 | [122.0, 388.0] | 2 | 0.1501634144012026 | 0.0 | 440 | 0 |
17 | 17 | [123.0, 388.0] | 2 | 0.1501634144012026 | 0.0 | 441 | 0 |
18 | 18 | [124.0, 388.0] | 2 | 0.1501634144012026 | 0.0 | 442 | 0 |
19 | 19 | [125.0, 388.0] | 2 | 0.1501634144012026 | 0.0 | 443 | 0 |To interact with elements—such as clicking, tapping, scrolling, writing text, or pressing keys—you can create instances of a collection of input classes. The advantage is that these instances are callable and can be reused, which saves overhead.
Most of these classes are implemented in Cython using low-overhead cdef classes for maximum performance.
cmd_sendkeys = te.get_cmd_sendkeys(
text="Hi there my friend",
min_press=1,
max_press=4
)
# Just call the instance to execute the command
cmd_sendkeys()cmd_sendevent_tap = te.get_cmd_sendevent_tap(100, 100)
cmd_sendevent_tap()
cmd_sendevent_click = te.get_cmd_sendevent_click(100, 100)
cmd_sendevent_click()cmd_input_tap = te.get_cmd_input_tap(100, 100)
cmd_input_tap()cmd_mouse_action = te.get_cmd_mouse_action(500, 500)
# The __call__ method accepts the following arguments:
cmd_mouse_action(
int offset_x=0,
int offset_y=0,
int action=0,
int event_multiply=1,
int natural_movement=1,
int use_every_n_element=10,
int min_x_variation=0,
int max_x_variation=0,
int min_y_variation=0,
int max_y_variation=0,
int sleep_time=0,
int debug=0,
int print_device_info=0,
)
# Check out the documentation for details on each parameter:
# https://github.com/hansalemaos/mouse_sendevent_android
# Sends the whole text
cmd_send_text = te.get_cmd_send_text("Hi there m'y frönd")
cmd_send_text()
# sends each text
cmd_send_text_natural = te.get_cmd_send_text_natural("Hi there my frönd")
cmd_send_text_natural()
# sends each letter of the text - unicode - supports á, é, í, ó, ú, ý, Á, É, Í, Ó, Ú, Ý, ç, Ç, â, ê, î, ô, û, Â, Ê, Î, Ô, Û, ã, ñ, õ, Ã, Ñ, Õ, ß, ẞ, ä, ë, ï, ö, ü, ÿ, Ä, Ë, Ï, Ö, Ü, Ÿ, à, è, ì, ò, ù, À, È, Ì, Ò, Ù
# Uses Androids input keycombination - contribs for more letters are welcome!
cmd_send_text_natural_with_unicode = cyandro.get_cmd_send_text_unicode("Hi there my frönd")
cmd_send_text_natural_with_unicode()
cmd_send_text_natural_with_unicode_each_letter = cyandro.get_cmd_send_text_unicode_natural("Hi there my frönd")
cmd_send_text_natural_with_unicode_each_letter()te.KeyCodes.long_press_KEYCODE_BACKSLASH()
te.KeyCodes.short_press_KEYCODE_BACKSLASH()te.adb_keyboard.download_adbkeyboard_apk()
te.adb_keyboard.install_adbkeyboard_apk()
te.adb_keyboard.activate_adb_keyboard()
te.adb_keyboard.send_unicode_text_with_delay(
text="Hello myç friend",
delay_range=(0.05, 0.3),
)
te.adb_keyboard.send_unicode_text("Hello myç friend")
te.disable_adb_keyboard()Sometimes, you need to do more than just parse and interact with elements on the screen—you need to execute commands directly on the device.
CyAndroEmu provides an interactive shell that doesn’t rely on either the Python subprocess module or Python threads.
This shell is implemented 100% in C++, runs in nogil mode, and offers excellent reaction times, making it perfect for high-performance automation tasks.
- Real-Time Interaction: Fast command execution with immediate feedback.
- Root Access Control: Easily switch between normal and superuser modes.
- DataFrame Integration: Collect system data in a structured format for automation workflows.
- Optimized for Performance: Runs in C++ with nogil mode for high responsiveness.
myshell = te.open_shell(
buffer_size=40960, # The max size of the C++ vector—works like collections.deque (ring buffer)
exit_command=b"exit", # The command to exit the shell, required by the C++ destructor.
# Executed automatically when 'myshell' goes out of scope.
print_stdout=False, # Prints stdout to the screen—useful for debugging purposes
print_stderr=False # Prints stderr to the screen—useful for debugging purposes
)Once the interactive shell is running, you can execute commands directly and capture their output effortlessly.
>>> stdout, stderr = myshell.write_and_wait("ls -lZ")
# stdout and stderr contain the command’s output in bytes.
# Use .decode() to convert the output to a human-readable string.
>>> print(stdout.decode())
total 48
-rw-r--r-- 1 root root u:object_r:app_data_file:s0 12264 2025-02-06 20:29 None
drwxr-x--- 16 root root u:object_r:app_data_file:s0 4096 2025-02-04 22:57 cyandroemu
-rw-r--r-- 1 root root u:object_r:app_data_file:s0 0 2025-02-02 01:40 pckldf.pkl
-rw-r----- 1 root root u:object_r:app_data_file:s0 17446 2025-02-04 22:51 pyshell.py
drwx------ 5 u0_a286 u0_a286 u:object_r:app_data_file:s0:c30,c257,c512,c768 4096 2025-02-01 23:45 tessdata
drwxr-xr-x 5 root root u:object_r:app_data_file:s0 4096 2025-02-02 00:13 tessdata_best
drwxr-xr-x 5 root root u:object_r:app_data_file:s0 4096 2025-02-02 00:11 tessdata_fast# To gain root access within the shell:
>>> myshell.su()
# This elevates the shell session, allowing privileged commands to be executed seamlessly.- get_df_* → Returns data as a pandas DataFrame for easy analysis.
- sh_* → Executes shell commands and usually returns a tuple[bytes, bytes] containing stdout and stderr, but not always—some commands may return different structures.
# This command checks the current user, and the result is returned in byte format (b'root').
>>> myshell.sh_whoami()
b'root'>>> buildprops=myshell.get_df_build_props()
>>> print(buildprops)
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
iloc | index | aa_file | aa_line | aa_line_content |
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0 | 0 | '/data/adb/modules/make_writeable/system.prop' | 0 | b'# This file will be read by resetprop' |
1 | 1 | '/data/adb/modules/make_writeable/system.prop' | 1 | b'# Example: Change dpi' |
2 | 2 | '/data/adb/modules/make_writeable/system.prop' | 2 | b'# ro.sf.lcd_density=320' |
3 | 3 | '/data/adb/modules/make_writeable/module.prop' | 0 | b'id=make_writeable' |
4 | 4 | '/data/adb/modules/make_writeable/module.prop' | 1 | b'name=Make Writeable' |
5 | 5 | '/data/adb/modules/make_writeable/module.prop' | 2 | b'version=1.0' |
6 | 6 | '/data/adb/modules/make_writeable/module.prop' | 3 | b'versionCode=01' |
7 | 7 | '/data/adb/modules/make_writeable/module.prop' | 4 | b'author=hansalemao'
...CyAndroEmu provides a wide range of shell methods to control Android devices efficiently. These methods are optimized for speed and reliability.
sh_save_sed_replace(self, file_path, string2replace, replacement, timeout=1000)
sh_svc_enable_wifi(self, timeout=10)
sh_svc_disable_wifi(self, timeout=10)
sh_trim_cache(self, timeout=10)
sh_force_open_app(self, package_name, sleep_time, timeout=3)
sh_get_main_activity(self, package_name, timeout=3)
sh_svc_power_shutdown(self, timeout=3)
sh_svc_power_reboot(self, timeout=3)
sh_dumpsys_dropbox(self, timeout=3)
sh_set_new_launcher(self, package_name, timeout=3)
sh_tar_folder(self, src, dst, timeout=1000000)
sh_extract_tar_zip(self, src_file, dst_folder, timeout=1000000)
sh_get_user_rotation(self, timeout=10)
sh_copy_dir_recursive(self, src, dst, timeout=1000)
sh_backup_file(self, src, timeout=1000)
sh_remove_folder(self, folder, timeout=1000)
sh_get_pid_of_shell(self, int64_t timeout=3)
sh_whoami(self, int64_t timeout=10)
sh_dumpsys_package(self, package, timeout=1000, bint convert_to_dict=True)
sh_get_all_wanted_permissions_from_package(self, package, timeout=1000)
sh_grant_permission(self, package, permission, timeout=10)
sh_grant_permission(self, package, permission, timeout=10)
sh_grant_all_wanted_permissions(self, package, timeout=1000)
sh_revoke_all_wanted_permissions(self, package, timeout=1000)
sh_parse_whole_dumpsys_to_dict(self, timeout=100,convert_to_dict=False)
sh_parse_dumpsys_to_dict(self, subcmd, timeout=100,convert_to_dict=False)
sh_get_available_keyboards(self, timeout=10)
sh_get_active_keyboard(self, timeout=10)
sh_get_all_information_about_all_keyboards(self, timeout=10,convert_to_dict=False)
sh_enable_keyboard(self, keyboard, timeout=10)
sh_disable_keyboard(self, keyboard, timeout=10)
sh_is_keyboard_shown(self, timeout=10)
sh_set_keyboard(self, keyboard, timeout=10)
sh_show_touches(self, timeout=10)
sh_dont_show_touches(self, timeout=10)
sh_show_pointer_location(self, timeout=10)
sh_dont_show_pointer_location(self, timeout=10)
sh_input_swipe(self, x1, y1, x2, y2, duration, timeout=10)
sh_input_tap(self, x, y, timeout=10)
sh_clear_file_content(self, file_path, timeout=10)
sh_makedirs(self, folder, timeout=10)
sh_touch(self, file_path, timeout=10)
sh_mv(self, src, dst, timeout=10)
sh_open_accessibility_settings(self, timeout=10)
sh_open_advanced_memory_protection_settings(self, timeout=10)
sh_open_airplane_mode_settings(self, timeout=10)
sh_open_all_apps_notification_settings(self, timeout=10)
sh_open_apn_settings(self, timeout=10)
sh_open_application_details_settings(self, timeout=10)
sh_open_application_development_settings(self, timeout=10)
sh_open_application_settings(self, timeout=10)
sh_open_app_locale_settings(self, timeout=10)
sh_open_app_notification_bubble_settings(self, timeout=10)
sh_open_app_notification_settings(self, timeout=10)
sh_open_app_open_by_default_settings(self, timeout=10)
sh_open_app_search_settings(self, timeout=10)
sh_open_app_usage_settings(self, timeout=10)
sh_open_automatic_zen_rule_settings(self, timeout=10)
sh_open_auto_rotate_settings(self, timeout=10)
sh_open_battery_saver_settings(self, timeout=10)
sh_open_bluetooth_settings(self, timeout=10)
sh_open_captioning_settings(self, timeout=10)
sh_open_cast_settings(self, timeout=10)
sh_open_channel_notification_settings(self, timeout=10)
sh_open_condition_provider_settings(self, timeout=10)
sh_open_data_roaming_settings(self, timeout=10)
sh_open_data_usage_settings(self, timeout=10)
sh_open_date_settings(self, timeout=10)
sh_open_device_info_settings(self, timeout=10)
sh_open_display_settings(self, timeout=10)
sh_open_dream_settings(self, timeout=10)
sh_open_hard_keyboard_settings(self, timeout=10)
sh_open_home_settings(self, timeout=10)
sh_open_ignore_background_data_restrictions_settings(self, timeout=10)
sh_open_ignore_battery_optimization_settings(self, timeout=10)
sh_open_input_method_settings(self, timeout=10)
sh_open_input_method_subtype_settings(self, timeout=10)
sh_open_internal_storage_settings(self, timeout=10)
sh_open_locale_settings(self, timeout=10)
sh_open_location_source_settings(self, timeout=10)
sh_open_manage_all_applications_settings(self, timeout=10)
sh_open_manage_all_sim_profiles_settings(self, timeout=10)
sh_open_manage_applications_settings(self, timeout=10)
sh_open_manage_default_apps_settings(self, timeout=10)
sh_open_manage_supervisor_restricted_setting(self, timeout=10)
sh_open_manage_write_settings(self, timeout=10)
sh_open_memory_card_settings(self, timeout=10)
sh_open_network_operator_settings(self, timeout=10)
sh_open_nfcsharing_settings(self, timeout=10)
sh_open_nfc_payment_settings(self, timeout=10)
sh_open_nfc_settings(self, timeout=10)
sh_open_night_display_settings(self, timeout=10)
sh_open_notification_assistant_settings(self, timeout=10)
sh_open_notification_listener_detail_settings(self, timeout=10)
sh_open_notification_listener_settings(self, timeout=10)
sh_open_notification_policy_access_settings(self, timeout=10)
sh_open_print_settings(self, timeout=10)
sh_open_privacy_settings(self, timeout=10)
sh_open_quick_access_wallet_settings(self, timeout=10)
sh_open_quick_launch_settings(self, timeout=10)
sh_open_regional_preferences_settings(self, timeout=10)
sh_open_satellite_setting(self, timeout=10)
sh_open_search_settings(self, timeout=10)
sh_open_security_settings(self, timeout=10)
sh_open_settings(self, timeout=10)
sh_open_settings(self, timeout=10)
sh_open_sound_settings(self, timeout=10)
sh_open_storage_volume_access_settings(self, timeout=10)
sh_open_sync_settings(self, timeout=10)
sh_open_usage_access_settings(self, timeout=10)
sh_open_user_dictionary_settings(self, timeout=10)
sh_open_voice_input_settings(self, timeout=10)
sh_open_vpn_settings(self, timeout=10)
sh_open_vr_listener_settings(self, timeout=10)
sh_open_webview_settings(self, timeout=10)
sh_open_wifi_ip_settings(self, timeout=10)
sh_open_wifi_settings(self, timeout=10)
sh_open_wireless_settings(self, timeout=10)
sh_open_zen_mode_priority_settings(self, timeout=10)
sh_open_developer_settings(self, timeout=10)
sh_rescan_media_folder(self, folder, timeout=10)
sh_rescan_media_file(self, file_path, timeout=10)
sh_dump_process_memory_to_sdcard(self, pid, timeout=100000)
sh_pm_clear(self, package, timeout=10)
sh_wm_change_size(self, width, height, timeout=10)
sh_wm_reset_size(self, timeout=10)
sh_wm_get_density(self, timeout=10)
sh_wm_change_density(self, density, timeout=10)
sh_wm_reset_density(self, timeout=10)
sh_am_screen_compat_on(self, package, timeout=10)
sh_am_screen_compat_off(self, package, timeout=10)
sh_enable_notifications(self, timeout=10)
sh_disable_notifications(self, timeout=10)
sh_still_image_camera(self, timeout=10)
sh_disable_network_interface(self, nic, timeout=10)
sh_enable_network_interface(self, nic, timeout=10)
sh_get_linux_version(self, timeout=10)
sh_expand_notifications(self, timeout=10)
sh_expand_settings(self, timeout=10)
sh_list_permission_groups(self, timeout=10)
sh_input_dpad_tap(self, x, y, timeout=10)
sh_input_keyboard_tap(self, x, y, timeout=10)
sh_input_mouse_tap(self, x, y, timeout=10)
sh_input_touchpad_tap(self, x, y, timeout=10)
sh_input_gamepad_tap(self, x, y, timeout=10)
sh_input_touchnavigation_tap(self, x, y, timeout=10)
sh_input_joystick_tap(self, x, y, timeout=10)
sh_input_touchscreen_tap(self, x, y, timeout=10)
sh_input_stylus_tap(self, x, y, timeout=10)
sh_input_trackball_tap(self, x, y, timeout=10)
sh_input_dpad_swipe(self, x1, y1, x2, y2, duration, timeout=10)
sh_input_dpad_draganddrop(self, x1, y1, x2, y2, duration, timeout=10)
sh_input_dpad_roll(self, x, y, timeout=10)
sh_input_keyboard_swipe(self, x1, y1, x2, y2, duration, timeout=10)
sh_input_keyboard_draganddrop(self, x1, y1, x2, y2, duration, timeout=10)
sh_input_keyboard_roll(self, x, y, timeout=10)
sh_input_mouse_swipe(self, x1, y1, x2, y2, duration, timeout=10)
sh_input_mouse_draganddrop(self, x1, y1, x2, y2, duration, timeout=10)
sh_input_mouse_roll(self, x, y, timeout=10)
sh_input_touchpad_swipe(self, x1, y1, x2, y2, duration, timeout=10)
sh_input_touchpad_draganddrop(self, x1, y1, x2, y2, duration, timeout=10)
sh_input_touchpad_roll(self, x, y, timeout=10)
sh_input_gamepad_swipe(self, x1, y1, x2, y2, duration, timeout=10)
sh_input_gamepad_draganddrop(self, x1, y1, x2, y2, duration, timeout=10)
sh_input_gamepad_roll(self, x, y, timeout=10)
sh_input_touchnavigation_swipe(self, x1, y1, x2, y2, duration, timeout=10)
sh_input_touchnavigation_roll(self, x, y, timeout=10)
sh_input_joystick_swipe(self, x1, y1, x2, y2, duration, timeout=10)
sh_input_joystick_draganddrop(self, x1, y1, x2, y2, duration, timeout=10)
sh_input_joystick_roll(self, x, y, timeout=10)
sh_input_touchscreen_swipe(self, x1, y1, x2, y2, duration, timeout=10)
sh_input_touchscreen_draganddrop(self, x1, y1, x2, y2, duration, timeout=10)
sh_input_touchscreen_roll(self, x, y, timeout=10)
sh_input_stylus_swipe(self, x1, y1, x2, y2, duration, timeout=10)
sh_input_stylus_draganddrop(self, x1, y1, x2, y2, duration, timeout=10)
sh_input_stylus_roll(self, x, y, timeout=10)
sh_input_trackball_swipe(self, x1, y1, x2, y2, duration, timeout=10)
sh_input_trackball_draganddrop(self, x1, y1, x2, y2, duration, timeout=10)
sh_input_trackball_roll(self, x, y, timeout=10)
sh_open_url(self, url, timeout=10)
sh_get_bios_information(self, timeout=10)
sh_printenv(self, timeout=10)
sh_freeze_proc(self, pid, timeout=10)
sh_unfreeze_proc(self, pid, timeout=10)
sh_show_fragments_on_screen_enable(self, timeout=10)
sh_show_fragments_on_screen_disable(self, timeout=10)
sh_read_write_remount(self, methods, timeout=100)Some shell methods are designed to return data as pandas DataFrames for easy analysis and manipulation.
get_df_files_with_context_printf(self, object folders, int64_t max_depth=1, int64_t timeout=10)
get_df_build_props(self, int64_t timeout=10)
get_df_files_with_ending(self, object folders, object endings, int64_t max_depth=10000, int64_t timeout=10)
get_df_top_procs(self, timeout=1000)
get_df_users(self, start=0, end=2000, timeout=10000)
get_df_groups_of_user(self, start=0, end=2000, timeout=10000)
get_df_netstat_tlnp(self, timeout=100)
get_df_mounts(self, timeout=100)
get_df_ps_el(self, timeout=1000)
get_df_packages(self, timeout=10)
get_df_netstat_connections_of_apps(self, resolve_names=True, timeout=10)
get_df_lsmod(self, timeout=1000)
get_df_lsof(self, timeout=1000000)- Add template matching using https://www.fftw.org/ without other dependencies, something like needlefinder but without scipy
- Add more useful commands to the interactive shell
- Support for Accelerated Hierarchical Density Clustering in C++
See the open issues for a full list of proposed features (and known issues).
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature) - Commit your Changes (
git commit -m 'Add some AmazingFeature') - Push to the Branch (
git push origin feature/AmazingFeature) - Open a Pull Request
Distributed under the MIT License. See LICENSE for more information.
If you’re interested in private classes with me to learn strategies for automating anything, or if you have an automation project (within legal boundaries!) that you’d like to bring to life, feel free to contact me. I speak German, English, and Portuguese fluently, and Spanish proficiently.