Skip to content
A simple, cross-platform GUI automation library for Python and Rust.
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
autopy 1.1.1 Sep 27, 2018
docs Add documentation for space key constant Sep 3, 2018
examples readme update (#26) Oct 14, 2018
scripts Add support for automated wheel packaging / PyPI deploys (#19) Sep 14, 2018
src Update to latest pyo3 Sep 19, 2018
.gitignore Initial commit Apr 3, 2018
.travis.yml Ensure importing module works via cibuildwheel Sep 19, 2018
CHANGELOG.md 1.1.1 Sep 27, 2018
Cargo.toml Update autopilot to 0.1.8 Sep 27, 2018
LICENSE Add license Apr 20, 2018
MANIFEST.in
Makefile Add build scripts May 1, 2018
README.md readme update (#26) Oct 14, 2018
appveyor.yml Add support for automated wheel packaging / PyPI deploys (#19) Sep 14, 2018
requirements.txt Initial commit Apr 3, 2018
rustfmt.toml Initial commit Apr 3, 2018
setup.py 1.1.0 Sep 19, 2018

README.md

PyPI version

AutoPy Introduction and Tutorial

Introduction

AutoPy is a simple, cross-platform GUI automation library for Python. It includes functions for controlling the keyboard and mouse, finding colors and bitmaps on-screen, and displaying alerts.

Currently supported on macOS, Windows, and X11 with the XTest extension.

Getting Started

Requirements

  • Python 2.7, or Python 3.5 and up.
  • Rust 1.23.0-nightly 2018-05-01 or later (unless using a binary wheel distribution).
  • macOS 10.6 and up.
  • Windows 7 and up.
  • X11 with the XTest extension.

Installation

First, see if a binary wheel is available for your machine by running:

$ pip install -U autopy

If that fails, install rustup and then run:

$ rustup default nightly
$ pip install -U setuptools-rust
$ pip install -U autopy

Another option is to compile from the latest source on the GitHub repository:

$ git clone git://github.com/autopilot-rs/autopy-rs.git
$ cd autopy
$ rustup default nightly
$ pip install -r requirements.txt
$ python setup.py build
# python setup.py install

Additional instructions for installing from source on Windows are available here.

Hello World

The following is the source for a "hello world" script in autopy. Running this code will cause an alert dialog to appear on every major platform:

import autopy
def hello_world():
    autopy.alert.alert("Hello, world")
hello_world()

Cross platform alerts

Tutorials

Controlling the Mouse

AutoPy includes a number of functions for controlling the mouse. For a full list, consult the API Reference. E.g., to immediately "teleport" the mouse to the top left corner of the screen:

>>> import autopy
>>> autopy.mouse.move(1, 1)

To move the mouse a bit more realistically, we could use:

>>> import autopy
>>> autopy.mouse.smooth_move(1, 1)

Even better, we could write our own function to move the mouse across the screen as a sine wave:

import autopy
import math
import time
import random
import sys

TWO_PI = math.pi * 2.0


def sine_mouse_wave():
    """
    Moves the mouse in a sine wave from the left edge of
    the screen to the right.
    """
    width, height = autopy.screen.size()
    height /= 2
    height -= 10  # Stay in the screen bounds.

    for x in range(int(width)):
        y = int(height * math.sin((TWO_PI * x) / width) + height)
        autopy.mouse.move(x, y)
        time.sleep(random.uniform(0.001, 0.003))


sine_mouse_wave()

Demonstration video

Working with Bitmaps

All of autopy's bitmap routines can be found in the module autopy.bitmap. A useful way to explore autopy is to use Python's built-in help() function, for example in help(autopy.bitmap.Bitmap). AutoPy's functions are documented with descriptive docstrings, so this should show a nice overview.

>>> import autopy
>>> autopy.bitmap.capture_screen()
<Bitmap object at 0x12278>

This takes a screenshot of the main screen, copies it to a bitmap, displays its memory address, and then immediately destroys it. Let's do something more useful, like look at its pixel data:

>>> import autopy
>>> autopy.bitmap.capture_screen().get_color(1, 1)
15921906

AutoPy uses a coordinate system with its origin starting at the top-left, so this should return the color of pixel at the top-left corner of the screen. The number shown looks a bit unrecognizable, but we can format it with Python's built-in hex function:

>>> import autopy
>>> hex(autopy.bitmap.capture_screen().get_color(1, 1))
'0xF2F2F2'

Alternatively, we can use:

>>> import autopy
>>> autopy.color.hex_to_rgb(autopy.screen.get_color(1, 1))
(242, 242, 242)

which converts that hex value to a tuple of (r, g, b) values. (Note that autopy.screen.get_color(), used here, is merely a more convenient and efficient version of autopy.bitmap.capture_screen().get_color().)

To save the screen capture to a file, we can use:

>>> import autopy
>>> autopy.bitmap.capture_screen().save('screengrab.png')

The filetype is either parsed automatically from the filename, or given as an optional parameter. Currently only jpeg and png files are supported.

>>> import autopy
>>> autopy.bitmap.Bitmap.open('needle.png')
<Bitmap object at 0x1001d5378>

Aside from analyzing a bitmap's pixel data, the main use for loading a bitmap is finding it on the screen or inside another bitmap. For example, the following prints the coordinates of the first image found in a bitmap (scanned from left to right, top to bottom):

import autopy


def find_image_example():
    needle = autopy.bitmap.Bitmap.open('needle.png')
    haystack = autopy.bitmap.Bitmap.open('haystack.png')

    pos = haystack.find_bitmap(needle)
    if pos:
        print("Found needle at: %s" % str(pos))

find_image_example()

Projects using AutoPy

  • AutoPyDriverServer, AutoPy through WebDriver or a webdriver-compatible server.
  • guibot, A tool for GUI automation using a variety of computer vision and desktop control backends.
  • spynner, Programmatic web browsing module with AJAX support for Python.
  • SUMO, An open source, highly portable, microscopic and continuous road traffic simulation package designed to handle large road networks.

API Reference

Hope you enjoy using autopy! For a more in depth overview, see the API Reference.

You can’t perform that action at this time.