Website Automation Agent

An autonomous browser automation agent built using Python and Playwright that automatically navigates to a webpage, detects form elements intelligently, fills them with predefined values, captures screenshots, and logs every action performed.

The project demonstrates modular browser automation using semantic element detection instead of hardcoded screen coordinates or external AI APIs.

---

Features

• Opens a Chromium browser automatically
• Navigates to the target webpage
• Detects form fields using intelligent Playwright selectors
• Automatically fills form inputs
• Scrolls when required
• Captures screenshots during execution
• Maintains detailed execution logs
• Modular and extensible architecture
• Robust error handling

---

Project Structure

Website-Automation/
│
├── main.py                  # Entry point
├── agent.py                 # Automation agent logic
├── browser_controller.py    # Browser interaction layer
├── tools.py                 # Browser tool definitions
├── config.py                # Runtime configuration
├── requirements.txt         # Python dependencies
├── README.md
├── ARCHITECTURE.md
├── agent.log                # Execution logs
└── runs/                    # Screenshots from each execution

---

Architecture

                +----------------------+
                |      main.py         |
                +----------+-----------+
                           |
                           v
                +----------------------+
                | WebsiteAutomationAgent|
                +----------+-----------+
                           |
                           v
                +----------------------+
                | BrowserController    |
                +----------+-----------+
                           |
                           v
                +----------------------+
                |     Playwright       |
                +----------+-----------+
                           |
                           v
                +----------------------+
                | Chromium Browser     |
                +----------------------+

---

Workflow

1. Launch Chromium browser.
2. Navigate to the target URL.
3. Scroll until the required form is visible.
4. Detect form elements using semantic selectors.
5. Fill the required fields automatically.
6. Capture screenshots after every major step.
7. Store execution logs.
8. Close the browser.

---

Intelligent Element Detection

Instead of relying on fixed screen coordinates, the automation agent searches for elements using multiple Playwright locator strategies.

The detection order is:

• Label
• Placeholder
• Accessible Role
• Name Attribute
• ID Attribute
• CSS Selector
• Generic Input/Textarea fallback

This makes the automation more reliable even if minor UI changes occur.

---

Browser Tools

The project provides the following browser automation capabilities:

• Open Browser
• Navigate to URL
• Click
• Scroll
• Send Keys / Fill Text
• Take Screenshot

These tools are implemented through the BrowserController and are used by the automation agent to perform browser interactions.

---

Logging

Every important action is recorded inside agent.log, including:

• Browser launch
• Navigation
• Form detection
• Form filling
• Screenshot creation
• Errors (if any)

---

Screenshots

Screenshots are automatically stored inside:

runs/<timestamp>/

Example:

runs/
└── 20260623-013626/
    ├── 01-page_loaded.png
    ├── 02-form_visible.png
    ├── 03-name.png
    ├── 04-description.png
    └── 05-final.png

These screenshots provide visual verification that the automation completed successfully.

---

Installation

Clone the repository

git clone <repository-url>
cd Website-Automation

Install dependencies

pip install -r requirements.txt

Install Playwright browser binaries

python -m playwright install

---

Running the Project

Run normally

python main.py

Run in headless mode

python main.py --headless

Browser-only test (without executing the automation workflow)

python main.py --dry-run

---

Technologies Used

• Python
• Playwright
• Python Dotenv
• Pillow

---

Design Decisions

• Modular architecture for easy maintenance.
• Separation of browser operations and agent logic.
• Semantic selectors instead of hardcoded coordinates.
• Screenshot-based verification.
• Comprehensive logging for debugging.
• Extensible design for future enhancements.

---

Future Improvements

• Dynamic form detection using computer vision.
• Support for multiple websites.
• Natural language task descriptions.
• Parallel browser sessions.
• Automatic retry and recovery mechanisms.

---

Assignment Requirements Covered

Requirement	Status
Open Browser	✅
Navigate to URL	✅
Click	✅
Scroll	✅
Send Keys	✅
Take Screenshot	✅
Intelligent Element Detection	✅
Error Handling	✅
Logging	✅
Modular Architecture	✅

---

Author

Developed as part of the Website Automation Agent Assignment demonstrating autonomous browser automation using Playwright and intelligent element detection.