Keyword Automation with Python Behave & Playwright

This automation project searches for keywords on Google, checks if arifacademy appears in the search results, and updates a Google Sheet with the findings.

📋 Features

✅ Read keywords from Google Sheet
✅ Search each keyword on Google using Playwright
✅ Check if arifacademy appears in top search results
✅ Record position number
✅ Capture screenshots of search results
✅ Update Google Sheet with date, keyword, status (Yes/No), and position
✅ BDD format with Behave for clear test scenarios

📁 Project Structure

Keyword Automation python/
├── features/
│   ├── keyword_search.feature      # BDD test scenarios
│   └── steps/
│       └── steps.py               # Step definitions
├── utils/
│   ├── google_sheets.py           # Google Sheets API integration
│   └── search_engine.py           # Playwright & Google search logic
├── config/
│   ├── .env.example               # Environment variables template
│   └── credentials.json           # Google OAuth credentials (add manually)
├── screenshots/                    # Search result screenshots
├── requirements.txt               # Python dependencies
├── behave.ini                     # Behave configuration
├── run_automation.py              # Main runner script
└── README.md                      # This file

🚀 Setup Instructions

1. Install Dependencies

pip install -r requirements.txt

2. Google Sheets Setup

Get Google Sheet ID:

• Open your Google Sheet
• Copy the ID from the URL: https://docs.google.com/spreadsheets/d/SHEET_ID_HERE/edit

Create Google OAuth Credentials:

1. Go to Google Cloud Console
2. Create a new project
3. Enable Google Sheets API
4. Create OAuth 2.0 credentials (Desktop application)
5. Download credentials.json and save to config/ folder

Prepare Your Google Sheet:

• Column A: Keywords (starting from A2)
• Column B: Date (auto-filled)
• Column C: Keyword (auto-filled)
• Column D: Found (Yes/No)
• Column E: Position (ranking number)
• Column F: Screenshot Path
• Column G: Status

3. Environment Configuration

1. Copy .env.example to .env:

cp config/.env.example .env

2. Edit .env and add:

GOOGLE_SHEET_ID=your_sheet_id_here
GOOGLE_CREDENTIALS_FILE=config/credentials.json
TARGET_DOMAIN=arifacademy
SEARCH_ENGINE_URL=https://www.google.com
RESULTS_PER_PAGE=10

🎯 Usage

Run Automation

python run_automation.py

Run Specific Feature

behave features/keyword_search.feature

Run with Detailed Output

behave features/keyword_search.feature -v

📊 Google Sheet Format

Expected columns in your Google Sheet:

A	B	C	D	E	F	G
Keyword	Date	Keyword	Found	Position	Screenshot	Status
seo tips	2024-01-15 10:30	seo tips	Yes	3	path/screenshot.png	Checked
web design	2024-01-15 10:35	web design	No	N/A		Checked

⚙️ Configuration Options

In .env file:

GOOGLE_SHEET_ID          # Your Google Sheet ID (required)
GOOGLE_CREDENTIALS_FILE  # Path to credentials.json
TARGET_DOMAIN           # Domain to search for (default: arifacademy)
RESULTS_PER_PAGE        # Number of results to check (default: 10)
USE_PROXY              # Enable proxy (optional)
PROXY_URL              # Proxy URL if enabled

🔍 How It Works

1. Read Keywords: Fetches keywords from Google Sheet column A
2. Search: Uses Playwright to automate Google search for each keyword
3. Check Results: Scans search results for arifacademy domain
4. Screenshot: Captures search results page showing the ranking
5. Update Sheet: Writes results back to Google Sheet with:
   • Date/Time
   • Keyword
   • Found status (Yes/No)
   • Position number (if found)
   • Screenshot path

📝 Example Output

✓ Read 5 keywords from Google Sheet
  1. seo tips
  2. web design
  3. digital marketing
  4. content strategy
  5. keyword research

✓ Searching 5 keywords on Google...

  [1/5] Searching: 'seo tips'
    ✓ Found at position 3
  [2/5] Searching: 'web design'
    ✗ Not found
  [3/5] Searching: 'digital marketing'
    ✓ Found at position 1
  [4/5] Searching: 'content strategy'
    ✓ Found at position 5
  [5/5] Searching: 'keyword research'
    ✗ Not found

✓ Updating Google Sheet with 5 results...
  ✓ Updated: seo tips
  ✓ Updated: web design
  ✓ Updated: digital marketing
  ✓ Updated: content strategy
  ✓ Updated: keyword research

✓ All results updated successfully!

🐛 Troubleshooting

Error: "GOOGLE_SHEET_ID not found"

• Make sure .env file exists and contains GOOGLE_SHEET_ID

Error: "credentials.json not found"

• Download OAuth credentials from Google Cloud Console
• Save to config/credentials.json

Error: "Permission denied" from Google Sheets

• Share the Google Sheet with the OAuth email
• Grant Edit access

Browser automation slow?

• Results appear in screenshots/ folder
• Check network connection
• Adjust timeouts in utils/search_engine.py

📦 Dependencies

• behave: BDD framework
• playwright: Browser automation
• google-api-python-client: Google Sheets API
• python-dotenv: Environment variable management

📄 License

This project is open source and available for use.

✨ Notes

• First run will prompt Google OAuth login
• Token is saved in token.pickle for future runs
• Screenshots are saved in screenshots/ with keyword and position
• All timestamps are in local timezone

---

Need help? Check the config/.env.example file or contact your administrator.