Azure DevOps Test Case Automation

A lightweight Python script that bulk-creates structured Test Case work items in Azure DevOps and automatically links them to a parent User Story — all via an AI-generated JSON file.

---

How It Works

1. Configuration: Supply your Azure DevOps credentials and workflow details (via terminal prompts or config.json).
2. AI Generation (n8n): The script triggers an n8n webhook, which dynamically generates comprehensive test cases based on your specified User Story, and saves them locally.
3. Manual Review: The script explicitly pauses after generation, giving you the time to manually review, modify, or correct the generated test_cases.json payload before uploading.
4. Verification: After you continue, the script smoke-tests your authentication and confirms project reachability.
5. Bulk Creation: It creates each test case as an Azure DevOps Work Item with step-by-step actions and expected results, natively linking them to your parent User Story.

---

Project Structure

Azure_TC_Automation_Script/
├── interactive_app.py         # 🌟 New Interactive terminal tool (Recommended)
├── ado_test_case_creator.py   # Core logic & file-based background automation
├── config.json                # Azure DevOps config (Only needed for ado_test_case_creator.py)
├── test_cases.json            # Your test cases payload (Auto-generated by N8N)
├── .gitignore                 # Excludes config.json and test_cases.json from source control
└── README.md                  # This file

---

Prerequisites

• Python 3.10+ (uses list[dict] type hints introduced in 3.10)
• The requests library:

pip install requests

---

Execution Modes

You can run the automation tool in three distinct ways depending on your workflow:

Mode 1: Standalone Executable (Easiest)

You can run this automation perfectly without having Python installed!

1. Download: Go to the Releases Page and download the interactive_app.exe file.
2. Run: Double click the .exe file to seamlessly launch the interactive terminal environment.

Mode 2: Interactive Terminal (via Python)

Run the brand new interactive application directly from the source code. It requires zero configuration files and provides fully secure, colored terminal prompts.

python interactive_app.py

What it does: It will ask you for your Azure Organization, Project, PAT (hidden input), User Story ID, and Description one by one directly in the console, saving you from navigating code or config files. It also supports accepting multi-line text inputs for the User Story description (e.g., pasting multiple paragraphs directly into the terminal and finishing the input with Ctrl+Z then Enter).

Mode 3: Config-Based Automation

Best for experienced users or pure automation pipelines where triggering test cases rapidly through files is preferred rather than manual prompting.

1. Setup your config.json file next to the scripts:

{
    "organization": "your-org-name",
    "project": "your-project-name",
    "pat": "your-personal-access-token",
    "api_version": "7.1",
    "user_story_id": "<USER-STORY-ID>",
    "user_story_description": "<USER-STORY-DESCRIPTION>",
    "n8n": {
        "webhook_url": "<YOUR-N8N-WEBHOOK-URL>",
        "timeout": 120
    }
}

2. Run the underlying script directly:

python ado_test_case_creator.py

Key	Description
organization	The slug from your ADO URL: https://dev.azure.com/{organization}/
project	The exact project name (case-sensitive) as it appears in Azure DevOps
pat	A Personal Access Token with Work Items → Read & Write scope
api_version	Azure DevOps REST API version — 7.1 is recommended and works by default
user_story_id	Mandatory. The exact Azure DevOps User Story ID you want to test
user_story_description	Mandatory. A text description of the logic the User Story implements
n8n.webhook_url	Full URL to your 24/7 self-hosted n8n webhook trigger
n8n.timeout	Max seconds script will wait for n8n to respond (default 120)

How to create a PAT

1. Sign in to your Azure DevOps organization: https://dev.azure.com/{your-organization}

2. Click the User settings icon in the top-right corner of the page (the person icon next to your profile picture).

3. Select Personal access tokens from the dropdown menu.

4. Click + New Token.

5. Fill in the token details:

   • Name — give it a recognizable name, e.g. TC Automation Script
   • Organization — select your organization from the dropdown
   • Expiration — choose a suitable expiry date (e.g. 30 or 90 days)
   • Scopes — select Custom defined, then scroll down to Work Items and check Read & write

6. Click Create.

7. Copy the token immediately — Azure DevOps will only show it once. Paste it into config.json as the value for "pat".

Caution

If you navigate away without copying the token, it cannot be retrieved. You will need to delete it and create a new one.

Caution

config.json is excluded from Git via .gitignore. Never commit it — it contains your PAT which grants write access to your Azure DevOps project.

---

The 4-Step Automation Process

Regardless of which execution mode you choose, the tool executes via a seamless 4-step pipeline:

1. AI Generation (via n8n)

It fires a POST payload containing your User Story to the N8N webhook, allowing your AI node to generate edge cases and positive/negative flows. It saves these back natively to test_cases.json.

2. Manual Review Pause

Important

The purpose of waiting for the user to press Enter is to allow you some time to review the test_cases.json file generated by the N8N workflow in case you need to apply any kind of modifications on the file before permanently submitting it to ADO.

3. Authentication Verification

Checks your project ID to proactively verify token health before interacting with ADO Work Items.

4. Bulk Creation

Pushes out all the Work Items! Each test case is linked to the parent User Story using a Hierarchy-Reverse relation (child → parent) naturally.

---

Example Console Run

Azure DevOps Test Case Automation
============================================================

[1/4] Triggering N8N workflow for User Story #40521...
  Waiting for workflow to complete (timeout: 120s)...
  ✓ Workflow completed (HTTP 200)
  ✓ Successfully retrieved 15 test cases from N8N.

  ⏸ [PAUSE] The script has paused so you can manually review/edit 'test_cases.json'.
  Press ENTER when you are ready to continue and upload to Azure DevOps...

[2/4] Verifying Authentication and Project Access...
  ✓ Auth OK — project 'NDCIntegrations' accessible

[3/4] Loading test cases from 'test_cases.json'...
  ✓ Loaded 15 test case(s).

[4/4] Creating 15 Test Case(s) in project 'NDCIntegrations'...
[1/15] Creating: 'Verify login with valid registered email and password (Positive)' → parent #40521
  ✓ Created Work Item #41389 → https://dev.azure.com/...
...

📊 Summary 📊
Successfully created 15 work item(s):
  # 41389  Verify login with valid registered email and password
          https://dev.azure.com/...

---

🛠 Building From Source

If you made modifications to the interactive_app.py script and want to re-compile your own standalone .exe version for distribution, use PyInstaller:

pip install pyinstaller

# Compile the interactive app into a single executable file with the icon
pyinstaller --onefile interactive_app.py --icon script.ico

The resulting executable will be generated inside the dist/ folder!

---

Troubleshooting

Error	Likely Cause	Fix
Authentication failed (401)	PAT is wrong, expired, or has insufficient scope	Regenerate your PAT with Work Items → Read & Write scope
Project not found (404)	organization or project is misspelled	Check them against your ADO URL — both are case-sensitive
Config file not found	config.json is missing (Mode 2)	Create it next to the script using the template above
Test cases file not found	test_cases.json is missing	N8N failed to trigger properly, check the webhook URL.
Missing required config keys	A key is missing from config.json	Ensure all keys are present: organization, project, pat, api_version
JSONDecodeError	config.json or test_cases.json contains invalid JSON	Validate the file at jsonlint.com