✈ Airflow Studio

A visual DAG builder for Apache Airflow. Drag, drop, and connect operators on a canvas — generate valid, idiomatic Python DAG files in both Traditional and Taskflow API syntax.

✨ Features

• Visual drag-and-drop canvas — ReactFlow-powered node graph with auto-validation for a seamless editing experience.
• Dual syntax generation — Traditional (with DAG(...)) and Taskflow API (@dag + @task). Toggle modes instantly.
• 11 built-in operators — Python, Bash, S3, BigQuery, Postgres, HTTP, SQL sensors, and more natively supported.
• Extensible registry — Add new operators with just YAML + Jinja2 parameters. Zero Python backend changes required!
• Task Groups — Create and manage Airflow Task Groups directly from the canvas UI. Drag task nodes onto a Task Group to assign them as children.
• Keyboard Shortcuts — Accelerate DAG building with standard keyboard shortcuts (Ctrl+Z/Cmd+Z for Undo, Delete for node removal, Esc to deselect, etc.).
• Dark / Light theme — Elegant, user-customizable UI themes toggled with one click and persisted via localStorage.
• Live validation — Instant cycle detection, required parameter validation, and dag_id checks as you build.
• Save & load workspaces — Persist DAG graphs to local storage or backend service and reload them later for continued editing.
• Code preview — See generated code instantly as you place nodes and visually construct your workflow, with one-click copy to clipboard.

---

🚀 Quick Start

Local Development

# Backend
cd backend
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
uvicorn main:app --host 0.0.0.0 --port 8000 --reload

# Frontend (separate terminal)
cd frontend
npm install
npm run dev

Service	URL
DAG Builder UI	http://localhost:3000
API + Swagger	http://localhost:8000/docs

Docker Compose

docker compose up -d

Service	URL
DAG Builder UI	http://localhost:3000
DAG Builder API	http://localhost:8000
Airflow Webserver	http://localhost:8080

Default Airflow credentials: admin / admin

---

🔌 Supported Operators

Category	Operator	Provider
Python/Bash	PythonOperator	core
Python/Bash	BashOperator	core
Python/Bash	BranchPythonOperator	core
Sensors	S3KeySensor	apache-airflow-providers-amazon
Sensors	HttpSensor	apache-airflow-providers-http
Sensors	SqlSensor	apache-airflow-providers-common-sql
Transfers	PostgresOperator	apache-airflow-providers-postgres
Transfers	BigQueryInsertJobOperator	apache-airflow-providers-google
Transfers	S3ToGCSOperator	apache-airflow-providers-google
Flow Control	TriggerDagRunOperator	core
Flow Control	ShortCircuitOperator	core

---

➕ Adding a New Operator

No Python code changes required — just YAML + Jinja2.

1. Add to codegen/registry.yaml

  - id: snowflake_operator
    label: Snowflake SQL
    category: transfers
    import_path: airflow.providers.snowflake.operators.snowflake
    class_name: SnowflakeOperator
    provider_package: apache-airflow-providers-snowflake
    template: snowflake_operator.j2
    required_params:
      - name: task_id
        type: string
        label: Task ID
      - name: sql
        type: text
        label: SQL statement
    optional_params:
      - name: snowflake_conn_id
        type: connection
        label: Snowflake connection
        default: snowflake_default
      - name: warehouse
        type: string
        label: Warehouse
        default: ""

2. Create codegen/templates/snowflake_operator.j2

{{ task_id }} = SnowflakeOperator(
    task_id="{{ task_id }}",
    sql={{ sql | to_python_string }},
    snowflake_conn_id="{{ snowflake_conn_id | default("snowflake_default") }}",
    {% if warehouse %}warehouse="{{ warehouse }}",{% endif %}
)

3. Restart the backend

The operator will automatically appear in the left palette. No frontend changes needed.

Supported param types: string, int, bool, text (multi-line), dict, list, connection

Available Jinja2 filters: python_bool, to_python_string, format_start_date, tojson

---

🔄 Code Generation Modes

Traditional (default)

with DAG(dag_id="my_dag", ...) as dag:
    extract = PythonOperator(task_id="extract", python_callable=extract_fn)
    transform = PythonOperator(task_id="transform", python_callable=transform_fn)
    extract >> transform

Taskflow API

@dag(dag_id="my_dag", ...)
def my_dag():
    @task()
    def extract(**context):
        pass

    @task()
    def transform(**context):
        pass

    extract_result = extract()
    transform_result = transform()

my_dag()

Toggle between modes using the Traditional / Taskflow pill in the toolbar. Sensors and transfers remain as traditional operators inside the @dag function (valid Airflow mixed-mode).

---

🛠 API Endpoints

Method	Path	Description
POST	/api/v1/dags/generate	Generate Python code from graph JSON
POST	/api/v1/dags/validate	Validate graph (cycles, required params)
POST	/api/v1/dags/save	Persist graph to storage
GET	/api/v1/dags/	List all saved DAGs
GET	/api/v1/dags/{id}	Load a saved DAG graph
DELETE	/api/v1/dags/{id}	Delete a saved DAG
GET	/api/v1/operators/	Operator registry (for palette)

---

🧪 Tests

cd backend
source venv/bin/activate
python -m unittest discover tests -v
# 36 tests, all passing

---

🏗 Architecture

Browser (React + ReactFlow + Zustand)
    │  drag-drop canvas, config panel, code preview
    │
    ▼  REST API
FastAPI Backend
    ├── /generate  →  DAGGenerator  →  Jinja2 templates  →  .py source
    ├── /validate  →  cycle detection, required params, dag_id check
    ├── /save      →  JSON file store (swappable for PostgreSQL)
    └── /operators →  registry.yaml  →  palette data
    │
    ▼  direct write / git commit
Apache Airflow
    ├── Scheduler  (picks up .py files from dags/ folder)
    └── Webserver  (run, monitor, backfill)

---

🎨 Theming

• Dark mode — Warm amber/stone palette (default)
• Light mode — Clean stone/white palette
• Font — Outfit geometric sans-serif + JetBrains Mono for code
• CSS Variables — All colors tokenized in globals.css, switchable via data-theme attribute

---

📋 Production Checklist

• Change AIRFLOW__WEBSERVER__SECRET_KEY in docker-compose.yml
• Set strong Postgres password
• Configure CORS allow_origins in backend/main.py
• Mount persistent volume for backend/data/
• Set up Airflow connections (AWS, GCP, etc.) via Airflow UI
• Add authentication to API (JWT middleware recommended)

---

📜 Changelog

All notable changes to Airflow Studio are documented in this section.

[0.2.0] - 2025-03-20

Added

• Keyboard Shortcuts: Canvas now supports keyboard shortcuts for faster DAG building.
  • Ctrl+Z / Cmd+Z — Undo last action
  • Ctrl+Y / Cmd+Shift+Z — Redo
  • Delete / Backspace — Delete selected node or edge
  • Ctrl+A / Cmd+A — Select all nodes
  • Escape — Deselect / close right panel

[0.1.0] - 2025-03-16

Added

• Task Groups: Users can now create and manage Airflow Task Groups directly from the canvas UI.
  • New task_group node type available in the operator panel
  • Drag any task node onto a Task Group to assign it as a child; React Flow handles parent-child rendering automatically
  • Task Group node renders with dashed border and translucent background for clear visual distinction
  • Right panel support for group_id and tooltip properties
  • Code generator (generator.py) updated to emit valid with TaskGroup(...) as <id>: Python blocks with correct indentation
  • from airflow.utils.task_group import TaskGroup injected automatically when Task Groups are present in the DAG

[0.0.1] - Initial Release

• Core canvas with drag-and-drop DAG building
• Operator registry with standard Airflow operators
• Python code generation via generator.py
• Right panel for task configuration

---

📄 License

MIT