Skip to content

Releases: ctolon/autoair

AutoAir 0.2.3

Choose a tag to compare

@github-actions github-actions released this 22 Oct 16:28

Security

  • Generated DAGs write embedded scripts into a folder private to the worker's user
    (<tmp>/airflow-scripts-<uid>, mode 0700, checked: not a symlink, owned by that user, no
    group or other access; otherwise a fresh private folder). An existing script is used only
    when it is a regular file of that user with exactly the expected content; anything else,
    including a symlink planted at the content-addressed path, is replaced atomically.

  • Standalone token verification accepts only Airflow's own answer: a 200 with the DAG list.
    A redirect (an SSO proxy or a wrong airflow_url sending the check to a login page), another
    2xx or an unexpected body used to count as a verified token, so a forged JWT's role claims
    could be trusted. Such answers now fail closed with a 502 that names the URL.

  • Processes that run DAG code (import checks, the validator's workers, Python imports, blueprint previews, test runs, language servers) no longer inherit Airflow's secrets or cloud credentials: AIRFLOW__DATABASE__SQL_ALCHEMY_CONN, AIRFLOW__CORE__FERNET_KEY, AIRFLOW__API_AUTH__JWT_SECRET, every AIRFLOW__*SECRET*/*_CMD setting, AIRFLOW_CONN_*, AIRFLOW_VAR_*, AWS_* keys, GOOGLE_APPLICATION_CREDENTIALS, AZURE_* secrets and other credential-like names are removed (sandbox.credential_env_pattern, new). Before, only AUTOAIR_* secrets were. Variables whose value is a URL with a password are removed whatever their name. DAG test runs with mode db/auto keep the metadata-DB settings and Fernet key dag.test() needs, with a random per-run JWT secret instead of Airflow's. Upgrade note: DAGs that read a variable or connection from the environment at parse time, and task tests that need one, get it back with sandbox.env_passthrough (new, fnmatch patterns, e.g. ["AIRFLOW_VAR_ENVIRONMENT"]).

  • Git commands of provider=local/gitea targets and repository bulk imports inherit only what git, ssh and curl read (PATH, HOME, locale, TMPDIR, SSH_AUTH_SOCK, CA and proxy variables, GIT_*), not the whole server environment; storage.git.env_passthrough (new) adds names such as KRB5CCNAME. SSH private keys from Airflow connections were cached in one id_autoair file per clone cache and never rewritten, so a user whose connection shared that cache (Gitea per-user credentials, a rotated key) pushed with the first user's key; each key now gets its own owner-only file named after its hash.

  • Recycling a merged pull-request branch moved it with git push --force, discarding a commit pushed to it meanwhile; it now pushes with --force-with-lease on the head it saw (Gitea / local git; Azure DevOps uses the ref update's compare-and-swap, GitHub, GitLab and Bitbucket re-check the head first) and starts over when the branch moved.

  • Deploy targets fail closed when AutoAir's state record (.autoair/state/<dag_id>.json) is unreadable: no file of that DAG is replaced or deleted (409 naming the file and how to recover) instead of the record being ignored, which disabled the hand-written and hand-edited (ejected) file protection. Without a state record, an existing DAG file that differs from the new one counts as hand-edited and is replaced or deleted only with force (an untouched loader stub excepted).

  • The hard-coded secret scan (policy.secret_assignment_pattern) also finds secrets passed inline, such as keyword arguments (Connection(password="...")), statements after ; and quoted command-line options; it was anchored to the start of a line. Deployments that override the pattern keep their own.

  • The hard-coded secret scan also finds secrets in dict literals and dict() calls in Python code (inline functions, @task bodies, raw code DAGs, hand-written parts of source-preserving deploys): an entry whose key matches policy.secret_param_pattern (and not the exclude pattern) and whose value is a string literal, such as {"token": "ghp_..."} or dict(password="..."), is a secret warning on the key's line. Values that are not plain literals (variables, os.environ[...], Variable.get(...), f-strings), Jinja or format placeholders, empty or short strings, environment variable names (GITHUB_TOKEN) and placeholders (<your-token>, changeme, ********) are not reported.

  • Standalone autoair serve with the default airflow_permissions strategy: Airflow's DAG and configuration permissions cannot be checked there, and an unchecked permission counted as granted, so every user with a valid Airflow token (an Airflow Viewer included) got the Author, Deployer and Admin roles. An unchecked permission now grants only iam.airflow_permission_roles.unverified (new, default ["Viewer"]); plugin mode, where Airflow answers these checks, is unchanged. Upgrade note: standalone deployments that relied on this grant roles explicitly with iam.strategies=["role_mapping"] (the verified token's role claim, alone or together with airflow_permissions) or iam.default_roles.

  • Servers without authentication (autoair serve --no-auth, autoair mcp-serve --http --no-auth, autoair mcp --http / autoair-mcp --http, autoair ai-serve --no-auth, autoair local --no-token) refuse to bind a non-loopback address and answer only requests whose Host (and Origin, when sent) is a loopback address or the host of server.public_url, against DNS rebinding and cross-site pages. iam.anonymous_roles defaults to ["Viewer"] (was ["Admin"]). Upgrade note: development setups that let anonymous callers edit and deploy set AUTOAIR_IAM__ANONYMOUS_ROLES='["Admin"]'.

  • autoair local is authenticated by default: it creates a random access token on every start, prints a sign-in link and opens the UI with it (/ui/#token=..., kept in the URL fragment, which never reaches a server). The token holder gets local.roles (new, default ["Admin"]), so the local user keeps editing and deploying; other programs and web pages on the machine can no longer use the API. --token / AUTOAIR_LOCAL__TOKEN still set the token; the new --no-token serves without authentication with iam.anonymous_roles (Viewer).

  • Test runs (POST /runner/test-task, /runner/test-dag) execute the DAG's Python on the server and now need code:execute in addition to run:test, like every other code-running route. The built-in roles are unaffected (every role with run:test also holds code:execute); custom roles granting run:test alone lose test runs.

  • Plugin mode: state-changing requests authenticated by Airflow's _token cookie (Airflow 3.1+) now need the X-AutoAir-CSRF header and a same-origin or allowed Origin, as standalone cookie sessions already did. sso.require_csrf_header controls it in both modes. The bundled UI already sends the header; scripts that use the browser cookie instead of a bearer token must add it.

  • Saving a template with hand-written DAG code (a raw code DAG, spec.raw_code, or a kept Python file, spec.source.code) through POST /api/templates or a marketplace install now needs code:raw, like saving that spec. Upgrade note: TemplateManager alone can no longer publish such templates.

  • Collaboration tickets are bound to the requesting user and the room, open a single connection (claimed in the shared state store, so this holds across replicas) and are valid for 60 s by default (collab_ticket_ttl, was 600).

  • Language-server WebSocket tickets (POST /api/lsp/session) could open any number of sockets within their 60 s TTL; like collaboration tickets they now name the user and open one socket, claimed in the shared state store (documents collection lsp_tickets, or <state>/lsp-tickets/), so a leaked ticket cannot be replayed on any replica.

  • autoair mcp --http (and autoair-mcp --http) served every tool without authentication and without the loopback and DNS-rebinding protections of the other unauthenticated servers. It now refuses a non-loopback --host, checks Host and Origin like mcp-serve --http --no-auth, and gives callers iam.anonymous_roles (default Viewer, read-only). Upgrade note: a local agent that saves or deploys through it needs AUTOAIR_IAM__ANONYMOUS_ROLES='["Admin"]' (or use stdio, autoair mcp, which acts as the local user).

  • Storing hand-written Python in a spec (source.code, the file a DAG was imported from or edited as in the code mode) needs code:raw when the code is new or differs from the stored spec's, whatever the source policy: PUT /api/specs/{dag_id} answers 403, the code mode apply (POST /api/interchange/python/sync) answers 403 when the applied file differs from the document's code, and MCP save_dag_spec refuses the save. Before, only a deploy that wrote that code as the DAG file needed it, so a user without code:raw could store code that a later deploy by someone else wrote. This replaces the deploy's source_refused answer for such users (the Deployer role included) with the save-time 403; deploying the generated file instead now means saving with the source policy generate and without source.code. Upgrade note: custom roles that edit in the code mode or import Python DAGs and deploy them need code:raw (Author, CodeDeployer and Admin have it).

  • Four-eyes approvals (iam.require_approval): AutoAir now records who saved each stored spec version (shared state: documents collection spec_version_authors, or <proposals URL>-authors/), so the saver of a version can no longer approve it (403) and their approval never counts towards putting it live, whoever deploys it; a version saved anonymously never satisfies four-eyes. Before, a version stored without a proposal (a pull request, a save made before approvals were required) could be approved by its own author and deployed by someone else. Versions saved before this release have no record and keep the previous rule (the deploying user counts as the author).

  • MCP validate_dag_spec imported (executed) the generated DAG with import_check=true while annotated r...

Read more

AutoAir 0.2.2

Choose a tag to compare

@github-actions github-actions released this 22 Oct 07:01

Added

  • Contributors: pre-commit hooks (uv run pre-commit install) for ruff, the generated
    configuration reference, the hard-coded audit, version agreement, prettier, eslint, the MDX
    check and actionlint, plus mypy before push; pre-commit is in the dev dependencies.

Security

  • CI and release workflows pin third-party GitHub Actions (setup-uv, Docker build actions, PyPI
    publish) to full commit SHAs, so a moved tag cannot change what builds and publishes AutoAir
    releases; Dependabot keeps the pins current. astral-sh/setup-uv moves to v10 (Node 24).
  • AI prompt-injection shield: removing <untrusted-data> tags from untrusted text now takes
    linear time; a document full of unclosed tags could stall a request for seconds.
  • Deploy target dialog: the suggested git provider comes from the remote's parsed hostname,
    matched on whole domain labels, so a look-alike host (bitbucket.org.example.com) or a
    provider name in the URL path no longer selects that provider's API.
  • Docker Compose demo (tls profile): the generated server key is readable by its owner
    only (mode 0600 instead of 0640).

Install: pip install autoair==0.2.2 ·
Image: ghcr.io/ctolon/autoair:0.2.2

AutoAir 0.2.1

Choose a tag to compare

@github-actions github-actions released this 22 Oct 06:13

Fixed

  • Deploy target dialog: object storage targets can be chosen like git and local ones. Airflow's
    S3DagBundle and GCSDagBundle entries are discovered, and an object storage prefix can be
    entered by hand (fsspec URL plus optional non-secret storage options; credentials still come
    from the server environment or AUTOAIR_STORAGE__STORAGE_OPTIONS). Before, switching to
    object storage was possible only through the API. storage.allowed_targets still applies.
  • Container image: pip check passes again. The Airflow base image ships litellm (an optional
    dependency of Vertex AI evaluation), which pins openai<3, while autoair[ai-openai] needs
    openai>=3. The image now removes litellm, which AutoAir does not use, and the build and
    the CI smoke test run pip check. Vertex AI evaluation with third-party judge models needs
    litellm and is not available in this image.
  • ai.llm_conn_id finds connections stored in the Airflow metadata database again. Outside a
    task the Task SDK's BaseHook.get_connection does not read the metadata DB, so on Airflow
    3.1+ the plugin, autoair serve and autoair ai-serve reported "conn_id isn't defined" for
    connections created in the Airflow UI or CLI. All connection lookups (LLM, git, knowledge
    base, validator key) now share one helper that checks env vars, the configured secrets
    backend and the metadata DB first, then the Task SDK. The
    AIRFLOW__SECRETS__BACKEND=airflow.secrets.metastore.MetastoreBackend workaround is no
    longer needed.
  • Python DAG import (import dialog, code mode, bulk import, MCP import_python_dag) keeps the
    python_callable of classic PythonOperator, BranchPythonOperator,
    ShortCircuitOperator and PythonSensor tasks (and other callable operator arguments).
    Before, the callable was dropped and the deploy failed with "python_callable missing". A
    function defined in the DAG file becomes an inline function (its source without decorators,
    with the file's imports it uses), a function imported from another module an import
    reference; a lambda, a functools.partial or a function that uses other names of the file
    is reported in the import warnings. Source-preserving deploys of such files compare these
    functions as code, so a canvas change is patched into the file instead of being refused.

Install: pip install autoair==0.2.1 ·
Image: ghcr.io/ctolon/autoair:0.2.1

AutoAir 0.2.0

Choose a tag to compare

@github-actions github-actions released this 22 Oct 05:29

Added

  • Kustomize manifests for Kubernetes (deploy/kustomize): a hardened base for autoair serve
    (non-root, read-only root file system, no service account token, probes on /api/health and
    /api/ready, PodDisruptionBudget, topology spread), Components for PostgreSQL state with an
    autoair db upgrade init container, the git deploy target, the validator service (no egress),
    the AI service, a separate MCP server, a Gateway API HTTPRoute (the production default) or an
    Ingress, HorizontalPodAutoscaler,
    NetworkPolicies and Prometheus ServiceMonitors, and dev, production and airflow-plugin
    overlays. Secrets are referenced, never included; credentials stay Airflow connections. CI
    builds every overlay and validates it with kubeconform. New page: Deployment → Kubernetes
    with Kustomize.
  • AutoAir docs MCP: AutoAir's own documentation (the website pages and the configuration
    reference) is packaged in the wheel at build time (hatch_build.py, autoair.docs.build) and
    served to coding agents by the MCP tools search_autoair_docs, read_autoair_doc and
    list_autoair_docs: offline, matched to the installed version, without Airflow, the AI extra
    or a knowledge-base store. autoair docs-mcp (stdio, or --http) serves only these tools,
    e.g. uvx --from 'autoair[mcp]' autoair docs-mcp; they are also on every AutoAir MCP endpoint
    (no AutoAir permission, feature flag features.autoair_docs). Setup for Claude Code, Cursor,
    OpenCode and Codex CLI: website page AI → Docs MCP.
  • Knowledge base: built-in source kind and component autoair (AutoAir's packaged
    documentation, no network), so kb_search and the assistants cite AutoAir pages.
  • Isolated import checks: autoair validator-serve runs the DagBag import check in a separate
    validator service, so the DAG code it imports (raw-code DAGs, Python task bodies) no longer
    executes in the Airflow API server or autoair serve (Python DAG import and test runs still
    do). Set validator.url
    (AUTOAIR_VALIDATOR__URL); requests are HMAC-signed with the password of the Airflow
    connection validator.secret_conn_id (no new secret setting). Workers get a fresh interpreter
    per check, memory/CPU/file-size/open-file limits, no new privileges, a network namespace where
    the kernel allows it and a Python audit hook (no subprocesses, sockets, DNS, writes outside
    their scratch folder or reads of other processes' /proc). Retries, a health probe and
    validator.fail_mode (open by default: an import_service warning and no import check
    while the validator is down; closed blocks deploys instead; local). /health, /ready, /metrics
    (autoair_validator_*) on the validator, autoair_import_checks_total on the main server.
    New Helm chart deploy/helm/autoair-validator (Deployment, Service, PodDisruptionBudget,
    NetworkPolicy denying all egress, read-only non-root pods) and Compose overlay
    deploy/compose/docker-compose.validator.yml. New page: Deployment → Validator service.
  • Astronomer Blueprint steps: templates of airflow-blueprint (installed packages with the
    airflow_blueprint.blueprints entry point, blueprints.template_dirs, blueprints.modules)
    appear in the palette's Blueprints (Astronomer) section and become kind: blueprint nodes
    with a config form generated from the template's Pydantic model, version selection, a task
    preview (inspector and collapsible node) and validation against the config schema. Python and
    loader output call Template().render(Config(...)); the YAML output mode reports them as
    unsupported, the dag-factory export skips them. The Python importer folds render() calls
    back into blueprint steps. New extra autoair[blueprint], feature switch
    features.blueprints, settings section blueprints, routes /api/blueprints/* and the MCP
    tool list_blueprints. Guide: User guide → Blueprint steps.
  • Airflow 3.4: plugin.dag_tab_tags (AUTOAIR_PLUGIN__DAG_TAB_TAGS) shows the AutoAir
    DAG tab only on DAGs with one of the listed tags, through the plugin's applies_to
    scoping. Empty (default) keeps the tab on every DAG page; ignored before Airflow 3.4.
  • Validation warns about task groups that depend on each other in a cycle
    (task_group_cycle), which Airflow 3.4 deprecates and Airflow 3.5 is planned to reject.
  • Editing existing Python DAGs keeps the hand-written file: each DAG has a source policy
    (auto, generate, preserve, strict-preserve; DAG settings and the deploy dialog).
    Imported and code-mode DAGs default to preserve: a deploy applies only the change to the
    existing file with a LibCST patch (keyword arguments, TaskFlow bodies, added and removed
    tasks and dependencies, renames), verifies it with a round-trip import and the DagBag
    check, and never regenerates the file on its own. The deploy dialog previews the DAG file
    as a unified diff (POST /api/specs/{dag_id}/source-preview, MCP preview_dag_source);
    refused changes explain why and offer raw code or a deliberate switch to generate.
    A stopped deploy says why by the file's state (the hand-written file AutoAir keeps, a file
    changed outside AutoAir, or one AutoAir did not create); "Keep as raw code DAG" and
    "Overwrite the file" follow the user's code:raw and deploy:overwrite_ejected; a source
    policy picked in the deploy dialog changes the DAG only when it deploys (Cancel keeps it).
    Code mode and the Code tab show the hand-written file. New settings preserve.*, feature flags
    source_preserve and strict_source_preserve. libcst is a new core dependency.
  • Generated DAG files carry a fingerprint line (# autoair:fingerprint sha256=…), so a file
    edited outside AutoAir is recognised on folder, object-storage and git targets even without
    AutoAir's state. Such a deploy stops and offers to apply the change to the edited file
    (merged with the outside edits); preserve.ejected_files (block, warn, overwrite)
    sets the behaviour. New permission deploy:overwrite_ejected (Admin; never from a
    wildcard) for a confirmed overwrite.
  • Tool-call approval for AI agents (pydantic-ai deferred tools): when the developer or admin
    assistant or a troubleshooting analysis calls a tool that changes something (remember,
    external MCP tools not marked readOnlyHint), the run pauses with an approval card showing
    the tool and its masked arguments; the user who started the run approves or denies each
    call (optionally with a reason) and the run resumes in the same answer or analysis.
    Approvals are stored with the state (multi-replica safe, decided at most once, expiring
    after ai.approvals.ttl_seconds) and audited. New settings ai.approvals.*, new endpoints
    GET /api/ai/approvals, GET /api/ai/approvals/{id} and
    POST /api/ai/approvals/{id}/stream. New page: AI → Tool-call approval.
  • Large DAGs: above 150 nodes the canvas can zoom out to an overview of the whole graph
    (Fit view), and below 40 % zoom task cards turn into light, colour-tinted boxes with
    their name (problems, run states, diff and selection stay visible). Benchmarks for large
    DAGs: scripts/gen_large_dag.py (synthetic specs), scripts/bench_large_dag.py (server
    timings) and frontend/e2e/large-graph.spec.ts (load, layout, frame rate, memory). New
    page: User guide → Large DAGs (measured limits and tuning).
  • preserve.allow_force_overwrite (AUTOAIR_PRESERVE__ALLOW_FORCE_OVERWRITE, default
    false): the "Overwrite (force)" choice is offered and accepted only when it is true, for
    the hand-written file AutoAir keeps for a DAG, a DAG file changed outside AutoAir and force
    deploys of the loader and YAML output modes and raw code. The caller still needs
    deploy:overwrite_ejected and an explicit confirmation; /api/meta reports
    allow_force_overwrite, and the source-plan and conflict dialogs hide the overwrite button
    otherwise. Source plans refuse with the new code overwrite_disabled.
  • server.harden_process (AUTOAIR_SERVER__HARDEN_PROCESS, default true) controls whether
    autoair serve, autoair local (and their uvicorn workers), autoair mcp-serve --http and
    autoair ai-serve make their process non-dumpable. Turn it off only to debug with py-spy,
    gdb or core dumps; the server then logs a warning at start-up. Never applied in the Airflow
    plugin.

Changed

  • spec_version is now 3. Version 3 adds only optional parts (source and the blueprint
    node kind); version 1 and 2 documents keep loading and are written as version 3 the next
    time they are saved or deployed. Upgrade note: AutoAir 0.1.0 refuses version 3 specs
    with "spec_version 3 is newer than this AutoAir supports (2); upgrade AutoAir", also in the
    loader stubs inside Airflow. Upgrade every AutoAir process that reads the same specs
    together (plugin, autoair serve replicas, MCP and AI services, the autoair package in
    Airflow's dag-processor and workers); rolling back to 0.1.0 needs the specs from before the
    upgrade.
  • Overwriting a DAG file AutoAir would otherwise not replace now needs
    preserve.allow_force_overwrite=true (default false). preserve.ejected_files=overwrite
    takes effect only with it (otherwise it behaves like block, also for administrators), and
    regenerating the kept hand-written file (switch to generate) needs it as well as
    deploy:overwrite_ejected. Deleting a kept hand-written DAG still needs only the
    permission. Files without AutoAir's marker are never overwritten by force. A deploy that
    would replace the kept hand-written file without a source plan (another output mode, a
    conversion to raw code) returns HTTP 200 with deployed: false, status: "ejected" and
    conflict.ejected_file, conflict.hand_written and conflict.force_allowed instead of
    HTTP 409; ejected results without a source plan also carry conflict.force_allowed.
    Upgrade note: deployments that relied on `preser...
Read more

AutoAir 0.1.0

Choose a tag to compare

@github-actions github-actions released this 21 Oct 12:37

First public release.

Added

Builder and canvas

  • Visual DAG editor (React Flow) inside Airflow (Browse → DAG Builder and an AutoAir
    tab on every DAG page on Airflow ≥ 3.1; a full page at /autoair/ui/ on 3.0) and as a
    standalone server (autoair serve).
  • Integration logos on nodes and custom icon upload with an in-browser resizer; notes; task
    groups (group, ungroup, collapse); insert-on-edge; command palette; context menus;
    copy/paste with id and XCom remapping; alignment guides; undo/redo; ELK auto-layout in a web
    worker; run-state and version-diff overlays; keyboard shortcuts; collapsible panels.
  • Full-screen code mode and an optional editable Code tab (features.code_edit_inline) that
    re-import edited Python into the visual spec; a read-only preview of the file generated for
    a chosen environment (POST /api/codegen?environment=<env>).
  • Editor modes: features.visual_editor and features.code_editor give visual-only and
    code-only deployments; switching them at runtime keeps the open DAG and an unapplied code
    draft.
  • Raw code DAGs (DagSpec.raw_code, permission code:raw): a hand-written DAG file stored,
    versioned, reviewed, proposed, deployed, promoted and audited like a visual DAG; validated
    by syntax, ruff, secret detection, policy (privileged and forbidden operators found by a
    static scan, dag_id rules, OPA with context.document_type=raw) and the DagBag import
    check, with problems located by line. File → New raw code DAG, Convert to raw
    code…
    and Open in visual builder; a Raw code badge and a type filter in the Open
    dialog.
  • English and Turkish UI, light and dark themes, UI branding (title, logo, accent, banner).

Authoring

  • Typed DAG spec (Pydantic v2) with migrations as the single source of truth; the TypeScript
    client is generated from the OpenAPI document.
  • Every installed operator, sensor and transfer, from official and third-party providers,
    Airflow libraries, plugins, DAG folders and packages, through a static AST catalog that does
    not import the modules. The catalog can be precomputed with autoair catalog build.
  • Operator packages describe their operators for the palette without AutoAir configuration:
    an autoair.yaml / autoair.json file in the package or an autoair.operator_metadata
    entry point (palette group and order, icons including sanitised SVGs, parameter widgets,
    examples, docs links). Settings catalog.connection_param_patterns, catalog.groups,
    catalog.default_group_order, catalog.param_overrides and icons.rules_extra take
    precedence over package metadata, which takes precedence over inference.
  • Catalog diagnostics: GET /api/catalog/diagnostics, the admin panel's Catalog tab and
    autoair catalog dump --diagnostics (sources, operators per source, filtered operators and
    why, failed modules, applied package metadata, build time).
  • Python references to callables and classes; functions written in place; embedded script
    files written to every worker at parse time.
  • TaskFlow: @task, @task.branch, short_circuit, virtualenv, external_python,
    sensor, bash, docker, kubernetes and @task.stub (language SDKs).
  • Dynamic task mapping (expand, expand_kwargs, map_index_template), task loops
    (Airflow 3.4), setup/teardown, trigger rules, callbacks and notifiers, retry policies
    (3.3), deadlines (3.1), human-in-the-loop operators with a form designer, OpenLineage
    toggles, disabled tasks.
  • DAG params with a live trigger-form preview; environment overlays (dev/staging/prod).
  • Scheduling: DAG time zone (DST aware), cron, presets, intervals, multi-cron, event dates,
    asset expressions, event-driven assets (AssetWatcher with SQS, Kafka, Pub/Sub, Redis,
    Service Bus, Kinesis), assets-or-time, assets-and-time (3.4) and partitioned schedules
    (3.2), with a live "next runs" preview from Airflow's timetable engine.
  • Jinja-aware editors for templated fields with completion and inline preview;
    drag-to-map upstream XComs; connection pickers filtered by type with inline "New
    connection" (create and test in Airflow); IDE-style operator docs; Markdown fields with
    snippets and preview.
  • Python and shell editors: jedi completion, hover and signature help that know the installed
    Airflow and providers, live ruff lint, ShellCheck diagnostics. Optional language servers
    (basedpyright, pylsp, bash-language-server) per session.
  • Output as readable Python, a JSON spec with a runtime loader, an Airflow 3.4 native YAML
    DAG, or dag-factory YAML. Generated imports follow the target Airflow version.
  • Import from hand-written Python DAGs (mapped tasks, partial() arguments and cron presets
    kept), from Airflow, and from YAML.

Validation and policy

  • Live validation: graph, catalog, Jinja, policy and Airflow version gates, plus a warm
    DagBag import-check pool.
  • Organisation policy: flat switches (required owner or description, minimum retries,
    required tags, forbidden operators, max tasks, dag_id prefix, operator allow/deny lists,
    privileged operators, Python code), configurable id and secret-detection patterns,
    declarative policy.rules, and an optional Open Policy Agent decision.
  • Hard-coded secret detection in parameters and inline code, with a hint to use Connections
    or Variables.
  • Rules-only best-practice review and leak/secret scan without AI (POST /api/review/rules,
    POST /api/lint/secrets, builder:use), offered in the Run menu when AI is off.

Lifecycle and deploy

  • Deploy targets: a local folder (LocalDagBundle), object storage through fsspec
    (S3/GCS DagBundles) and Git: the git CLI with any remote, or the GitHub, GitLab, Bitbucket,
    Azure DevOps and Gitea APIs.
  • Git pull-request mode (branch and PR per DAG) or direct push, environment promotion
    (validated for the target environment like a deploy), PR labels, reviewers and drafts, and
    GitHub App authentication.
  • Deploy target dialog: discovers where Airflow reads DAGs (DAG folder,
    GitDagBundles, a git-sync checkout) and switches without a restart. Credentials always come
    from Airflow connections: shared, per user, or forced by an administrator.
  • Git tokens typed per deploy: the deploy, promote, delete, conflict-retry and bulk-import
    dialogs accept the user's own token, sent only in the X-AutoAir-Git-Token header of that
    request and never stored, logged or audited; HTTPS or loopback only. Settings
    storage.git.request_token (off / optional / required),
    request_token_overrides_forced, request_token_allow_insecure and mcp_request_token.
  • Deploy governance: a mandatory context block and X-AutoAir-* git trailers on every
    commit and PR, plus commit, PR and branch templates and a live preview.
  • ETag optimistic locking with a three-way semantic merge dialog; protection of ejected
    (hand-edited) files; version history with a visual graph diff.
  • Test a task or a whole DAG without the metadata database; trigger, backfill (dry run),
    clear and pause/unpause; a run overlay with logs and XComs.
  • Template gallery, reusable fragments and operator templates; signed (Ed25519) template
    marketplaces (autoair marketplace keygen|build|verify).
  • Operator template editor, available in every editor mode from the palette, the template
    gallery, the command palette and the admin panel: catalog operator picker, parameter form,
    a Default / Locked / Required switch per field, live validation, a code preview, and a
    version history with a diff (GET /api/templates/{id}/versions[/{version}],
    templates_history_limit). Saves and deletes take If-Match / If-None-Match: * and
    answer 409 on a conflict; another team's template needs admin:config.

Collaboration, identity and access

  • Real-time co-editing (Yjs): presence, remote cursors and selections, per-user undo, and
    WebSocket access through short-lived tickets that require DAG edit permission.
  • IAM on top of Airflow's auth manager: built-in and custom roles; airflow_permissions,
    role_mapping and custom_view strategies (FAB, Keycloak, SimpleAuthManager); four-eyes
    approvals; direct-push roles; autoair iam roles|sync-fab; an audit trail with CSV export.
  • Standalone sign-in through the Airflow UI session (SSO), with Airflow username and
    password, or with an API token; every token is verified by Airflow, and sign-out is
    propagated.
  • Runtime feature flags: the admin panel's Features tab (GET/PUT /api/admin/features)
    switches features.* without a restart. Overrides are shared by replicas through the state
    backend (features_override_ttl) and audited with a reason; features_locked and
    AUTOAIR_FEATURES__<NAME> lock a flag.
  • Admin panel: overview, configuration, audit browser, replicas, templates, users, AI usage
    and rules, MCP servers, proposals, catalog, knowledge base, database and caches; audit
    export and scan sizes are configurable (admin.export_limit, admin.export_default,
    admin.scan_limit, admin.usage_scan_limit).

AI and MCP

  • AI on pydantic-ai as an optional extra (autoair[ai] plus one
    provider extra: ai-anthropic, ai-openai, ai-google, ai-ollama, ai-bedrock,
    ai-groq, ai-mistral, ai-cohere, or ai-all). Providers anthropic, openai,
    azure, gemini, google-cloud, ollama, bedrock, groq, mistral, cohere and
    OpenAI-compatible servers. Without the extra, AI is reported off with the reason
    (GET /api/meta → ai_unavailable_reason, GET /api/ai/status → reason).
  • AI assistance: generate a DAG from a prompt, edit with AI (reviewed as a diff), explain and
    fix problems, review, write docs, troubleshoot runs. Central or per-user models (per-user
    keys are encrypted); ai.llm_conn_id and per-user connections read Airflow's common.ai
    pydanticai connection format.
  • Agents on pydantic-ai with typed tools, structured output, streaming...
Read more