Add PydanticAIHook to common.ai provider

[kaxil]

Adds PydanticAIHook to the common.ai provider — a hook for LLM access via pydantic-ai. This ships the connection and hook foundation for AIP-99. Future PRs will add operators (LLMSQLQueryOperator) and decorators (@task.llm_sql_query) on top.

The hook handles Airflow connection credentials and creates pydantic-ai Model and Agent objects. It works with any provider pydantic-ai supports: OpenAI, Anthropic, Google, Bedrock, Groq, Mistral, Ollama, vLLM, etc.

Usage

from airflow.sdk import dag, task
from airflow.providers.common.ai.hooks.pydantic_ai import PydanticAIHook

@dag(schedule=None)
def my_llm_pipeline():
    @task
    def summarize(text: str) -> str:
        hook = PydanticAIHook(llm_conn_id="openai_default", model_id="openai:gpt-5")
        agent = hook.create_agent(output_type=str, instructions="Summarize concisely.")
        result = agent.run_sync(text)
        return result.output

Connection fields:

Field	Purpose	Example
Password	API key	sk-...
Host	Base URL (optional)	http://localhost:11434/v1 for Ollama
Extra JSON	Model identifier	{"model": "openai:gpt-5"}

Cloud providers (Bedrock, Vertex) that use native auth chains leave password empty — pydantic-ai picks up AWS_PROFILE, GOOGLE_APPLICATION_CREDENTIALS, etc. automatically.

Why these choices

get_conn() returns pydantic-ai Model, not Agent or Connection. Airflow convention is that get_conn() returns a reusable SDK client (OpenAIHook → OpenAI client, DbApiHook → DBAPI connection). A pydantic-ai Model is the connection-level object (credentials + model ID). An Agent is session-level (binds a model to task-specific config), so it lives in create_agent().

No abstract LLMHook base class. Every Airflow LLM hook (OpenAIHook, CohereHook, GenAIHook) extends BaseHook directly. LLMs don't share a stable interface beyond "send text, get text" — divergence starts immediately with structured output, tools, streaming, vision. Pydantic-ai's Model protocol already handles abstraction. We can extract a base class later if a second framework creates real evidence of what a shared interface should look like.

Credential injection via provider_factory. infer_model() doesn't accept api_key/base_url directly — it takes a provider_factory callback that creates provider instances with credentials. Google Vertex/GLA are special-cased since they use ADC and don't accept api_key.

Known issues

CI image build conflict: pydantic-ai-slim requires opentelemetry-api>=1.28.0, which transitively pulls protobuf>=5.0. This conflicts with yandexcloud (protobuf<5) when all providers are installed together with --resolution highest. Same issue was identified in #61794. The provider is state: not-ready so this doesn't affect releases — it only affects the CI all-providers image build. Waiting for https://lists.apache.org/thread/qbx1b8p3296z5pj1hlg3qfggftgjw4m3

Co-Authored-By: GPK gopidesupavan@gmail.com

[gopidesupavan]

Cool thanx kaxil :)

[kaxil]

Cool thanx kaxil :)

All your hard work man 🙏