Typing experiments

[dargilco]

Description

Please add an informative description that covers that changes made by the pull request and link all relevant issues.

If an SDK is being regenerated based on a new API spec, a link to the pull request containing these API spec changes should be included above.

All SDK Contribution checklist:

• The pull request does not introduce [breaking changes]
• CHANGELOG is updated for new features, bug fixes or other significant changes.
• I have read the contribution guidelines.

General Guidelines and Best Practices

• Title of the pull request is clear and informative.
• There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

• Pull request includes test coverage for the included changes.

[Copilot]

Pull request overview

Adds a typing “patch” layer for azure-ai-projects so the OpenAI client returned by AIProjectClient.get_openai_client() can accept Azure-specific grader types for evals.create(), and updates a local typing experiment script to use the new type.

Changes:

• Added a new _patch.pyi type stub to widen evals.create(testing_criteria=...) accepted types.
• Updated get_openai_client return annotation in _patch.py to reference OpenAI directly.
• Updated the typing experiment sample to use find_dotenv() and to reference AzureAIGraderCoherenceParam from _patch.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File	Description
sdk/ai/azure-ai-projects/typing-experiments/modified_sample_evaluations_graders.py	Updates the experiment script to use the new grader TypedDict and tweaks dotenv loading; currently contains a runtime import issue and double-create resource leak.
sdk/ai/azure-ai-projects/azure/ai/projects/_patch.pyi	Introduces the type stub that widens accepted grader types and re-types get_openai_client(); packaging/syntax compatibility concerns noted.
sdk/ai/azure-ai-projects/azure/ai/projects/_patch.py	Minor signature typing change for get_openai_client(); one now-redundant type: ignore remains.

[Copilot]

This script creates an evaluation twice (client.evals.create(...) is called twice) but only deletes the second one (the first eval_object is overwritten). This will leave an evaluation resource behind and can create clutter/cost. Consider deleting both evaluations (track both ids) or remove the redundant first create call.

Suggested change

	testing_criteria=testing_criteria,
	)
	eval_object = client.evals.create(
	name="OpenAI graders test",
	data_source_config=data_source_config,
	testing_criteria=[
	AzureAIGraderCoherenceParam(
	type="azure_ai_evaluator",
	name="coherence",
	evaluator_name="builtin.coherence",
	initialization_parameters={"deployment_name": f"{model_deployment_name}"},
	data_mapping={"query": "{{item.query}}", "response": "{{item.response}}"},
	)],
	testing_criteria=[
	AzureAIGraderCoherenceParam(
	type="azure_ai_evaluator",
	name="coherence",
	evaluator_name="builtin.coherence",
	initialization_parameters={"deployment_name": f"{model_deployment_name}"},
	data_mapping={"query": "{{item.query}}", "response": "{{item.response}}"},
	)
	],

[Copilot]

This stub uses PEP 604 unions (e.g., Headers | None, str | Omit). Since this package supports Python 3.9, some type-checking setups targeting 3.9 will reject this syntax. Prefer Union[...] / Optional[...] forms in the .pyi to keep it compatible with Python 3.9 type checkers.

Suggested change

	metadata: Optional[Metadata] | Omit = ...,
	name: str | Omit = ...,
	extra_headers: Headers | None = None,
	extra_query: Query | None = None,
	extra_body: Body | None = None,
	timeout: float | httpx.Timeout | None | NotGiven = ...,
	metadata: Union[Optional[Metadata], Omit] = ...,
	name: Union[str, Omit] = ...,
	extra_headers: Optional[Headers] = None,
	extra_query: Optional[Query] = None,
	extra_body: Optional[Body] = None,
	timeout: Union[float, httpx.Timeout, None, NotGiven] = ...,

[Copilot]

Adding _patch.pyi may not affect consumers unless it is included in the published sdist/wheel. In this package, MANIFEST.in only includes py.typed and pyproject.toml's [tool.setuptools.package-data] only lists py.typed, so the new .pyi is likely not shipped. Please update packaging config to include azure/ai/projects/_patch.pyi (or *.pyi) as package data.

[Copilot]

OpenAI is imported in this module (from openai import OpenAI), so # type: ignore[name-defined] on the return annotation is no longer necessary and may hide real type issues. Consider removing the type: ignore[name-defined] here.

Suggested change

	def get_openai_client(self, **kwargs: Any) -> OpenAI: # type: ignore[name-defined] # pylint: disable=too-many-statements
	def get_openai_client(self, **kwargs: Any) -> OpenAI: # pylint: disable=too-many-statements

[Copilot]

AzureAIGraderCoherenceParam is imported from azure.ai.projects._patch, but it only exists in _patch.pyi (type stub) and is not defined in _patch.py, so this import will fail at runtime. Either define/export AzureAIGraderCoherenceParam in _patch.py (e.g., as a TypedDict) or keep the TypedDict definition local to this script (or gate the import behind TYPE_CHECKING and avoid using it at runtime).