Skip to content

Migrate the rest of python unit tests to conformance tests#1266

Merged
nan-yu merged 18 commits intogoogle:mainfrom
nan-yu:conformance-test
Apr 29, 2026
Merged

Migrate the rest of python unit tests to conformance tests#1266
nan-yu merged 18 commits intogoogle:mainfrom
nan-yu:conformance-test

Conversation

@nan-yu
Copy link
Copy Markdown
Collaborator

@nan-yu nan-yu commented Apr 23, 2026

Description

Replace this paragraph with a description of what this PR is changing or adding, and why. Consider including before/after screenshots.

List which issues are fixed by this PR. For larger changes, raising an issue first helps reduce redundant work.

Pre-launch Checklist

If you need help, consider asking for advice on the discussion board.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the initial implementation of the A2UI C++ SDK, covering schema management, validation, and a streaming parser. It also expands the shared conformance test suite and updates the Python SDK tests to leverage these new cases. Feedback on the C++ implementation suggests ensuring consistent error handling in catalog selection to match conformance expectations, adopting modern memory management with std::unique_ptr in tests, and using custom delimiters for embedded raw strings to prevent parsing errors. Additionally, improvements were recommended for optimizing component lookup complexity and removing an unused dependency in the CMake configuration.

Comment thread agent_sdks/cpp/src/schema/manager.cc Outdated
Comment thread agent_sdks/cpp/tests/test_conformance.cc Outdated
Comment thread agent_sdks/cpp/scripts/generate_embedded_schemas.py Outdated
Comment thread agent_sdks/cpp/src/parser/streaming_impl.cc Outdated
Comment thread agent_sdks/cpp/CMakeLists.txt Outdated
@nan-yu
Copy link
Copy Markdown
Collaborator Author

nan-yu commented Apr 24, 2026

@jiahaog I cannot add you as a reviewer, but feel free to take a look.

I may need to rebase if #1129 is submitted first.

Copy link
Copy Markdown
Member

@jiahaog jiahaog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, just minor comments.

Comment thread agent_sdks/conformance/conformance_schema.json
Comment thread agent_sdks/conformance/conformance_schema.json Outdated
Comment thread agent_sdks/conformance/conformance_schema.json Outdated
Comment thread agent_sdks/conformance/conformance_schema.json Outdated
Copy link
Copy Markdown
Collaborator

@jacobsimionato jacobsimionato left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! The shared validations approach looks fantastic to me.

More general comment about the project: The agent SDKs are growing and maturing quickly with new languages supported and this sophisticated testing infrastructure. I think we should step back and revisit the design soon to make sure it incorporates most of the pressing needs that we are aware of before before scaling to more languages or deeper infra.

My two main concerns are:

  • APIs to model different inference strategies in a somewhat interchangeable way
  • Integration with rendering SDKs (e.g. the "core" SDK) for the purposes of:
    • Client orchestrated inference
    • Server side rendering, validation, compaction of updates

I just shared draft https://docs.google.com/document/d/1VxcAaUh2sMh14soGVQEkRwz5enA6j4tzDPWJZK9tUqA/edit?resourcekey=0-Ban02LRlWm__B7yOkm8gmA&tab=t.0#heading=h.c0uts5ftkk58 describing the latter

@nan-yu nan-yu force-pushed the conformance-test branch from 06efb7e to ea458b1 Compare April 27, 2026 20:01
@nan-yu
Copy link
Copy Markdown
Collaborator Author

nan-yu commented Apr 27, 2026

LGTM! The shared validations approach looks fantastic to me.

More general comment about the project: The agent SDKs are growing and maturing quickly with new languages supported and this sophisticated testing infrastructure. I think we should step back and revisit the design soon to make sure it incorporates most of the pressing needs that we are aware of before before scaling to more languages or deeper infra.

My two main concerns are:

  • APIs to model different inference strategies in a somewhat interchangeable way

  • Integration with rendering SDKs (e.g. the "core" SDK) for the purposes of:

    • Client orchestrated inference
    • Server side rendering, validation, compaction of updates

I just shared draft https://docs.google.com/document/d/1VxcAaUh2sMh14soGVQEkRwz5enA6j4tzDPWJZK9tUqA/edit?resourcekey=0-Ban02LRlWm__B7yOkm8gmA&tab=t.0#heading=h.c0uts5ftkk58 describing the latter

I see your point that the underlying architecture would change after integration with the rendering SDK, but I'd still want to get this in to unblock the c++ validator. It also stands as an example on how to set up the test harness.

@nan-yu nan-yu force-pushed the conformance-test branch from ea458b1 to 8dbf517 Compare April 28, 2026 18:05
@nan-yu nan-yu force-pushed the conformance-test branch from 8dbf517 to 8493bf8 Compare April 28, 2026 18:13
@nan-yu nan-yu force-pushed the conformance-test branch from 8493bf8 to 19b999e Compare April 28, 2026 18:19
@nan-yu nan-yu merged commit 13902a3 into google:main Apr 29, 2026
12 checks passed
@github-project-automation github-project-automation Bot moved this from Todo to Done in A2UI Apr 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants