Add IAsyncEnumerable pagination support (#58)#70
Merged
Conversation
…tions (#58) - Add IPageable<T> interface for page types with Value + NextLink - Add ConnectorPageable<TPage, TItem> implementing IAsyncEnumerable<TItem> with AsPages() for page-level access - Update BlobMetadataPage and GetMessagesFromConversationResponse to implement IPageable<T> - Change ListFolderAsync, GetMessagesFromChannelAsync, and GetMessagesFromChatAsync to return ConnectorPageable (breaking change) - Fix CallConnectorAsync in all 6 connectors to support absolute NextLink URLs via Uri.IsWellFormedUriString - Add 12 new tests: 8 for ConnectorPageable, 4 for OneDrive pagination integration (single-page, multi-page, AsPages, absolute URL) Closes #58 Co-authored-by: Dobby <dobby@microsoft.com>
Member
Author
|
Companion samples PR demonstrating the pagination pattern: Azure/Connectors-NET-Samples#25 |
There was a problem hiding this comment.
Pull request overview
Adds SDK-wide async pagination support so generated connector operations can be consumed via await foreach, aligning with Azure SDK pagination patterns and enabling automatic nextLink traversal.
Changes:
- Introduces
IPageable<T>andConnectorPageable<TPage, TItem>to support item- and page-level async iteration over paginated connector responses. - Updates select generated connector operations to return
ConnectorPageableinstead ofTask<Page>and wires upnextLinktraversal. - Updates generated
CallConnectorAsyncURL construction to accept absolutenextLinkURLs and adds unit tests covering pagination behavior.
Reviewed changes
Copilot reviewed 4 out of 10 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/Microsoft.Azure.Connectors.Sdk.Tests/OnedriveforbusinessClientTests.cs | Adds OneDrive folder listing pagination tests (single/multi-page, AsPages(), absolute nextLink behavior). |
| tests/Microsoft.Azure.Connectors.Sdk.Tests/ConnectorPageableTests.cs | New unit tests for ConnectorPageable enumeration semantics, constructor validation, and cancellation token propagation. |
| src/Microsoft.Azure.Connectors.Sdk/IPageable.cs | Adds a public interface describing a paginated page shape (Value + NextLink). |
| src/Microsoft.Azure.Connectors.Sdk/ConnectorPageable.cs | Adds IAsyncEnumerable<T> implementation that auto-follows NextLink, plus AsPages() for page-level iteration. |
| src/Microsoft.Azure.Connectors.Sdk/Generated/TeamsExtensions.cs | Makes messages list response pageable and updates message listing methods to return ConnectorPageable; adjusts URL building for absolute paths. |
| src/Microsoft.Azure.Connectors.Sdk/Generated/SharepointonlineExtensions.cs | Updates URL building to support absolute paths/nextLinks. |
| src/Microsoft.Azure.Connectors.Sdk/Generated/OnedriveforbusinessExtensions.cs | Makes folder listing response pageable, changes ListFolderAsync to return ConnectorPageable, and updates URL building for absolute paths/nextLinks. |
| src/Microsoft.Azure.Connectors.Sdk/Generated/Office365Extensions.cs | Updates URL building to support absolute paths/nextLinks. |
| src/Microsoft.Azure.Connectors.Sdk/Generated/MsgraphgroupsanduserExtensions.cs | Updates URL building to support absolute paths/nextLinks. |
| src/Microsoft.Azure.Connectors.Sdk/Generated/KustoExtensions.cs | Updates URL building to support absolute paths/nextLinks. |
daviburg
commented
Apr 29, 2026
- Revert all Generated/ file changes (must go through BPM generator) - Change IPageable<T>.Value from List<T> to IReadOnlyList<T> - Fix nextPageFunc doc: clarify URL is as-returned by service - Add cancellationToken.ThrowIfCancellationRequested() in item and page iteration loops for prompt cancellation within large pages - Add .ConfigureAwait(continueOnCapturedContext: false) to all await foreach loops in tests - Fix useless variable in CancellationToken test: use explicit GetAsyncEnumerator + MoveNextAsync instead of unused loop variable - Remove 4 OneDrive integration tests that depended on generated code changes (will be re-added with companion BPM PR) Co-authored-by: Dobby <dobby@microsoft.com>
Regenerated using CodefulSdkGenerator with pagination support: - OneDrive: BlobMetadataPage implements IPageable<BlobMetadata>, ListFolderAsync returns ConnectorPageable - Teams: GetMessagesFromConversationResponse implements IPageable<ChatMessage>, GetMessagesFromChannelAsync and GetMessagesFromChatAsync return ConnectorPageable - All connectors: ResolveUrl() with SSRF host validation, ManagedIdentityCredential updated to new API Co-authored-by: Dobby <dobby@microsoft.com>
Member
Author
|
Companion BPM generator PR: AzureUX-BPM#15561655. Generated code now regenerated from the updated generator — pagination types, ConnectorPageable returns, ResolveUrl with SSRF host validation, and ManagedIdentityCredential fix all included. |
Generated code (from BPM generator update): - Remove async/await wrapper from ConnectorPageable lambdas (direct Task<T> delegation avoids unnecessary state machine) - Add scheme validation to ResolveUrl (validates both scheme and host, not just host, to prevent protocol downgrade attacks) - Expand 'ct' to 'cancellationToken' in generated lambda parameters Test fixes: - Expand 'ct' to 'cancellationToken' in all test lambdas - Add try/finally with DisposeAsync for IAsyncEnumerator in CancellationToken test Co-authored-by: Dobby <dobby@microsoft.com>
- Remove async/await wrappers from ConnectorPageable lambdas (direct Task<T> delegation, no state machine overhead) - Add scheme validation to ResolveUrl (validates both scheme and host to prevent protocol downgrade) - Expand 'ct' to 'cancellationToken' in generated lambda parameters Co-authored-by: Dobby <dobby@microsoft.com>
…om non-paginated connectors Co-authored-by: Dobby <dobby@microsoft.com>
This was referenced Apr 30, 2026
daviburg
added a commit
that referenced
this pull request
Apr 30, 2026
- Generated Office365usersExtensions.cs and AzureloganalyticsExtensions.cs via CodefulSdkGenerator - Updated ConnectorNames.cs and ManagedConnectors.cs (alphabetical order) - Added Office365usersClientTests.cs (12 tests: constructor, dispose, mocked API, error handling, serialization) - Updated README.md validated connectors table - Updated connection-setup skill connector list Depends on #70 for IPageable<T> and ConnectorPageable<TPage,TItem> pagination abstractions. Resolves #8
daviburg
added a commit
that referenced
this pull request
Apr 30, 2026
- Generated Office365usersExtensions.cs and AzureloganalyticsExtensions.cs via CodefulSdkGenerator - Updated ConnectorNames.cs and ManagedConnectors.cs (alphabetical order) - Added Office365usersClientTests.cs (12 tests: constructor, dispose, mocked API, error handling, serialization) - Updated README.md validated connectors table - Updated connection-setup skill connector list Depends on #70 for IPageable<T> and ConnectorPageable<TPage,TItem> pagination abstractions. Resolves #8
daviburg
added a commit
that referenced
this pull request
Apr 30, 2026
- Generated Office365usersExtensions.cs and AzureloganalyticsExtensions.cs via CodefulSdkGenerator - Updated ConnectorNames.cs and ManagedConnectors.cs (alphabetical order) - Added Office365usersClientTests.cs (12 tests: constructor, dispose, mocked API, error handling, serialization) - Updated README.md validated connectors table - Updated connection-setup skill connector list Depends on #70 for IPageable<T> and ConnectorPageable<TPage,TItem> pagination abstractions. Resolves #8
daviburg
added a commit
that referenced
this pull request
Apr 30, 2026
## Summary Adds the **Office 365 Users** (\office365users\) connector client to the SDK, generated via CodefulSdkGenerator. ### Changes **Generated code:** - \Office365usersExtensions.cs\ — typed client with 12 operations: MyProfile, UserProfile, Manager, DirectReports, SearchUser, RelevantPeople, TrendingDocuments, UserPhoto, UpdateMyProfile, UpdateMyPhoto, HttpRequest - \AzureloganalyticsExtensions.cs\ — also generated (was missing from source, already in NuGet package) **Registration:** - Updated \ConnectorNames.cs\ and \ManagedConnectors.cs\ (alphabetical order) **Tests:** - 12 new tests in \Office365usersClientTests.cs\: constructor, dispose, mocked API (MyProfile, UserProfile, Manager, DirectReports), error handling, serialization round-trips (GraphUser, User, Person), IPageable interface verification - All **152 tests pass** (0 failures, includes 8 pagination tests from #70) **Docs:** - Updated README validated connectors table - Updated connection-setup skill connector list ### E2E Validation - Connection created and consented on \sdk-test-gateway\ (brazilsouth) - \GetMyProfile\ and \GetManager\ tested locally via \unc start\ - Rebased on main after #70 merged — pagination types (\IPageable<T>\, \ConnectorPageable\) come from #70 Resolves #8
daviburg
added a commit
that referenced
this pull request
May 1, 2026
## Summary Add comprehensive unit tests and documentation for the **azureloganalytics** connector (Phase 3.2 - Azure Services). > **Note:** The generated code (\AzureloganalyticsExtensions.cs\), connector registration (\ConnectorNames.cs\, \ManagedConnectors.cs\), and pagination infrastructure (\IPageable.cs\, \ConnectorPageable.cs\) were merged to main as part of #70 and #75. This PR adds the remaining pieces. ## What's included ### Tests - 13 unit tests in \AzureloganalyticsClientTests.cs\: constructor, dispose, mocked API calls, error handling, serialization round-trips, multi-page pagination - All 167 tests pass (expanded from 145 baseline after #70 and #75 merged) ### Docs - Updated \README.md\ validated connectors table (added Azure Log Analytics with generated operations) - Updated connection-setup skill supported names list (added \�zureloganalytics\) ## E2E Validation Connection \�zureloganalytics-test\ created and consented on the \sdk-test-gateway\ Connector Gateway in \�razilsouth\. ## Merge sequencing This PR is now **ready to merge** — no remaining dependencies. Rebased onto main after #70 (pagination) and #75 (Office 365 Users) merged. Closes #8 Co-authored-by: Dobby <dobby@microsoft.com>
daviburg
added a commit
to Azure/Connectors-NET-Samples
that referenced
this pull request
May 1, 2026
* Update ListOneDriveFolder to use IAsyncEnumerable pagination - ListFolderAsync now returns IAsyncEnumerable<BlobMetadata> that automatically follows NextLink across all pages - Replace manual page.Value/page.NextLink with await foreach pattern - Bump SDK to 0.6.0-preview.1.dev (local build with pagination support) - Add local NuGet source for pre-release validation Depends on: Azure/Connectors-NET-SDK#70 Co-authored-by: Dobby <dobby@microsoft.com> * Add GetChannelMessages demonstrating IAsyncEnumerable pagination E2E - New GetChannelMessages function uses await foreach to iterate Teams channel messages across all pages automatically - Validated E2E: 37 messages returned across 2 pages (20 + 17) from live Teams connector with automatic NextLink following Co-authored-by: Dobby <dobby@microsoft.com> * Fix CI: revert to published SDK version, remove local NuGet source Reverts function code to pre-pagination pattern so CI builds with the currently published SDK (0.6.0-preview.1). The pagination samples (await foreach on ListFolderAsync, GetChannelMessages) will be re-added after SDK v0.7.0-preview.1 is published. Co-authored-by: Dobby <dobby@microsoft.com> * Revert "Fix CI: revert to published SDK version, remove local NuGet source" This reverts commit a0833e9. * Update to published SDK 0.7.0-preview.1 from nuget.org, remove local source Co-authored-by: Dobby <dobby@microsoft.com> --------- Co-authored-by: Dobby <dobby@microsoft.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements
IAsyncEnumerable<T>auto-pagination for paginated connector operations, following the Azure SDK pagination standard.Closes #58
Breaking Changes
This is a pre-release breaking change (no customer previews shipped yet):
OnedriveforbusinessClient.ListFolderAsyncTask<BlobMetadataPage>ConnectorPageable<BlobMetadataPage, BlobMetadata>TeamsClient.GetMessagesFromChannelAsyncTask<GetMessagesFromConversationResponse>ConnectorPageable<GetMessagesFromConversationResponse, ChatMessage>TeamsClient.GetMessagesFromChatAsyncTask<GetMessagesFromConversationResponse>ConnectorPageable<GetMessagesFromConversationResponse, ChatMessage>What Changed
New Infrastructure
IPageable<T>— Interface for page types withValue+NextLinkConnectorPageable<TPage, TItem>—IAsyncEnumerable<TItem>that followsNextLinkautomatically, withAsPages()for page-level accessPaginated Methods (Breaking)
ListFolderAsync,GetMessagesFromChannelAsync,GetMessagesFromChatAsyncnow returnConnectorPageableinstead ofTask<Page>Bug Fix (All Connectors)
CallConnectorAsyncin all 6 connectors now supports absolute NextLink URLs viaUri.IsWellFormedUriString— previously would incorrectly prependconnectionRuntimeUrlto absolute URLsTests
ConnectorPageable(empty/null pages, single/multi-page, AsPages, cancellation, constructor validation)Usage
Cross-Language Design
This pattern maps directly to all three SDK languages:
await foreach/IAsyncEnumerable<T>async for/__aiter__for await...of/Symbol.asyncIteratorSamples
Companion PR in Connectors-NET-Samples will demonstrate the pattern end-to-end.