Skip to content

Add Data Factory tools to Fabric MCP Server#2584

Merged
AmosHersch merged 21 commits into
microsoft:mainfrom
Ebram-Tawfik:integrate-datafactory-mcp
May 11, 2026
Merged

Add Data Factory tools to Fabric MCP Server#2584
AmosHersch merged 21 commits into
microsoft:mainfrom
Ebram-Tawfik:integrate-datafactory-mcp

Conversation

@Ebram-Tawfik
Copy link
Copy Markdown
Contributor

@Ebram-Tawfik Ebram-Tawfik commented May 5, 2026

Summary

Adds Data Factory pipeline and dataflow management tools to the Fabric MCP Server, consuming the shared handler layer from the Microsoft.DataFactory.MCP.Core NuGet package.

New Commands

Command Description
list-pipelines List all pipelines in a workspace
create-pipeline Create a new pipeline
get-pipeline Get pipeline details by ID
run-pipeline Trigger a pipeline run
list-dataflows List all dataflows in a workspace
create-dataflow Create a new dataflow
execute-query Execute an M (Power Query) expression against a dataflow

Architecture

  • Thin GlobalCommand<T> wrappers following existing Fabric MCP patterns
  • Business logic delegated to PipelineHandler / DataflowHandler / DataflowQueryHandler from the NuGet Core package
  • Registered via DataFactoryAreaSetup : IAreaSetup in Program.cs
  • AOT-compatible with custom JsonSerializerContext

Changes

  • New project: tools/Fabric.Mcp.Tools.DataFactory/ — 7 commands, options, models, area setup
  • Program.cs: Register DataFactoryAreaSetup in area list
  • Fabric.Mcp.Server.slnx: Add project to solution
  • Directory.Packages.props: Add Microsoft.DataFactory.MCP.Core version entry (0.20.0-beta)
  • Cleanup: Removed submodule (replaced by NuGet package reference)

Copilot AI review requested due to automatic review settings May 5, 2026 19:52
@Ebram-Tawfik Ebram-Tawfik requested review from a team as code owners May 5, 2026 19:52
@github-project-automation github-project-automation Bot moved this to Untriaged in Azure MCP Server May 5, 2026
@Ebram-Tawfik Ebram-Tawfik force-pushed the integrate-datafactory-mcp branch from baefdd2 to ddb2304 Compare May 5, 2026 19:55
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds the DataFactory.MCP.Core submodule and wires it into Fabric.Mcp.Server as a project reference, along with centrally-managed package versions required to build the integrated dependency.

Changes:

  • Add DataFactory.MCP.Core project reference to Fabric.Mcp.Server
  • Add external/DataFactory.MCP git submodule configuration (pinned commit + .gitmodules)
  • Add missing central package versions needed by DataFactory.MCP.Core

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
servers/Fabric.Mcp.Server/src/Fabric.Mcp.Server.csproj Adds a project reference to the DataFactory MCP core project in the new submodule
external/DataFactory.MCP Pins the submodule to a specific commit for reproducible builds
Directory.Packages.props Adds central package versions required by the submodule project
.gitmodules Registers the DataFactory.MCP Git submodule source and path

Comment thread servers/Fabric.Mcp.Server/src/Fabric.Mcp.Server.csproj Outdated
Comment thread servers/Fabric.Mcp.Server/src/Fabric.Mcp.Server.csproj Outdated
Comment thread .gitmodules Outdated
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 5, 2026

Thank you for your contribution @Ebram-Tawfik! We will review the pull request and get back to you soon.

@Ebram-Tawfik Ebram-Tawfik force-pushed the integrate-datafactory-mcp branch from aa7af73 to d7fca50 Compare May 5, 2026 20:04
@Ebram-Tawfik Ebram-Tawfik changed the title Add DataFactory.MCP.Core project reference to Fabric.Mcp.Server Add DataFactory.MCP.Core as submodule reference in Fabric.Mcp.Server May 5, 2026
Copy link
Copy Markdown
Contributor

@jongio jongio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR introduces a git submodule to pull in DataFactory.MCP.Core. I have a few concerns that need to be addressed before merging.

CI will break: The build pipeline (eng/pipelines/templates/jobs/build.yml) uses checkout: self without submodules: recursive. Once merged, Fabric.Mcp.Server builds will fail because external/DataFactory.MCP won't exist in CI. The CONTRIBUTING.md update is helpful for local dev, but CI needs the equivalent configuration.

AOT compatibility: DataFactory.MCP.Core doesn't set IsAotCompatible=true. Fabric.Mcp.Server enables AOT/Trim analyzers with TreatWarningsAsErrors=true. Referencing a project without AOT annotations will likely produce analyzer warnings that become build-breaking errors.

Submodule vs NuGet package: Git submodules are fragile in practice - they break git clone without --recurse-submodules, complicate CI checkout steps, create merge conflicts on pointer updates, and confuse contributors. The conventional .NET approach is publishing DataFactory.MCP.Core as a NuGet package (internal feed works fine) and adding a PackageReference. This sidesteps the AOT/build-props conflicts entirely since the package arrives pre-compiled.

Dead code until follow-up: The project reference compiles DataFactory.MCP.Core into the binary (including Apache.Arrow and embedded HTML/CSS/JS resources) but nothing registers its tools via IAreaSetup. The server grows for zero functional benefit right now. Consider combining this with the registration PR so each merge leaves the server in a working incremental state.

I'd suggest either switching to a NuGet package reference, or at minimum addressing the CI and AOT issues first. Happy to discuss tradeoffs.

@github-project-automation github-project-automation Bot moved this from Untriaged to In Progress in Azure MCP Server May 6, 2026
- Add DataFactory.MCP as git submodule at external/DataFactory.MCP
- Add project reference using $(RepoRoot) for portability
- Add Apache.Arrow 22.1.0 to Directory.Packages.props (only new dependency needed)
- Microsoft.Identity.Client and Microsoft.Extensions.Http already satisfied transitively
- Build verified: 0 warnings, 0 errors

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Ebram-Tawfik Ebram-Tawfik force-pushed the integrate-datafactory-mcp branch from d7fca50 to 20655d2 Compare May 6, 2026 18:50
…ory rename

Update submodule pointer and project reference to match the
renamed project in DataFactory.MCP repo.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Ebram-Tawfik Ebram-Tawfik force-pushed the integrate-datafactory-mcp branch from 6488a29 to 64ed49e Compare May 6, 2026 19:37
…Factory

Rename the external submodule path for consistency with Fabric naming
conventions. Updates .gitmodules, project reference, and docs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Ebram-Tawfik Ebram-Tawfik force-pushed the integrate-datafactory-mcp branch from 34653f1 to d84da2c Compare May 6, 2026 20:14
Ebram Tawfik and others added 2 commits May 6, 2026 14:04
The submodule now includes DataFactory.MCP.Fabric which provides
IAreaSetup + GlobalCommand<T> wrappers for direct Fabric MCP Server
integration without needing a separate wrapper project in this repo.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…module

The DataFactory.MCP.Fabric adapter project now lives in the DataFactory
repo itself (submodule), eliminating the need for a separate wrapper
project in this repo.

Changes:
- Delete tools/Fabric.Mcp.Tools.DataFactory/ (redundant wrapper)
- Add explicit ProjectReference to submodule's DataFactory.MCP.Fabric
- Update Program.cs to use DataFactory.MCP.Fabric.DataFactoryAreaSetup
- Fix submodule csproj with conditional paths for Microsoft.Mcp.Core

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik and others added 5 commits May 6, 2026 16:17
- All commands now delegate to PipelineHandler/DataflowHandler
- Commands are thin shims (~15 lines each)
- Handlers registered via AddDataFactoryMcpServices() in submodule
- Update submodule to latest

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace ProjectReference to submodule Core project with PackageReference
to Microsoft.DataFactory.MCP.Core 0.19.0-beta from nuget.org.
Add version entry in Directory.Packages.props for CPM compliance.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
No longer needed - consuming Microsoft.DataFactory.MCP.Core via NuGet package instead.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Ebram-Tawfik Ebram-Tawfik changed the title Add DataFactory.MCP.Core as submodule reference in Fabric.Mcp.Server Add Data Factory tools to Fabric MCP Server May 7, 2026
Ebram Tawfik and others added 3 commits May 6, 2026 21:34
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add ExecuteQueryCommand for executing M (Power Query) expressions against dataflows
- Add ExecuteQueryOptions, DataflowId/QueryName/QueryText option definitions
- Register ExecuteQueryCommand in DI and command group
- Add ExecuteQueryCommandResult to JSON source generation context
- Update Microsoft.DataFactory.MCP.Core package version to 0.19.1-beta

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Updated to version with source-generated JSON for full ILLinker
trim safety (zero IL2026 warnings).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Ebram-Tawfik Ebram-Tawfik requested a review from jongio May 7, 2026 19:30
Ebram Tawfik and others added 4 commits May 7, 2026 12:49
0.20.0-beta exists on nuget.org but hasn't propagated to the
azure-sdk-for-net DevOps feed upstream yet. Using 0.19.1-beta
to unblock CI; will bump once 0.20.0-beta is cached.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Package is now available on the azure-sdk-for-net DevOps feed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The smoke test runs 'fabmcp tools list' and parses stdout as JSON.
AuthenticationStateManager logs an info message during DI construction,
which pollutes stdout and causes ConvertFrom-Json to fail.

Match Azure.Mcp.Server pattern: LogToStandardErrorThreshold = LogLevel.Trace
sends all console logs to stderr, keeping stdout clean for JSON output.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Regenerate Microsoft.Mcp.slnx and Fabric.Mcp.Server.slnx via Update-Solution.ps1
- Fix IMPORTS ordering in DataFactoryAreaSetup.cs (dotnet format)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 10 comments.

Comment thread tools/Fabric.Mcp.Tools.DataFactory/src/DataFactoryAreaSetup.cs
Comment thread servers/Fabric.Mcp.Server/src/Program.cs
Create test project Fabric.Mcp.Tools.DataFactory.UnitTests with 45 tests:
- DataFactoryAreaSetupTests: name, title, service registration
- Command tests for all 7 commands: constructor, metadata, options, null guards
- Covers Pipeline (list, create, get, run) and Dataflow (list, create, execute-query)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

@jongio jongio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addresses my previous feedback. The switch from submodule to NuGet package is the right call - CI is green across all 15 checks and the build story is much simpler.

Commands follow Fabric toolset patterns: sealed classes, primary constructors, CommandMetadata, proper option binding. Area setup registers all 7 commands correctly.

Follow-up items (non-blocking):

  • Unit tests cover constructor, metadata, options, and null guards but don't test ExecuteAsync paths. Other Fabric toolsets (OneLake) include execution success/error tests. Worth adding in a subsequent PR.

  • ExecuteQueryCommandResult.Data is typed as object?, which is tricky for source-generated System.Text.Json under AOT. Native builds pass today, but if the handler returns complex types, serialization could fail at runtime. Worth watching.

- Create tools/Fabric.Mcp.Tools.DataFactory/README.md with full docs
- Add Data Factory tools entry to CHANGELOG under 2.0.0-beta.1
- Add Data Factory Operations table to server README Available Tools
- Add TOC entry and overview bullet for Data Factory integration

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Replace relative path with absolute GitHub URL in DataFactory README
- Remove [[microsoft#2584]] references from CHANGELOG (added by CI after merge)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@AmosHersch AmosHersch enabled auto-merge (squash) May 11, 2026 16:56
@AmosHersch AmosHersch merged commit a6d033a into microsoft:main May 11, 2026
15 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in Azure MCP Server May 11, 2026
srinuthati78 pushed a commit to srinuthati78/mcp that referenced this pull request May 18, 2026
* Add DataFactory.MCP.Core as submodule reference in Fabric.Mcp.Server

- Add DataFactory.MCP as git submodule at external/DataFactory.MCP
- Add project reference using $(RepoRoot) for portability
- Add Apache.Arrow 22.1.0 to Directory.Packages.props (only new dependency needed)
- Microsoft.Identity.Client and Microsoft.Extensions.Http already satisfied transitively
- Build verified: 0 warnings, 0 errors

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Update DataFactory submodule ref after Core→Fabric.Mcp.Tools.DataFactory rename

Update submodule pointer and project reference to match the
renamed project in DataFactory.MCP repo.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Rename submodule folder from DataFactory.MCP to Fabric.Mcp.Tools.DataFactory

Rename the external submodule path for consistency with Fabric naming
conventions. Updates .gitmodules, project reference, and docs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Update DataFactory submodule to include Fabric adapter project

The submodule now includes DataFactory.MCP.Fabric which provides
IAreaSetup + GlobalCommand<T> wrappers for direct Fabric MCP Server
integration without needing a separate wrapper project in this repo.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove redundant wrapper; use DataFactory.MCP.Fabric adapter from submodule

The DataFactory.MCP.Fabric adapter project now lives in the DataFactory
repo itself (submodule), eliminating the need for a separate wrapper
project in this repo.

Changes:
- Delete tools/Fabric.Mcp.Tools.DataFactory/ (redundant wrapper)
- Add explicit ProjectReference to submodule's DataFactory.MCP.Fabric
- Update Program.cs to use DataFactory.MCP.Fabric.DataFactoryAreaSetup
- Fix submodule csproj with conditional paths for Microsoft.Mcp.Core

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Move DataFactory.MCP.Fabric to tools/Fabric.Mcp.Tools.DataFactory

- Copy source files from external submodule to tools/ folder
- Update namespaces from DataFactory.MCP.Fabric to Fabric.Mcp.Tools.DataFactory
- Add global:: prefix for DataFactory.MCP.Core namespace references
- Remove explicit ProjectReference to external csproj (wildcard covers it)
- Update Program.cs to use new namespace
- Add project to solution file
- Update submodule to latest commit

* Update DataFactory commands to use domain handlers

- All commands now delegate to PipelineHandler/DataflowHandler
- Commands are thin shims (~15 lines each)
- Handlers registered via AddDataFactoryMcpServices() in submodule
- Update submodule to latest

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Switch DataFactory.MCP.Core from project reference to NuGet package

Replace ProjectReference to submodule Core project with PackageReference
to Microsoft.DataFactory.MCP.Core 0.19.0-beta from nuget.org.
Add version entry in Directory.Packages.props for CPM compliance.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove DataFactory.MCP submodule

No longer needed - consuming Microsoft.DataFactory.MCP.Core via NuGet package instead.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove .gitmodules and CONTRIBUTING.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Restore CONTRIBUTING.md from main

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Replace placeholder command IDs with proper GUIDs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add execute-query command and update NuGet to 0.19.1-beta

- Add ExecuteQueryCommand for executing M (Power Query) expressions against dataflows
- Add ExecuteQueryOptions, DataflowId/QueryName/QueryText option definitions
- Register ExecuteQueryCommand in DI and command group
- Add ExecuteQueryCommandResult to JSON source generation context
- Update Microsoft.DataFactory.MCP.Core package version to 0.19.1-beta

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: bump Microsoft.DataFactory.MCP.Core to 0.20.0-beta

Updated to version with source-generated JSON for full ILLinker
trim safety (zero IL2026 warnings).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Use DataFactory.MCP.Core 0.19.1-beta (available on DevOps feed)

0.20.0-beta exists on nuget.org but hasn't propagated to the
azure-sdk-for-net DevOps feed upstream yet. Using 0.19.1-beta
to unblock CI; will bump once 0.20.0-beta is cached.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Bump DataFactory.MCP.Core to 0.20.0-beta

Package is now available on the azure-sdk-for-net DevOps feed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix smoke test: redirect console logs to stderr

The smoke test runs 'fabmcp tools list' and parses stdout as JSON.
AuthenticationStateManager logs an info message during DI construction,
which pollutes stdout and causes ConvertFrom-Json to fail.

Match Azure.Mcp.Server pattern: LogToStandardErrorThreshold = LogLevel.Trace
sends all console logs to stderr, keeping stdout clean for JSON output.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix analyze: update solution files and import ordering

- Regenerate Microsoft.Mcp.slnx and Fabric.Mcp.Server.slnx via Update-Solution.ps1
- Fix IMPORTS ordering in DataFactoryAreaSetup.cs (dotnet format)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add unit tests for DataFactory tools

Create test project Fabric.Mcp.Tools.DataFactory.UnitTests with 45 tests:
- DataFactoryAreaSetupTests: name, title, service registration
- Command tests for all 7 commands: constructor, metadata, options, null guards
- Covers Pipeline (list, create, get, run) and Dataflow (list, create, execute-query)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: add DataFactory README, update CHANGELOG and server README

- Create tools/Fabric.Mcp.Tools.DataFactory/README.md with full docs
- Add Data Factory tools entry to CHANGELOG under 2.0.0-beta.1
- Add Data Factory Operations table to server README Available Tools
- Add TOC entry and overview bullet for Data Factory integration

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: fix broken relative link, remove PR refs from CHANGELOG

- Replace relative path with absolute GitHub URL in DataFactory README
- Remove [[microsoft#2584]] references from CHANGELOG (added by CI after merge)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Ebram Tawfik <ebramtawfik@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants