Skip to content
This repository has been archived by the owner on Jun 18, 2024. It is now read-only.

DataFlow Validation not working with Data Explorer #4

Closed
luigibrancati opened this issue Dec 2, 2021 · 1 comment
Closed

DataFlow Validation not working with Data Explorer #4

luigibrancati opened this issue Dec 2, 2021 · 1 comment

Comments

@luigibrancati
Copy link

luigibrancati commented Dec 2, 2021

Hello everyone,

Sorry if this is the wrong place, I couldn't find anywhere else where to open an issue/bug regarding the package @microsoft/azure-data-factory-utilities. I noticed it is used here too, so I figured this might be the best place to post.

In my company we've been using the azure-data-factory-utilities package to validate our datafactory on Azure before building it and then automatically deploying the arm templates, more specifically we use run build validate <datafactory> to validate our datafactory.
We recently run into an issue with this package. We tried to deploy a new pipeline which has a DataFlow activity that uses a few Data Explorer linked services (both as source and sink). While on ADF everything runs and validates fine, the packages doesn't correctly validate the new pipeline and returns the following error

 CmdApiApp: Starting to validate all resources
 ModelService: _backgroundFetch - finished, duration: 2225.322690999994, total nameds: 236
 ERROR === CmdApiApp: 
Data Factory validation failed. Found 2 validation errors.

 ERROR === CmdApiApp: Resource 'ReconciliationDataIngestion_DataFlow' has the following validation error: Dataset is using 'AzureDataExplorer' linked service type, which is not supported in data flow.
 ERROR === CmdApiApp: Resource 'ReconciliationDataIngestion_DataFlow' has the following validation error: Dataset is using 'AzureDataExplorer' linked service type, which is not supported in data flow.
 ERROR === CmdApiApp: 

Since the package hasn't been updated in a few months, we suspect this is due to a misalignment between the package and the new features introduced in ADF in October 2021.

@luigibrancati
Copy link
Author

This issue has been resolved by a recent update.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant