Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ExecuteDataFlowActivity- dataflow reference is everytime null #39402

Open
gerald-csapo opened this issue Mar 26, 2024 · 1 comment
Open

ExecuteDataFlowActivity- dataflow reference is everytime null #39402

gerald-csapo opened this issue Mar 26, 2024 · 1 comment
Assignees
Labels
ARM customer-reported Issues that are reported by GitHub users external to the Azure organization. Mgmt This issue is related to a management-plane library. needs-team-attention This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that

Comments

@gerald-csapo
Copy link

gerald-csapo commented Mar 26, 2024

When process activities for a pipeline and process objects with instance ExecuteDataFlowActivity the dataflow is evertime null

I created pipeline and dataflow on Data Factory portal; published everything.

image

When running follwing I would expect to get the DataFlowReference...

		Factories factories = manager.factories();
		for (Factory factory : factories.listByResourceGroup(scannerConfig.getResourceGroupName())) {
			Pipelines pipelines = manager.pipelines();
			for (PipelineResource pipelineResource : pipelines.listByFactory(scannerConfig.getResourceGroupName(), factory.name())) {
				List<Activity> activities = pipelineResource.activities();
				for (Activity activity1 : activities) {
					if (activity1 instanceof ExecuteDataFlowActivity) {
						ExecuteDataFlowActivity activity = (ExecuteDataFlowActivity) activity1;
						if (((ExecuteDataFlowActivity) activity).dataFlow() == null) { // reference is null
							System.out.println("-- pipeline " + pipelineResource.name());
							System.out.println(JsonHandler.INSTANCE.beanToPrettyJsonString(activity));
						}
					}
				}
			}
		}

Output:
-- pipeline pipeline1

{
  "type" : "ExecuteDataFlow",
  "name" : "run Data flow1",
  "description" : null,
  "state" : null,
  "onInactiveMarkAs" : null,
  "dependsOn" : [ ],
  "userProperties" : [ ],
  "linkedServiceName" : null,
  "policy" : {
    "timeout" : "0.12:00:01",
    "retry" : 0,
    "retryIntervalInSeconds" : 30,
    "secureInput" : false,
    "secureOutput" : false
  },
  "typeProperties" : {
    **"dataFlow" : null,**
    "staging" : {
      "linkedService" : null,
      "folderPath" : null
    },
    "integrationRuntime" : null,
    "compute" : {
      "computeType" : "General",
      "coreCount" : 8
    },
    "traceLevel" : "Coarse",
    "continueOnError" : null,
    "runConcurrently" : true,
    "sourceStagingConcurrency" : null
  }
}

Setup:

  • OS: Win10
  • IDE: EclipseIDE
  • Library/Libraries:
  • com.azure:azure-identity 1.11.4
  • com.azure:azure-core 1.47.0
  • com.azure:azure-core-management 1.12.0
  • com.azure.resourcemanager:azure-resourcemanager-datafactory 1.0.0-beta.27
@github-actions github-actions bot added ARM customer-reported Issues that are reported by GitHub users external to the Azure organization. Mgmt This issue is related to a management-plane library. needs-team-attention This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Mar 26, 2024
Copy link

Thank you for your feedback. Tagging and routing to the team member best able to assist.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ARM customer-reported Issues that are reported by GitHub users external to the Azure organization. Mgmt This issue is related to a management-plane library. needs-team-attention This issue needs attention from Azure service team or SDK team question The issue doesn't require a change to the product in order to be resolved. Most issues start as that
Projects
None yet
Development

No branches or pull requests

2 participants