Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to read Pipeline parameters configured in DataFactory from notebook #19792

Closed
tsrikanth06 opened this issue Nov 30, 2018 — with docs.microsoft.com · 4 comments
Closed

Comments

Copy link

You created a parameter with name 'name', but inside the notebook you are reading as 'input'. How does it map 'input' to 'name', aslo getArgument() takes two arguments
error: not enough arguments for method getArgument: (argName: String, defaultValue: String)String.

Even after providing default value, getArgument did not read the parameter I passed via DataFactory.
Can you please give a code snippet on how to read pipeline parameters from notebook.
Aslo while configuring notebook in dataFactory, there is 'User Properties', whats the difference between 'User Properties' and Pipeline 'Parameters'. How to read 'User Parameters' from notebook.


Document Details

Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

@MohitGargMSFT
Copy link
Member

@tsrikanth06 Thanks for your feedback! We will investigate and update soon on it.

@jason-j-MSFT
Copy link
Contributor

@tsrikanth06 I just tried the tutorial and it worked fine for me.

'input' gets mapped to 'name' because 'input' = @pipeline().parameters.name.

When the pipeline is triggered, you pass a pipeline parameter called 'name':
https://docs.microsoft.com/en-us/azure/data-factory/transform-data-using-databricks-notebook#trigger-a-pipeline-run

Here is more information on pipeline parameters:
https://docs.microsoft.com/en-us/azure/data-factory/control-flow-expression-language-functions

Copy link
Author

yeah it works, Thanks
I built a pipeline and notebook on Failure triggers web activity. But when notebook fails, like with exception, DataFactory fails with errorCode:3204 and failureType: UserError and it does not trigger web activity.
How can I treat UserError as Failed or is there a way my notebook can throw exception and dataFactory can catch the exception and treat it as Failure

@rzdebskiy
Copy link
Contributor

The issue reappears - today I also had the same problem. Looks like "input" is reserved word, when you rename it to "input_path" is starts working. Here is open issue on this - #42281. Please fix it in tutorial otherwise tutorial just doens't work - https://docs.microsoft.com/en-us/azure/data-factory/transform-data-using-databricks-notebook

@PRMerger15 PRMerger15 added the Pri2 label Nov 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants