You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/data-factory/author-global-parameters.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,11 +20,11 @@ Global parameters are constants across a data factory that can be consumed by a
20
20
21
21
To create a global parameter, go to the *Global parameters* tab in the *Manage* section. Select **New** to open the creation side-nav.
22
22
23
-
23
+
24
24
25
25
In the side-nav, enter a name, select a data type, and specify the value of your parameter.
26
26
27
-
27
+
28
28
29
29
After a global parameter is created, you can edit it by clicking the parameter's name. To alter multiple parameters at once, select **Edit all**.
Copy file name to clipboardExpand all lines: articles/data-factory/author-management-hub.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -51,7 +51,7 @@ To override the generated Resource Manager template parameters when publishing f
51
51
52
52
Triggers determine when a pipeline run should be kicked off. Currently triggers can be on a wall clock schedule, operate on a periodic interval, or depend on an event. For more information, learn about [trigger execution](concepts-pipeline-execution-triggers.md#trigger-execution). In the management hub, you can create, edit, delete, or view the current state of a trigger.
Copy file name to clipboardExpand all lines: articles/data-factory/monitor-integration-runtime.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -253,13 +253,13 @@ If you join your Azure-SSIS IR to a VNet, you'll see the **VALIDATE VNET / SUBNE
253
253
254
254
On the **DIAGNOSE CONNECTIVITY** tile of your Azure-SSIS IR monitoring page, you can select the **Test connection** link to pop up a window, where you can check the connections between your Azure-SSIS IR and relevant package/configuration/data stores, as well as management services, via their fully qualified domain name (FQDN)/IP address and designated port (see [Testing connections from your Azure-SSIS IR](https://docs.microsoft.com/azure/data-factory/ssis-integration-runtime-diagnose-connectivity-faq)).
255
255
256
-
256
+
257
257
258
258
#### STATIC PUBLIC IP ADDRESSES tile
259
259
260
260
If you bring your own static public IP addresses for Azure-SSIS IR, you'll see the **STATIC PUBLIC IP ADDRESSES** tile on your Azure-SSIS IR monitoring page (see [Bringing your own static public IP addresses for Azure-SSIS IR](https://docs.microsoft.com/azure/data-factory/join-azure-ssis-integration-runtime-virtual-network#publicIP)). On this tile, you can select links designating your first/second static public IP addresses for Azure-SSIS IR to pop up a window, where you can copy their resource ID (`/subscriptions/YourAzureSubscripton/resourceGroups/YourResourceGroup/providers/Microsoft.Network/publicIPAddresses/YourPublicIPAddress`) from a text box. On the pop-up window, you can also select the **See your first/second static public IP address settings** link to manage your first/second static public IP address in Azure portal.
261
261
262
-
262
+
5. In the **Properties** window for the **Copy** activity at the bottom, switch to the **Source** tab, and click **+ New**. You create a source dataset for the copy activity in this step.
@@ -269,7 +269,7 @@ In this step, you create a pipeline with one Copy activity and two Web activitie
269
269
270
270
22. Select **Copy** activity in the pipeline designer, and click **+->** button, and select **Error**.
271
271
272
-
272
+
273
273
23. Drag the **red** button next to the Copy activity to the second Web activity **SendFailureEmailActivity**. You can move the activities around so that the pipeline looks like in the following image:
274
274
275
275
@@ -300,7 +300,7 @@ In this step, you create a pipeline with one Copy activity and two Web activitie
2. To **view activity runs** associated with this pipeline run, click the first link in the **Actions** column. You can switch back to the previous view by clicking **Pipelines** at the top. Use the **Refresh** button to refresh the list.
1. In the **Activities** pane, expand the **Move and Transform** accordion. Drag and drop the **Data Flow** activity from the pane to the pipeline canvas.
82
82
83
83
## Build transformation logic in the data flow canvas
84
84
85
85
Once you create your Data Flow, you'll be automatically sent to the data flow canvas. In this step, you'll build a data flow that takes the moviesDB.csv in ADLS storage and aggregates the average rating of comedies from 1910 to 2000. You'll then write this file back to the ADLS storage.
86
86
87
87
1. In the data flow canvas, add a source by clicking on the **Add Source** box.
93
93
1. Choose **Azure Data Lake Storage Gen2**. Click Continue.
94
94
95
-
95
+
96
96
1. Choose **DelimitedText**. Click Continue.
97
97
98
-
98
+
99
99
1. Name your dataset **MoviesDB**. In the linked service dropdown, choose **New**.
100
100
101
-
101
+
102
102
1. In the linked service creation screen, name your ADLS gen2 linked service **ADLSGen2** and specify your authentication method. Then enter your connection credentials. In this tutorial, we're using Account key to connect to our storage account. You can click **Test connection** to verify your credentials were entered correctly. Click Create when finished.
1. If your debug cluster has started, go to the **Data Preview** tab of the source transformation and click **Refresh** to get a snapshot of the data. You can use data preview to verify your transformation is configured correctly.
111
111
1. Next to your source node on the data flow canvas, click on the plus icon to add a new transformation. The first transformation you're adding is a **Filter**.
1. Name your filter transformation **FilterYears**. Click on the expression box next to **Filter on** to open the expression builder. Here you'll specify your filtering condition.
115
115
116
-
116
+
117
117
1. The data flow expression builder lets you interactively build expressions to use in various transformations. Expressions can include built-in functions, columns from the input schema, and user-defined parameters. For more information on how to build expressions, see [Data Flow expression builder](concepts-data-flow-expression-builder.md).
118
118
119
119
In this tutorial, you want to filter movies of genre comedy that came out between the years 1910 and 2000. As year is currently a string, you need to convert it to an integer using the ```toInteger()``` function. Use the greater than or equals to (>=) and less than or equals to (<=) operators to compare against literal year values 1910 and 200-. Union these expressions together with the and (&&) operator. The expression comes out as:
@@ -132,35 +132,35 @@ Once you create your Data Flow, you'll be automatically sent to the data flow ca
132
132
133
133
1. Fetch a **Data Preview** to verify the filter is working correctly.
134
134
135
-
135
+
136
136
1. The next transformation you'll add is an **Aggregate** transformation under **Schema modifier**.
137
137
138
-
138
+
139
139
1. Name your aggregate transformation **AggregateComedyRatings**. In the **Group by** tab, select **year** from the dropdown to group the aggregations by the year the movie came out.
140
140
141
-
141
+
142
142
1. Go to the **Aggregates** tab. In the left text box, name the aggregate column **AverageComedyRating**. Click on the right expression box to enter the aggregate expression via the expression builder.
143
143
144
-
144
+
145
145
1. To get the average of column **Rating**, use the ```avg()``` aggregate function. As **Rating** is a string and ```avg()``` takes in a numerical input, we must convert the value to a number via the ```toInteger()``` function. This is expression looks like:
146
146
147
147
```avg(toInteger(Rating))```
148
148
149
149
Click **Save and Finish** when done.
150
150
151
-
151
+
152
152
1. Go to the **Data Preview** tab to view the transformation output. Notice only two columns are there, **year** and **AverageComedyRating**.
153
153
154
154
155
155
1. Next, you want to add a **Sink** transformation under **Destination**.
156
156
157
-
157
+
158
158
1. Name your sink **Sink**. Click **New** to create your sink dataset.
159
159
160
-
160
+
161
161
1. Choose **Azure Data Lake Storage Gen2**. Click Continue.
162
162
163
-
163
+
164
164
1. Choose **DelimitedText**. Click Continue.
165
165
166
166
@@ -176,13 +176,13 @@ You can debug a pipeline before you publish it. In this step, you're going to tr
176
176
177
177
1. Go to the pipeline canvas. Click **Debug** to trigger a debug run.
180
180
1. Pipeline debug of Data Flow activities uses the active debug cluster but still take at least a minute to initialize. You can track the progress via the **Output** tab. Once the run is successful, click on the eyeglasses icon to open the monitoring pane.
186
186
1. Click on a transformation to get detailed information about the columns and partitioning of the data.
0 commit comments