-
Notifications
You must be signed in to change notification settings - Fork 56
/
service-gateway-onprem-faq.yml
241 lines (192 loc) · 16.5 KB
/
service-gateway-onprem-faq.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
### YamlMime:FAQ
metadata:
title: On-premises data gateway FAQ
description: This article is the on-premises data gateway FAQ. It collects frequently asked questions about the gateway into one spot.
ms.topic: faq
ms.date: 05/06/2024
title: On-premises data gateway FAQ
summary: |
sections:
- name: General
questions:
- question: |
Do I need a gateway for cloud data sources, such as Azure SQL Database?
answer: |
No, services can generally connect to cloud data sources without a gateway. However, you might need a data gateway if your data sources are behind a firewall, require a VPN, or are on virtual networks.
- question: |
Why do I get the errors "InvalidConnectionCredentials" or "AccessUnauthorized" when accessing cloud data sources using OAuth2 credentials even though the credentials are updated recently.
answer: |
When using OAuth2 credentials, the gateway currently doesn't support refreshing tokens automatically when access tokens expire. Tokens typically expire 1 hour after the refresh starts, but can expire in less than 1 hour, depending on the data source and the tenant policies. This limitation for long running refreshes exists for VNET gateways and on-premises data gateways. Dataflows specifically were recently updated to support token refresh, so they are no longer subject to the 1 hour timeout. However, other artifacts, such as pipelines, copy jobs, or semantic models, are still subject to the limitation.
- question: |
What are the requirements for the gateway?
answer: |
Review the requirements section in the [installation article](service-gateway-install.md#requirements).
- question: |
Does the gateway have to be installed on the same machine as the data source?
answer: |
No, the gateway connects to the data source by using the provided connection information. In this sense, consider the gateway as a client application. The gateway just needs to connect to the specified server.
- question: |
How many releases of the on-premises data gateway does Microsoft actively support?
answer: |
Currently, Microsoft actively supports only the last six releases of the on-premises data gateway. We release a new update for data gateways every month.
- question: |
Are there any licensing prerequisites required to install gateways?
answer: |
There are no licensing restrictions for installing and registering a gateway. But any cloud service might have licensing restrictions on how gateways are used within that service. For Power BI, a premium license is required. All Fabric licenses support the gateway. Other services might have licensing restrictions of their own as well.
- question: |
Are there any requirements for network bandwidth?
answer: |
Check that your network connection has good throughput. Each environment is different, and throughput depends on the amount of data that is sent. To ensure a minimum level of throughput between your on-premises data source and Azure datacenters, use [Azure ExpressRoute](/azure/expressroute/expressroute-introduction). To help measure your throughput, you can use the [Azure Speed Test app](https://azurespeedtest.azurewebsites.net/).
- question: |
Where are my credentials stored?
answer: |
The credentials that you enter for a data source are encrypted and stored in the gateway cloud service. The credentials are decrypted at the gateway on premises. For more information about credential encryption in Power BI, go to [Power BI security whitepaper](/power-bi/guidance/whitepaper-powerbi-security).
- question: |
What is the actual Windows service called?
answer: |
On your local computer, in the Services app, the service is called "On-premises data gateway service". In Task Manager, on the Services tab, the service name is "PBIEgwService". By default, the Windows service uses "NT SERVICE\PBIEgwService" as the Service SID (SSID).
- question: |
Can the gateway Windows service run with a Microsoft Entra ID account?
answer: |
No, the Windows service needs a valid Windows account.
- question: |
Are there any inbound connections to the gateway from the cloud?
answer: |
No, the gateway uses outbound connections to Azure Relay.
- question: |
What if I block outbound connections? What do I need to open?
answer: |
Go to [Enable outbound Azure connections](service-gateway-communication.md#enable-outbound-azure-connections).
- question: |
Do I need to unblock the Azure Datacenter IP list? Where do I get the list?
answer: |
If you block outbound IP traffic, you might need to unblock the Azure Datacenter IP list. The gateway communicates with Azure Relay by using an IP address and a fully qualified domain name. The Azure Datacenter IP list is updated weekly. For more information, go to [Enable outbound Azure connections](service-gateway-communication.md#enable-outbound-azure-connections).
- question: |
What is the latency for running queries from the gateway to a data source? What is the best architecture?
answer: |
To avoid network latency, install the gateway as close as possible to the data source. If you can install the gateway on the actual data source, this closer location minimizes latency.
Also, consider the proximity to the Azure datacenters. For example, if your service uses the West US datacenter, and you have SQL Server hosted in an Azure virtual machine, you might also want your Azure VM in the West US region. This configuration minimizes latency and avoids egress charges on the Azure VM.
- question: |
How are results sent back to the cloud?
answer: |
The results are sent through Azure Relay. For more information, go to [On-premises data gateway architecture](service-gateway-onprem-indepth.md).
- question: |
Can I place the gateway in a perimeter network (also known as DMZ, demilitarized zone, and screened subnet)?
answer: |
The gateway requires connectivity to the data source. So, if the data source isn't reachable in your perimeter network, the gateway might not have access.
For example, assume your SQL Server computer isn't in your perimeter network. Also, assume you can't connect to that computer from the perimeter network. If you place the gateway in your perimeter network, the gateway can't reach the SQL Server computer.
- question: |
If the server on which the gateway is installed is only using an IPv6 address and connects with a data source (for example, Azure SQL/File Server/Database on Azure VM) using an IPv6 address only, is it possible to communicate from Power BI service to those data sources through the gateway?
answer: |
Yes.
- question: |
If the server on which the gateway is installed is using dual stack to be assigned both IPv4 and IPv6 addresses, would IPv4 or IPv6 be the priority for gateway communication?
answer: |
This priority depends on Windows or the relevant data source drivers. This behavior is configurable in both Windows and various drivers, but isn't under the gateway's control.
- question: |
Does the gateway support cross-tenant Microsoft Entra ID access?
answer: |
No, cross-tenant Microsoft Entra ID access isn't supported. A Power BI semantic model in a tenant can't connect to data sources from a different tenant using a Microsoft Entra ID account. This limitation applies whether the data source uses an on-premises data gateway or not. To overcome this limitation, use an authentication type other than OAuth.
- question: |
Can I force the gateway to use HTTPS traffic with Azure Relay instead of TCP?
answer: |
Yes, for more information, go to [Force HTTPS communication with Azure Relay](service-gateway-communication.md#force-https-communication-with-azure-relay). Turning on this feature has little effect on performance.
- question: |
Are the on-premises data gateway and Data Management Gateway, which is used by Azure Machine Learning Studio and Azure Data Factory, the same thing?
answer: |
No, they're different products. To get more information about Data Management Gateway, which is now called Self-hosted Integration Runtime, go to [Create and configure a self-hosted integration runtime](/azure/data-factory/create-self-hosted-integration-runtime).
- question: |
Can the person who sets up the gateway in the Azure portal be different from the one who installs that gateway?
answer: |
Yes, you must use PowerShell to add other owners to the same gateway. These users can create the gateway in the Azure portal. However, they should connect to the portal and the gateway by using the same tenant.
- question: |
Does Azure ExpressRoute eliminate the need for a gateway?
answer: |
No. A gateway is still required when connecting to on-premises data sources.
- question: |
How can I keep my data within the same region while using an on-premises data gateway?
answer: |
More information: [Keep data in the region where it's stored](service-gateway-azure-relay.md#keep-data-in-the-region-where-its-stored)
- name: High availability and disaster recovery
questions:
- question: |
Are there any plans for enabling high-availability scenarios with the gateway?
answer: |
To help avoid a single point of failure, you can [set up on-premises data gateways as clusters](service-gateway-high-availability-clusters.md) for high availability. By default, cloud services such as Power Apps and Power BI use the primary gateway and fall back to the secondary gateway if the primary is unavailable.
- question: |
What options are available for disaster recovery?
answer: |
When you install the gateway, you supply a recovery key. You can use the key to [restore or migrate](service-gateway-migrate.md) a gateway.
- question: |
What is the benefit of the recovery key?
answer: |
The key provides a way to add a new gateway to a cluster or to migrate, recover, or take over a gateway.
## Troubleshooting
For more information, go to [Troubleshoot the on-premises data gateway](service-gateway-tshoot.md).
- question: |
Where are the gateway logs located?
answer: |
Go to [Troubleshooting tools](service-gateway-tshoot.md#troubleshooting-tools).
- question: |
How can I tell what queries are sent to the on-premises data source?
answer: |
You can enable query tracing by turning on [additional logging](service-gateway-performance.md#slow-performing-queries). The logs include the queries that are sent. Remember to turn off query tracing when you're done troubleshooting. Having query tracing enabled causes the logs to be larger.
You can also look at your data source's tools for tracing queries. For example, if SQL Server and SQL Server Analysis Services are data sources, you can use SQL Server Extended Events or SQL Server Profiler to trace queries.
- question: |
What do I need to do if I've reached the maximum limit of 1000 data sources per user, and how do I avoid reaching this limit?
answer: |
Users are limited to 1000 data sources per user.
If you've reached the maximum number of data sources limit, verify that the number of data sources per user hasn't surpassed the limit. To resolve any related issues, you can manually [remove the data sources](/power-bi/connect-data/service-gateway-data-sources#remove-a-data-source) from the admin center or, alternatively, use the following Gateway PowerShell script to find and bulk-delete any data sources that exceed the limit.
```powershell
## https://learn.microsoft.com/powershell/module/datagateway/?view=datagateway-ps
## PowerShell version of '7.0.0' to run
## required module "DataGateway" Install-Module -Name DataGateway and sign in the same user who exceeded the 1000 limit
Connect-DataGatewayServiceAccount
## get the gateway information per the sign in person
$gatewayClusters = Get-DataGatewayCluster | where-Object {$_.Type -EQ 'Personal'};
foreach ($gw in $gatewayClusters)
{
$datasources = Get-DataGatewayClusterDatasource -GatewayClusterId $gw.Id;
foreach ($datasource in $datasources)
{
$datasource
"gateway cluster id={0}, Personal Gateway={1}, datasource id={2}, datasourceType={3}, datasource connection details={4}" -f $gw.Id, $datasource.OnPremGatewayRequired, $datasource.Id, $datasource.DatasourceType, $datasource.ConnectionDetails
## conditional logic to determine if name matches set
## Remove-DataGatewayClusterDatasource -GatewayClusterId $gw.Id -GatewayClusterDatasourceId $datasource.Id
}
}
```
How do I avoid reaching this limit? If you're an ISV or any other Power BI Embedded app owner with many customers, use [service principal profiles](/power-bi/developer/embedded/embed-multi-tenancy) for multi-tenancy apps in Power BI embedded. If you're not an ISV, you might reach this limit because you're creating a new data source for every CSV or Excel file. To solve this, you might want to use the "upload file box" in Power BI Desktop to select multiple Excel files, which creates multiple data source connections. In this scenario, to ensure that only a single data source is selected, we recommend that you select the folder containing those Excel files instead.
- name: Administration
questions:
- question: |
Can I have more than one admin for a gateway?
answer: |
Yes, when you manage a gateway, you can go to the administrator's tab to add more admins. You can also have security groups as admins.
- question: |
Does the gateway admin need to be an admin on the machine where the gateway is installed?
answer: |
No, the gateway admin manages the gateway from within the service.
- name: Migration
questions:
- question: |
If gateway migration fails, can gateway activity be continued from the old gateway server? Does this require any manual operation?
answer: |
If the migration process fails on the new server, the on-premises data gateway still exists in the old server and you can still run that gateway unless the server is offline or the gateway software has been uninstalled. However, to restore the gateway on the old server, you might be asked to use your gateway recovery key.
- question: |
How long does a complete on-premises data gateway migration take?
answer: |
Since migration includes only two tasks, installing the gateway and restoration and configuration on new server, migration can usually be completed in 10 to 15 minutes.
- question: |
If we assume that the whole migration process will take 5 minutes, for all the refreshes that occur in these 5 minutes, where will the traffic go? Will it go through the old gateway server before the migration completes?
answer: |
Yes. However, there's a good chance of failures during this time, and migrations like these should be done in a downtime window. More information: [Minimize migration downtime](/data-integration/gateway/service-gateway-migrate#minimize-migration-downtime)
- name: Known Issues
questions:
- question: |
Are there any known issues?
answer: |
If an OAuth refresh that takes longer that one hour is canceled with the error "Invalid Connection Credentials" or timeout, the issue is likely that the credential expired. All artifacts other than dataflows can be subject to this timeout, but dataflows have been recently updated to support token refresh, and are not subject to the limitation anymore.
additionalContent: |
## Next steps
* [Troubleshoot the on-premises data gateway](service-gateway-tshoot.md)