/
nsxt-prepare-mgmt-plane.html.md.erb
206 lines (162 loc) · 11.1 KB
/
nsxt-prepare-mgmt-plane.html.md.erb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
---
title: Creating the PKS Management Plane
owner: PKS
---
<strong><%= modified_date %></strong>
Prepare the vSphere and NSX-T infrastructure for the PKS Management Plane where the PKS, Ops Manager, BOSH Director, and Harbor Registry VMs are deployed.
##<a id='prerequisites'></a>Prerequisites
Before you begin this procedure, ensure that you have reviewed the following documentation for installing PKS on vSphere with NSX-T:
<ul>
<li>
<a href="./vsphere-nsxt-requirements.html">Prerequisites and Resource Requirements</a>
</li>
<li>
<a href="./nsxt-topologies.html">NSX-T Deployment Topologies for PKS</a>
</li>
<li>
<a href="./nsxt-prepare-env.html">Preparing to Deploy PKS with NSX-T on vSphere</a>
</li>
</ul>
In addition, ensure that you have successfully deployed NSX-T for PKS. For more information, see <a href="./nsxt-deploy.html">Deploying NSX-T for PKS</a>.
##<a id='about'></a> About the PKS Management Plane
The PKS Management Plane is the network for PKS Management components, including PKS, Ops Manager, and BOSH Director. The PKS Management Plane includes a vSphere resource pool for Management Plane components, as well as a NSX Tier-1 Logical Switch, Tier-1 Logical Router, and Router Port, as well as NSX NAT rules.
If you are using either the the <a href="nsxt-topologies.html#topology-nat">NAT deployment topology</a> or the <a href="nsxt-topologies.html#topology-no-nat-logical-switch">No-NAT with Logical Switch deployment topology</a>, create a [Tier-1 (T1) Logical Switch](https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.admin.doc/GUID-23194F9A-416A-40EA-B9F7-346B391C3EF8.html), and a [Tier-1 Logical Router and Port](https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.admin.doc/GUID-DAEF8457-8363-4F33-84DA-68AA36A2DE3C.html). Link the T1 logical router to the T0 logical router, and select the Edge Cluster defined for PKS. Enable route advertisement for the T1 Logical Router and advertise All NSX connected routes for the PKS Management Plane VMs (PKS, Ops Manager, and BOSH Director).
If you are using the <a href="nsxt-topologies.html#topology-nat">NAT Topology</a>, create the following NAT rules on the T0 Router.
* Destination NAT (DNAT) rule that maps an external IP address from the **PKS MANAGEMENT CIDR** to the IP where you deploy Ops Manager on the PKS Management logical switch. For example, a DNAT rule that maps `10.172.1.2` to `172.31.0.2`, where `172.31.0.2` is the IP address you assign to Ops Manager when connected to `ls-pks-mgmt`.
* (Optional) Destination NAT (DNAT) rule that maps an external IP address from the **PKS MANAGEMENT CIDR** to the IP where you deploy Harbor on the PKS Management logical switch. For example, a DNAT rule that maps `10.172.1.3` to `172.31.0.3`, where `172.31.0.3` is the IP address you assign to Harbor when connected to `ls-pks-mgmt`.
* Source NAT (SNAT) rule to allow the PKS Management VMs to communicate with your vCenter and NSX Manager environments. For example, an SNAT rule that maps `172.31.0.0/24` to `10.172.1.1`, where `10.172.1.1` is a routable IP address from your **PKS MANAGEMENT CIDR**.
* SNAT rule for PKS management components to access ESXi Hosts.
* (Optional) SNAT rules for access to other management servers, such as DNS, NTP, and LDAP/AD.
##<a id='create-rp'></a>Step 1. Create vSphere Resource Pool for the PKS Management Plane
1. Log in to vCenter for your vSphere environment.
1. Select **Compute Cluster > New Resource Pool**.
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-01.png">
1. Name the resource pool, such as `RP-MGMT-PKS`.
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-02.png">
1. Click **OK**
1. Verify resource pool creation.
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-03.png">
##<a id='create-ls'></a>Step 2. Create NSX-T Logical Switch for the PKS Management Plane
1. In NSX Manager, select **Switching > Add**.
1. Create a new logical switch. For example:
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-04.png">
1. Click **Add**.
1. Verify logical switch creation.
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-05.png">
##<a id='create-rp'></a>Step 3. Create NSX-T Tier-1 Router for the PKS Management Plane
Defining a T1 router involves creating the router and attaching it to the logical switch, creating a router port, and advertising the routes.
### Create T1 Router
1. In NSX Manager, select **Routing > Add > Tier-1 Router**.
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-06.png">
1. Configure the T1 router. For example:
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-07.png">
1. Click **Add**.
1. Verify T1 router creation.
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-08.png">
### Create T1 Router Port
1. Select the T1 router you created.
1. Select **Configuration > Router Ports**.
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-09.png">
1. Click **Add** and configure the T1 router port. For example:
* **Name**: `T1-MGMT-PKS-PORT`
* **Logical Switch**: `select LS-MGMT-PKS`
* **IP Address/mask**: `10.0.0.1/24`
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-10.png">
1. Click **Add**.
1. Verify T1 router port creation.
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-11.png">
### Advertise the T1 Routes
1. Select the T1 router > Routing > Route Advertisement.
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-12.png">
1. Advertise the T1 route as follows:
* **Status**: enabled
* **Advertise all NSX connected routes**: yes
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-13.png">
1. Click **Save**.s
1. Verify route advertisement.
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-14.png">
### Verify T1 Router
1. Select the **T1 Router > Overview**.
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-15.png">
1. Select **Tier-0 Connection > Connect**, then select the T0 router and click **Connect**.
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-16.png">
1. Verify connectivity between T1 and T0 routers.
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-17.png">
1. Select the **T1 router > Router ports**. The T1 router created for the PKS Management Plane should have 2 ports: one connected to the T0 router, and
a second port connected to logical switch defined for the PKS Management Plane. This second port will be the default gateway for all VMs connected to this LS.
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-18.png">
##<a id='create-dnat-om'></a>Step 4. Create DNAT Rule on T0 Router for Ops Manager
Create a DNAT rule on the T0 Router to access the Ops Manager Web UI, which is required to deploy PKS.
The Destination NAT (DNAT) rule on the T0 maps an external IP address from the **PKS MANAGEMENT CIDR** to the IP where you deploy Ops Manager on the PKS Management logical switch that you created on the T0 router. For example, a DNAT rule that maps `10.172.1.2` to `172.31.0.2`, where `172.31.0.2` is the IP address you assign to Ops Manager when connected to `ls-pks-mgmt`.
To create a DNAT rule for Ops Manager:
1. In NSX Manager, select **Routing > Routers**.
1. Select the **T0 Router > Services > NAT**.
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-19.png">
1. Add and configure a DNAT rule with the routable IP address as the destination and the internal IP address for Ops Manager as the translated IP. For example:
* **Priority**: 1000
* **Action**: DNAT
* **Destination IP**: 10.40.14.1
* **Translated IP**: 10.0.0.2
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-20.png">
1. Click **Add**.
1. Verify the DNAT rule.
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-21.png">
##<a id='create-dnat-harbor'></a>Step 5. Create DNAT Rule on T0 Router for Harbor Registry
If you are using VMware Harbor Registry with PKS, create a similar DNAT rule on T0 router to access the Harbor Web UI. This DNAT rule maps the private Harbor IP address to a routable IP address from the floating IP pool on the PKS Management network. See <a href="https://docs.pivotal.io/partners/vmware-harbor/integrating-pks.html#create-dnat">Create DNAT Rule</a> in the VMware Harbor Registry documentation for instructions.</p>
##<a id='create-snat-infra'></a>Step 6. Create SNAT rule on T0 router for vCenter and NSX Manager
Create a SNAT rule on T0 router for PKS management components to access vCenter and NSX manager. The Source NAT (SNAT) rule on the T0 allows the PKS Management VMs to communicate with your vCenter and NSX Manager environments. For example, a SNAT rule that maps `172.31.0.0/24` to `10.172.1.1`, where `10.172.1.1` is a routable IP address from your **PKS MANAGEMENT CIDR**.
<p class="note"><strong>Note</strong>: Limit the Destination CIDR for the SNAT rules to the subnets that contain your vCenter and NSX Manager IP addresses.</p>
1. Select **T0 router > Services > NAT**.
1. Click ADD and configure the SNAT rule. For example:
* **Priority**: 1010
* **Action**: SNAT
* **Source**: 10.0.0.0/24
* **Destination IP**: 10.40.206.0/24
* **Translated IP**: 10.40.14.2
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-22.png">
1. Click **Add**.
1. Verify SNAT rule creation.
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-23.png">
##<a id='create-snat-etc'></a>Step 7. Create SNAT Rules on T0 Router for DNS, NTP, and LDAP/AD
1. In NSX Manager, select **T0 router > Services > NAT**.
1. Add a SNAT rule for DNS. For example:
* **Priority**: 1010
* **Action**: SNAT
* **Source**: 10.0.0.0/24
* **Destination IP**: 10.20.20.1
* **Translated IP**: 10.40.14.2
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-24.png">
1. Click **Add**.
1. Add a SNAT rule for NTP. For example:
* **Priority**: 1010
* **Action**: SNAT
* **Source**: 10.0.0.0/24
* **Destination IP**: 10.113.60.176
* **Translated IP**: 10.40.14.2
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-26.png">
1. Click **Add**.
1. Add a SNAT rule for LDAP/AD. For example:
* **Priority**: 1010
* **Action**: SNAT
* **Source**: 10.0.0.0/24
* **Destination IP**: 10.40.207.0/24
* **Translated IP**: 10.40.14.2
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-27.png">
1. Click **Add**.
1. Verify SNAT rule creation.
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-28.png">
##<a id='create-snat-esxi'></a>Step 8. Create SNAT Rule on T0 Router for ESXi Hosts
Create a SNAT rule on T0 router for PKS management components to access ESXi Hosts (Management IP). The Destination IP is the Management IP subnet where ESXi Hosts are networked.
<p class="note"><strong>Note</strong>: Ops Manager and BOSH must use the NFCP protocol to the actual ESX hosts to which it is uploading stemcells. Specifically, <strong>Ops Manager & BOSH Director -> ESXi</strong>.</p>
1. Select **T0 router > Services > NAT**.
1. Click **Add** and configure the SNAT rule. For example:
* **Priority**: 1010
* **Action**: SNAT
* **Destination IP**: 10.115.40.0/24
* **Translated IP**: 10.40.14.2
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-29.png">
1. Click **Add**.
1. Verify SNAT rule creation:
<img src="images/nsxt/mgmt-plane/create-mgmt-plane-29.png">
##<a id='next'></a> Next Step
After you complete this procedure, follow the instructions in <a href="./nsxt-prepare-compute-plane.html">Creating the PKS Compute Plane</a>.