-
Notifications
You must be signed in to change notification settings - Fork 10
/
s4s_tune_wmp.xml
378 lines (364 loc) · 14.4 KB
/
s4s_tune_wmp.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE chapter
[
<!ENTITY % entities SYSTEM "generic-entities.ent">
%entities;
]>
<chapter xmlns="http://docbook.org/ns/docbook" xml:id="cha-memory-protection"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0">
<title>Tuning &wmp;</title>
<info>
<dm:docmanager xmlns:dm="urn:x-suse:ns:docmanager">
<dm:bugtracker/>
<dm:translation>yes</dm:translation>
</dm:docmanager>
</info>
<para>
Keeping &sap; applications in physical memory is essential for their
performance. In older product versions, the Page Cache Limit prevented a swap
out to disk by a growing page cache (in &productname; 11 SP1 onwards and in
&productname; 12). In &productname; 15, the Page Cache Limit has been
replaced by the more advanced &wmp;.
</para>
<para>
&wmp; puts &sap; instances into a dedicated cgroup (v2) and tells the kernel,
by the <varname>memory.low</varname> parameter, the amount of memory to keep
in physical memory. This protects the processes in this cgroup against any
form of memory pressure outside that cgroup, including a growing page cache.
&wmp; cannot protect against memory pressure inside this cgroup. It covers
the memory of <emphasis>all</emphasis> instances together on one host.
</para>
<para>
The value for <varname>memory.low</varname> depends on the kind of &sap;
instance and the workload and needs to be configured manually. If the system
is under extreme pressure, the Linux kernel will ignore the
<varname>memory.low</varname> value and try to stabilize the whole system,
even by swapping or invoking the OOM killer.
</para>
<para>
For more information about cgroups, see
<link xlink:href="https://documentation.suse.com/sles-15/html/SLES-all/cha-tuning-cgroups.html"/>.
</para>
<sect1 xml:id="sec-memory-protection-architecture">
<title>Architecture</title>
<para>
&wmp; relies on two components:
</para>
<variablelist>
<varlistentry>
<term>cgroup2 memory controller (Linux kernel)</term>
<listitem>
<para>
The cgroup2 memory controller parameter <parameter>memory.low</parameter>
allows defining an amount of memory, which the Linux kernel will keep in
physical memory. This amount of memory will be excluded from the
reclaiming process unless the entire system is in a critical memory
situation.
</para>
<para>
&wmp; uses <parameter>memory.low</parameter> to prevent memory from SAP
processes from being paged or swapped out to disk. Apart from the memory
controller, cgroup1 controllers are still available, but are not mounted
any more.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>&systemd;</term>
<listitem>
<para>
&systemd; provides the infrastructure to create and maintain the cgroup
hierarchy and allows the configuration of cgroup parameters.
</para>
</listitem>
</varlistentry>
</variablelist>
</sect1>
<sect1 xml:id="sec-memory-protection-support">
<title>Support for &wmp;</title>
<para>
&wmp; is supported for &productname; &productnumber; on &x86-64; and &power;
for one or multiple &sap; systems on one host, such as App Server (&netweaver;, &s4h;). &suse; &ha; cluster solutions are supported.
</para>
<para>
&wmp; does not cover databases other than &hana;. Depending on their start
method, the processes might run inside or outside the dedicated cgroup. If
they run inside, the memory consumption needs to be taken into account when
determining <varname>memory.low</varname>.
</para>
<important>
<title>Restrictions of &wmp;</title>
<para>
Using &wmp; comes with benefits, but you should be aware of certain
restrictions:
</para>
<itemizedlist>
<listitem>
<para>
&wmp; cannot protect against memory pressure inside the dedicated cgroup.
</para>
</listitem>
<listitem>
<para>
&wmp; cannot protect &sap; systems or their instances from each other. All
&sap; processes share the same memory limit. If you have multiple &sap;
systems (for example, &netweaver; and &s4h;), &wmp; cannot shield one &sap;
application from the other.
</para>
</listitem>
</itemizedlist>
</important>
<para>
To use &wmp;, the &sap; system must use &systemd;.
Details about the &systemd; integration can be found in <link
xlink:href="https://launchpad.support.sap.com/%3Cmark%3E/notes/3139184">SAP
Notes: 139184 - Linux: systemd integration for sapstartsrv</link> and <link
xlink:href="https://launchpad.support.sap.com/%3C/mark%3E/notes/3189534">SAP
Host Agent and 3189534 - Linux: systemd integration for sapstartsrv and SAP
HANA</link>.
</para>
<important>
<para>
Starting with &sles4sap; 15 SP5, the package <package>sapwmp</package> is
deprecated. For infomation about migration, see <xref linkend="sec-sapwmp-migrate" />.
</para>
</important>
</sect1>
<sect1 xml:id="sec-memory-protection-setup">
<title>Setting up &wmp;</title>
<remark>toms 2020-09-28: Some introduction para is missing here.</remark>
<sect2 xml:id="sec-memory-protection-setup-preparation">
<title>Preparing for &wmp;</title>
<para>
The &sap; Start Service puts &sap; instances into the dedicated
<literal>SAP.slice</literal> cgroup. To use &wmp;, switch to the unified cgroup2 hierarchy and set <varname>MemoryLow=</varname> correctly.
</para>
<procedure>
<step>
<para>
Stop all &sap; instances and the &sap; Host Agent.
</para>
<para>
The service can be enabled, but all &sap; processes need to be
terminated.
</para>
</step>
<step>
<para>
Add <varname>systemd.unified_cgroup_hierarchy=true</varname> to the
kernel command line by adding it to
<varname>GRUB_CMDLINE_LINUX_DEFAULT</varname> in
<filename>/etc/default/grub</filename> like:
</para>
<screen>GRUB_CMDLINE_LINUX_DEFAULT="... systemd.unified_cgroup_hierarchy=true swapaccount=1"</screen>
<para>
With this change, only cgroup2 controllers are be mounted on
<filename>/sys/fs/cgroup</filename>. Cgroup1 controllers, except the
memory controller, are still available and can be used though. Tools
using cgroup1 may not work out of the box any more and may need
reconfiguration. Also, the required mount structure for cgroup1 needs to
be provided.
</para>
<para>
The parameter <varname>swapaccount=1</varname> is not needed for &wmp; to
work, but it aids the analysis in support cases to show the amount of
swapped out memory for each cgroup.
</para>
</step>
<step>
<para>
Rewrite the GRUB2 configuration:
</para>
<screen>&prompt.sudo;<command>grub2-mkconfig</command> -o /boot/grub2/grub.cfg</screen>
<para>
After reboot (which is performed later), the cgroup hierarchy is switched to v2
(unified hierarchy) only.
</para>
</step>
<step xml:id="st-conf-memorylow-sap-slice">
<para>
Configure <varname>MemoryLow</varname> for the
<systemitem>SAP.slice</systemitem>:
</para>
<screen>&prompt.sudo;<command>systemctl</command> set-property SAP.slice MemoryLow=...</screen>
<para>
This command creates a drop-in in
<filename>/etc/systemd/system.control/SAP.slice.d/</filename> to set
<varname>MemoryLow</varname>.
</para>
<para>
<systemitem>SAP.slice</systemitem> is the name of the cgroup that
contains the SAP processes. <varname>MemoryLow</varname> is the &systemd;
equivalent of the cgroup parameter
<varname>memory.low</varname> mentioned in the introduction. The value
for <varname>MemoryLow</varname> depends on the type of the &sap;
application and the workload.
</para>
<variablelist>
<varlistentry>
<term>For &hana;</term>
<listitem>
<para>
Since &hana; has a Global Allocation Limit, its value can be used
directly.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>&sap; Application Server (&netweaver;, &s4h;)</term>
<listitem>
<para>
For the Application Server, the sizing for the workload should
indicate the value for <varname>MemoryLow</varname>.
</para>
</listitem>
</varlistentry>
</variablelist>
<para>
Keep in mind the following.
</para>
<itemizedlist>
<listitem>
<para>
All SAP instances on one host are inside the
<systemitem>SAP.slice</systemitem>. <varname>MemoryLow</varname> must
cover the amount of memory of <emphasis>all</emphasis> instances
together on that host. You cannot protect &sap; systems or their
instances from each other.
</para>
</listitem>
<listitem>
<para>
If you are using a database other than &hana;, some database processes
can be part of <systemitem>SAP.slice</systemitem>. Their memory
consumption needs to be taken into account when determining the
<varname>MemoryLow</varname> value.
</para>
</listitem>
<listitem>
<para>
Never choose a value for <varname>MemoryLow</varname> very close to or
larger than your physical memory. System services and additional
installed software require memory too. If they are forced to use swap
too extensively, at the expense of the &sap; application, your system
can become unresponsive.
</para>
</listitem>
</itemizedlist>
<note>
<title>Correctly calculate <varname>MemoryLow</varname> value</title>
<para>
<varname>MemoryLow</varname> takes the memory size in bytes. If the
value is suffixed with K, M, G, or T, the specified memory size is
parsed as Kibibytes, Mebibytes, Gibibytes, or Tebibytes (with the base
1024 instead of 1000, see
<link xlink:href="https://en.wikipedia.org/wiki/Binary_prefix"/>),
respectively. Alternatively, a percentage value may be specified, which
is taken relative to the installed physical memory on the system.
</para>
<para>
The underlying cgroup memory controller rounds up the value to a
multiple of the page size. To avoid confusion, set the value for
<varname>MemoryLow</varname> to a multiple of the page size.
</para>
</note>
</step>
</procedure>
<para>
Now the system is ready for a reboot.
</para>
</sect2>
<sect2 xml:id="sec-memory-protection-reboot-and-verification">
<title>Reboot and verification</title>
<remark>toms 2020-09-16: introduction of the procedure is missing.</remark>
<procedure>
<step>
<para>
Reboot the system.
</para>
</step>
<step>
<para>
After rebooting, verify that cgroups v2 has indeed been used:
</para>
<screen>&prompt.root;<command>grep</command> cgroup /proc/mounts
cgroup /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime 0 0</screen>
</step>
<step>
<para>
Verify that the low memory value has been set:
</para>
<screen>&prompt.user;<command>systemctl</command> show -p MemoryLow SAP.slice
MemoryLow=18487889920 <- Should be your chosen value (always in bytes)!
# cat /sys/fs/cgroup/SAP.slice/memory.low
18487889920 <- Should be your chosen value!</screen>
<para>
The variable <varname>MemoryLow</varname> can be set to any value, but
the content of the variable is always a multiple of the page size. Keep
this in mind if you notice a slight difference between the values.
</para>
</step>
</procedure>
</sect2>
</sect1>
<sect1 xml:id="sec-memory-protection-operation-change-the-value-of-memorylow">
<title>Changing the value of MemoryLow</title>
<para>
To change the value of <varname>MemoryLow</varname> run:
</para>
<screen>&prompt.root;<command>systemctl</command> set-property SAP.slice MemoryLow=...</screen>
<para>
The changes takes effect immediately.
</para>
<para>
The underlying cgroup memory controller rounds up the value to a
multiple of the page size. To avoid confusion, set the value of
<varname>MemoryLow</varname> to a multiple of the page size.
</para>
<important>
<title>Value of <varname>MemoryLow</varname></title>
<para>
Never set <varname>MemoryLow</varname> to a value lower than the memory
already accounted in <systemitem>SAP.slice</systemitem>. To check, run:
</para>
<screen>&prompt.root;<command>systemctl</command> show -p MemoryCurrent SAP.slice</screen>
</important>
</sect1>
<sect1 xml:id="sec-sapwmp-migrate">
<title>Migration from &wmp; using <package>sapwmp</package> (SLES for &sles4sap; 15 SP4 and
earlier)</title>
<para>
It is necessary to perform a migration when you are upgrading from a Service Pack SP4 or earlier, and you have not yet switched to &systemd;-enabled instances.
</para>
<procedure>
<title>Migration procedure</title>
<step>
<para>
Switch to a &systemd;-enabled &sap; system. For more info, refer to <link
xlink:href="https://launchpad.support.sap.com/%3C/mark%3E/notes/3189534">139184
- Linux: systemd integration for sapstartsrv and SAP Host Agent and
3189534 - Linux: systemd integration for sapstartsrv and SAP HANA</link>.
</para>
</step>
<step>
<para>
Remove the lines calling <option>sapwmp-capture</option> from all instance profiles (for example: <command>Execute_20 = local /usr/lib/sapwmp/sapwmp-capture -a</command>. All &sap; services require a restart after the change.
</para>
</step>
<step>
<para>
Monitor and readjust <option>MemoryLow=</option>. A &systemd;-enabled &sap; Host Agent now has a cgroup (<systemitem class="service">saphostagent.service</systemitem>) below <systemitem class="service">SAP.slice</systemitem>, and it is accounted for protection.
</para>
<note>
<para>
It is not necessary to set <option>MemoryLow=infinity</option> for each &sap; service or the &sap; Host Agent service, if the <systemitem>cgroup2</systemitem> mount has the option <option>memory_recursiveprot</option> set (it is set by default). To check that, run the following command:
</para>
<screen>&prompt.user; mount | grep cgroup2
cgroup2 on /sys/fs/cgroup type cgroup2
(rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)</screen>
</note>
</step>
</procedure>
</sect1>
</chapter>