set 100G memory but only 66G in centos 6.8/6.9 #321

wendal · 2017-04-27T10:28:13Z

centos 6.9
win10 x64

I assign 100G memory to guest vm, but inside centos, it show only 66G

any idea for that??

dcui · 2017-04-27T10:33:25Z

Did you check "Enable Dynamic Memory" in the Hyper-V "Setting -> Memory" for the VM?

What's the output of "free -m" and "cat /proc/meminfo" in the VM?

wendal · 2017-04-27T10:54:46Z

sorry, I download the wrong ubuntu iso, i386 version.

dcui · 2017-04-27T11:09:00Z

Ah.. yes, 64GB -- that's the max addressable physical address for 32-PAE

wendal · 2017-04-27T11:12:46Z

I try ubuntu 14.04.05 x64, it show 98G, without lis-next.

so it look like a bug at centos 6.9 kernel?? 2.6.32-696

wendal · 2017-04-27T11:14:16Z

after I install lis-next from source code, centos 6.9 fail to start after boot menu...

wendal · 2017-04-27T11:24:36Z

centos 6.8 x64 livecd show this, 66G memory:

any idea for that ?? thank you a lot

wendal · 2017-04-27T12:00:14Z

centos 7.3 1611, show 98G

dcui · 2017-04-27T12:09:26Z

I reproduced the issue with CentOS 6.8/6.9. Note the line "WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 33408MB of RAM.":

BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 00000000f7ff0000 (usable)
BIOS-e820: 00000000f7ff0000 - 00000000f7fff000 (ACPI data)
BIOS-e820: 00000000f7fff000 - 00000000f8000000 (ACPI NVS)
BIOS-e820: 0000000100000000 - 0000000fe0000000 (usable)
BIOS-e820: 0000001000000000 - 0000001928000000 (usable)
SMBIOS version 2.3 @ 0xF57D0
SMBIOS 2.3 present.
DMI: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090006 04/28/2016
AMI BIOS detected: BIOS may corrupt low RAM, working around it.
e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved)
Hypervisor detected: Microsoft HyperV
HyperV: features 0x2e7f, hints 0xc2c
HyperV: LAPIC Timer Frequency: 0x30d40
e820 update range: 0000000000000000 - 0000000000001000 (usable) ==> (reserved)
e820 remove range: 00000000000a0000 - 0000000000100000 (usable)
last_pfn = 0x1928000 max_arch_pfn = 0x400000000
x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
e820 update range: 0000001100000000 - 0000001928000000 (usable) ==> (reserved)
WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 33408MB of RAM.
------------[ cut here ]------------
WARNING: at arch/x86/kernel/cpu/mtrr/cleanup.c:972 mtrr_trim_uncached_memory+0x2de/0x309() (Not tainted)
Hardware name: Virtual Machine
Modules linked in:
Pid: 0, comm: swapper Not tainted 2.6.32-696.el6.x86_64 #1
Call Trace:
[] ? warn_slowpath_common+0x91/0xe0
[] ? warn_slowpath_null+0x1a/0x20
[] ? mtrr_trim_uncached_memory+0x2de/0x309
[] ? setup_arch+0x4e7/0xca6
[] ? vprintk_default+0xe/0x10
[] ? printk+0x4f/0x58
[] ? start_kernel+0xdc/0x436
[] ? x86_64_start_reservations+0x125/0x129
[] ? x86_64_start_kernel+0x115/0x124
---[ end trace a7919e7f17c0a725 ]---
update e820 for mtrr
modified physical RAM map:
modified: 0000000000000000 - 0000000000010000 (reserved)
modified: 0000000000010000 - 000000000009fc00 (usable)
modified: 000000000009fc00 - 00000000000a0000 (reserved)
modified: 00000000000e0000 - 0000000000100000 (reserved)
modified: 0000000000100000 - 00000000f7ff0000 (usable)
modified: 00000000f7ff0000 - 00000000f7fff000 (ACPI data)
modified: 00000000f7fff000 - 00000000f8000000 (ACPI NVS)
modified: 0000000100000000 - 0000000fe0000000 (usable)
modified: 0000001000000000 - 0000001100000000 (usable)
modified: 0000001100000000 - 0000001928000000 (reserved)
last_pfn = 0x1100000 max_arch_pfn = 0x400000000
last_pfn = 0xf7ff0 max_arch_pfn = 0x400000000

dcui · 2017-04-27T12:20:20Z

So the question is: why are CentOS 7.3 and Ubuntu 14.04.5 able to show ~100GB rather than 60+ GB...

wendal · 2017-04-27T12:33:57Z

after upgrade kernel by http://elrepo.reloumirrors.net/kernel/el6/x86_64/RPMS/kernel-ml-4.10.12-1.el6.elrepo.x86_64.rpm

it works, 98G show in free -m

wendal · 2017-04-27T12:39:53Z

after startup, centos 6.9 with kernel 4.10.12

wendal · 2017-04-27T12:42:28Z

is there any way to copy text from hyper-v guest?

dmesg show this at centos with kernel 4.10.12 :

dcui · 2017-04-27T12:51:15Z

It looks the built-in kernels of CentOS 6.8/6.9 are not so resilient to the BIOS defect. Good to know you have a workaround by installing the 4.x kernel.

No easy way to copy text from the VM terminal to the host. You can try SSH or VNC. :-)

wendal · 2017-04-27T12:52:28Z

ok, thank you a lot ^_^

wendal · 2017-04-27T12:56:51Z

BTW, is it possible a bug in hyper-v, not centos 6.8/6.9?

at the same win10 x64, centos 6.9 VM in virtualbox 5.1.20 can access full memory size, but it is very slow after ~60G, about 20mb/s, but still working.

dcui · 2017-04-27T13:00:25Z

Yeah, I think there is a bug in Hyper-V's guest BIOS: "WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 33408MB of RAM."

I just don't know how the 4.x kernel can cope with this.

wendal · 2017-04-27T13:41:39Z

add a topic at https://social.technet.microsoft.com/Forums/en-US/a92bd5c8-f47e-41b1-86c9-d5b3426f286b/cpu-mtrrs-dont-cover-all-of-memory-losing-33408mb-of-ram?forum=win10itprovirt

dcui · 2017-06-30T03:43:30Z

Old RHEL 6.x kernel has a bug, which can be exposed when the VM is running on WS 2016, where the reported max supported physical address has 44 bits. On WS 2012 R2, the max physical address has only 42 bits, and the bug is not triggered.

The old kernel needs to pick the patch:
x86: Fix /proc/mtrr with base/size more than 44bits (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d5c78673b1b28467354c2c30c3d4f003666ff385)

or, we can use the "disable_mtrr_trim" kernel parameter to work around the issue (there is not perf issue with this, because Hyper-V always ignores the MTRR setting for conventional RAM of a VM).

wendal · 2017-06-30T14:55:48Z

Great!

alexng-canuck · 2017-07-25T23:03:51Z

@wendal @dcui Is there any action item here with respect to LIS? Can we close this issue?

kattisrinivasan · 2017-07-25T23:05:31Z

Alex, For relevant RHEL/Centos Distros, we could add the required kernel command line option as part of LIS installation. K. Y From: Alex Ng [mailto:notifications@github.com] Sent: Tuesday, July 25, 2017 4:04 PM To: LIS/lis-next <lis-next@noreply.github.com> Cc: Subscribed <subscribed@noreply.github.com> Subject: Re: [LIS/lis-next] set 100G memory but only 66G in centos 6.8/6.9 (#321) Is there any action item here with respect to LIS? Can we close this issue? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FLIS%2Flis-next%2Fissues%2F321%23issuecomment-317898342&data=02%7C01%7Ckys%40microsoft.com%7C374df30dfb7d4c8ae50208d4d3b16d11%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636366206374237700&sdata=T%2BmTIZoRc%2B8d5kqu6KAhZD3gtu7wXQ2ORv7k9kahFyI%3D&reserved=0>, or mute the thread<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAA2AznGYWcWmE9diQ56sByeJISQMtBilks5sRnRXgaJpZM4NKB37&data=02%7C01%7Ckys%40microsoft.com%7C374df30dfb7d4c8ae50208d4d3b16d11%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636366206374237700&sdata=%2Fl9uAS7eHUJI4hUxnUlvKkvo%2BzVPRNlt72u3JV7URGQ%3D&reserved=0>.

alexng-canuck · 2017-07-25T23:13:49Z

Fair enough. We can look into adding the "disable_mtrr_trim" only for RH6.X installations.

@vyadavmsft; can you look into adding this for the RHEL 6.X RPMs? I will see what we can do in the rhel6.x install script for folks playing with the git sources.

This works around issue LIS#321. Summarized here: Old RHEL 6.x kernel has a bug, which can be exposed when the VM is running on WS 2016, where the reported max supported physical address has 44 bits. On WS 2012 R2, the max physical address has only 42 bits, and the bug is not triggered. The old kernel needs to pick the patch: x86: Fix /proc/mtrr with base/size more than 44bits https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d5c78673b1b28467354c2c30c3d4f003666ff385 or, we can use the "disable_mtrr_trim" kernel parameter to work around the issue. There is not perf issue with this, because Hyper-V always ignores the MTRR setting for conventional RAM of a VM).

Fix some requested issues #387 and #321

wendal changed the title ~~set 100G memory but only 66G in centos~~ set 100G memory but only 66G in centos 6.8/6.9 Apr 27, 2017

alexng-canuck assigned alexng-canuck and vyadavmsft Jul 25, 2017

alexng-canuck mentioned this issue Jul 27, 2017

Fix some requested issues #387 and #321 #407

Merged

alexng-canuck pushed a commit that referenced this issue Jul 31, 2017

Merge pull request #407 from alexngmsft/master

b7bd7bc

Fix some requested issues #387 and #321

alexng-canuck closed this as completed Aug 1, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

set 100G memory but only 66G in centos 6.8/6.9 #321

set 100G memory but only 66G in centos 6.8/6.9 #321

wendal commented Apr 27, 2017 •

edited

Loading

dcui commented Apr 27, 2017 •

edited

Loading

wendal commented Apr 27, 2017 •

edited

Loading

dcui commented Apr 27, 2017

wendal commented Apr 27, 2017

wendal commented Apr 27, 2017

wendal commented Apr 27, 2017 •

edited

Loading

wendal commented Apr 27, 2017

dcui commented Apr 27, 2017

dcui commented Apr 27, 2017

wendal commented Apr 27, 2017

wendal commented Apr 27, 2017

wendal commented Apr 27, 2017 •

edited

Loading

dcui commented Apr 27, 2017

wendal commented Apr 27, 2017

wendal commented Apr 27, 2017

dcui commented Apr 27, 2017

wendal commented Apr 27, 2017

dcui commented Jun 30, 2017

wendal commented Jun 30, 2017

alexng-canuck commented Jul 25, 2017 •

edited

Loading

kattisrinivasan commented Jul 25, 2017 via email

alexng-canuck commented Jul 25, 2017

set 100G memory but only 66G in centos 6.8/6.9 #321

set 100G memory but only 66G in centos 6.8/6.9 #321

Comments

wendal commented Apr 27, 2017 • edited Loading

dcui commented Apr 27, 2017 • edited Loading

wendal commented Apr 27, 2017 • edited Loading

dcui commented Apr 27, 2017

wendal commented Apr 27, 2017

wendal commented Apr 27, 2017

wendal commented Apr 27, 2017 • edited Loading

wendal commented Apr 27, 2017

dcui commented Apr 27, 2017

dcui commented Apr 27, 2017

wendal commented Apr 27, 2017

wendal commented Apr 27, 2017

wendal commented Apr 27, 2017 • edited Loading

dcui commented Apr 27, 2017

wendal commented Apr 27, 2017

wendal commented Apr 27, 2017

dcui commented Apr 27, 2017

wendal commented Apr 27, 2017

dcui commented Jun 30, 2017

wendal commented Jun 30, 2017

alexng-canuck commented Jul 25, 2017 • edited Loading

kattisrinivasan commented Jul 25, 2017 via email

alexng-canuck commented Jul 25, 2017

wendal commented Apr 27, 2017 •

edited

Loading

dcui commented Apr 27, 2017 •

edited

Loading

wendal commented Apr 27, 2017 •

edited

Loading

wendal commented Apr 27, 2017 •

edited

Loading

wendal commented Apr 27, 2017 •

edited

Loading

alexng-canuck commented Jul 25, 2017 •

edited

Loading