Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't load to Ram an item bigger than 2 GB #248

Closed
Taviruni opened this issue Dec 5, 2020 · 67 comments
Closed

Can't load to Ram an item bigger than 2 GB #248

Taviruni opened this issue Dec 5, 2020 · 67 comments
Labels

Comments

@Taviruni
Copy link

Taviruni commented Dec 5, 2020

Hi, First I want to say this is a FANTASTIC version. Thanks very much for all your effort and time to make it for us.
I liked a lot it is also capable to boot *.gz and *.lz4 files just as the MBR version as I have tested.

This is the issue I found: It seems something is wrong as grub4dos for UEFI do not let me load on Ram more than 2 GB, I got this message:

(hd0,1)
Out of memory
boot_image_handle not found

Press any key to continue....

I'm using grub4dos-0.4.6a_for_UEFI-2020-12-05 and can't load my 2.30 GB Mini-10x64.vhd on Ram in order to Ramboot from it (it has SVBus driver installed), the PC has 8 GB so this should no be a problem, as using the grub4dos MBR version it is loaded to Ram and Ramboots very fine, the VHD has a 64 MB FAT-32 ID: 0C, Primary Partition Active (to let me boot as MBR or UEFI), and a second partition NTFS where the Mini-10 is installed on Compact LZX mode used espace on this partition is 1.70 GB, if I made a 2046 MB VHD with same partions layout it Ramboots fantastic.
It seems something is wrong as grub4dos for UEFI do not let me load on Ram more than 2 GB, The VHD file is located on a NTFS partition into a folder named VHD:

This is my menu.lst entry on EFI\grub\menu.lst

title 10x64-UEFI.vhd (2046 MB) SVBus RAMDISK for UEFI boot from HD
find --set-root /VHD/10x64-UEFI.vhd
map --mem /VHD/10x64-UEFI.vhd (hd)
chainloader (hd-1)

For additional info see this post please: http://reboot.pro/topic/22400-grub4dos-for-uefi/page-4#entry217176, also I opened a topic on MSFN forum as reboot.pro forum has been failing very frequently: https://msfn.org/board/topic/182107-grub4dos-for-uefi/
My Mini-10x64 was made with the info on my Topic: http://reboot.pro/topic/22383-reducing-win10-and-older-oss-footprint/
but readapted to use a VHD with 2 partitions.

Thanks in advance, and once again this is fantastic improvement to grub4dos my prefered boot loader.

Your friend

alacran

@a1ive a1ive added the efi label Dec 5, 2020
@a1ive
Copy link
Collaborator

a1ive commented Dec 6, 2020

Please execute the following command:
displaymem -a

@Taviruni
Copy link
Author

Taviruni commented Dec 7, 2020

I executed the code and made picture with the celphone, they are 5 big size pictures, I will move them to the PC and upload them, in the meantime take a look to this: a lot of virtual floppies, CDs and VHs are created on the VHD loaded on Ram.
UEFI
SVBus VHD-3
SVBus VHD-4
Ram

alacran

@Taviruni
Copy link
Author

Taviruni commented Dec 7, 2020

This are the images of the displaymem -a command:

WhatsApp Image 2020-12-07 at 5 42 21 AM
WhatsApp Image 2020-12-07 at 5 44 16 AM
WhatsApp Image 2020-12-07 at 5 44 18 AM
WhatsApp Image 2020-12-07 at 5 44 24 AM
WhatsApp Image 2020-12-07 at 5 44 30 AM
WhatsApp Image 2020-12-07 at 5 44 38 AM
WhatsApp Image 2020-12-07 at 5 44 46 AM

alacran

@yaya2007
Copy link
Collaborator

yaya2007 commented Dec 8, 2020

have a try.
title 10x64-UEFI.vhd (2046 MB) SVBus RAMDISK for UEFI boot from HD
find --set-root /VHD/10x64-UEFI.vhd
map --mem --top /VHD/10x64-UEFI.vhd (hd)
chainloader (hd-1)
BOOTX64.rar.txt

@Taviruni
Copy link
Author

Taviruni commented Dec 8, 2020

Thanks very much, this solved the issue on this PC.
It took me some time as I had to made a new VHD, but finaly after testing this new version it is Rambooting very fine.

title Mini-10-UEFI.vhd (2360 MB MB) SVBus RAMDISK UEFI boot from HD
find --set-root /VHD/Mini-10-UEFI.vhd
map --mem --top /VHD/Mini-10-UEFI.vhd (hd)
chainloader (hd-1)

Once again thanks for this fantastic UEFI version, now it is working great.

alacran (from reboot.pro)

@yaya2007
Copy link
Collaborator

yaya2007 commented Dec 9, 2020

The problem is solved. Good.
This program has been modified several times. In order to determine which part of the program works, I hope you can test it again. Thank you.
title Mini-10-UEFI.vhd (2360 MB MB) SVBus RAMDISK UEFI boot from HD
find --set-root /VHD/Mini-10-UEFI.vhd
map --mem /VHD/Mini-10-UEFI.vhd (hd)
chainloader (hd-1)

@Taviruni
Copy link
Author

Taviruni commented Dec 10, 2020

Sorry, I didn't saw your comment until now. I'm using currently the version released in 2020-12-10 and it is working very fine in all my tests using map --mem --top

Do you still want me to test using only the map --mem command with this new version? or is not necessary anymore.
I'm ready to run all tests you want to help you improve this fantastic version for UEFI.

By the way I saw on your program topic on wuyou.net that liuzhaoyzz was finally able to run 10x64 from Ram, with the new version, but now has some troubles trying to run 7x64 from Ram, which is not fully compatible with recent versions of UEFI firmware, only older versions support it.
Just let me know if you want me to make some tests, here or on reboot.pro topic or on MSFM if reboot.pro is offline.

alacran (from reboot.pro)

@yaya2007
Copy link
Collaborator

yes,test it.

@Taviruni
Copy link
Author

Done, it booted fine using following commands:

title Mini-10x64-2004-2.vhd (2360 MB) SVBus RAMDISK UEFI boot from HD (no --top)
find --set-root /VHD/Mini-10x64-2004-2.vhd
map --mem /VHD/Mini-10x64-2004-2.vhd (hd)
chainloader (hd-1)

alacran (from reboot.pro)

@yaya2007
Copy link
Collaborator

Is this image installed in memory above 4GB or below? Is it the same computer as the screenshot above? If not, please check the memory distribution of this computer.

@Taviruni
Copy link
Author

Taviruni commented Dec 11, 2020

Is this image installed in memory above 4GB or below? I don't know but haven't changed the setting on the firmware, it has being always set to Enabled which is the preset option, MB is an old Asus B 75 8 GB Ram.
Same Pc, just new G4E version.

Memory Remap Feature

alacran (from reboot.pro)

@Taviruni
Copy link
Author

Taviruni commented Dec 11, 2020

As on some cases when a previous RamOS do not load or boot fine, on next successful boot I have found ghost virtual Floppies, CDs and VHDs, it could be good if you can show me or tech me a command to delete those ghost virtual drives, this could be useful to run more accurate tests, don't having them potentially interfering with results of next test, it wouldn't be a problem if they appear only in the RamOS that didn't boot fine, the problem is they appear when booting any other VHD.

I read this Post: http://bbs.wuyou.net/forum.php?mod=redirect&goto=findpost&ptid=422652&pid=4191192

But the idea of making a copy is not very useful when I have 4 or 5 VHDs for testing, same applies for the defragmenting option, especially now that I started running tests with the files located on a USB 3.0 device and those ghost images were seen when Rambooting any VHD on internal HD or USB device.

alacran (from reboot.pro)

@yaya2007
Copy link
Collaborator

Memory Remap Feature cannot be opened

@Taviruni
Copy link
Author

Taviruni commented Dec 12, 2020

After disabled Memory Remap Feature, none of the VHDs are loaded to memory with or without --top, and after enabling it again same thing, now I can't load any of the VHDs to Ram, I get allways same message on all of them:

Memory Remap

Also grub2 new version from Wintofllash (or a1ve on reboot.pro) do not load the VHDs on Ram and before changing this setting it was working very fine too.

alacran (from reboot.pro)

@yaya2007
Copy link
Collaborator

I can't open the link here.
"disabled Memory Remap Feature",How to achieve this ?

@yaya2007
Copy link
Collaborator

have a try.
BOOTX64.rar.txt

@Taviruni
Copy link
Author

Taviruni commented Dec 12, 2020

"disabled Memory Remap Feature",How to achieve this ? I did it in the firmware/Bios on Memory Remap Feature option

Just fixed the PC problem: went back to firmware/Bios, loaded Optimized Settings, made my personal settings as language, etc, as they were before, and saved (F10), after reboot all is back to normal, tested again the 2.3 GB VHD and it loaded fine to Ram and also booted fine again with or without --top parameter. Also a1ive new grub2 version is working fine again.

I assume there is a bug in the PC firmware and the setting was not re-enabled right, but loading Optimized Settings fixed the issue.

Just downloaded your file on previous message, and will test it and report back. Do you want me to test something especifically or just test loading and booting from RAM VHDs?
A little more info about version changes/improvements could be good to let me be more accurate on testing certain features.

By the way it seems there is an issue on map command, when mapping WinPE ISO file and also mapping VHD files, mam --mem and map --mem --top are wotking fine, for more info please see this post on reboot.pro: http://reboot.pro/topic/22400-grub4dos-for-uefi/page-7#entry217302

See you latter my friend.
alacran (from reboot.pro)

@yaya2007
Copy link
Collaborator

In this version, the floppy disk image is removed to see if it works properly.

@Taviruni
Copy link
Author

Taviruni commented Dec 12, 2020

v2020-12-12 Test Report

Locations of files used to test:
MBR NTFS Formated (hd0,4)/Isos/Win10XPE_x64.ISO
MBR FAT-32 Formated (hd1,0)/VHD/Mini-10x64-2004-2.vhd

Commands tested: map; map --mem, and map --mem --top

All booted fine, not a single issue. No more issues when using command map alone.

Tests Done:
title Start Win10XPE_x64.ISO as FILEDISK UEFI boot from HD
find --set-root /Isos/Win10XPE_x64.ISO
map /Isos/Win10XPE_x64.ISO (0xff)
chainloader (0xff)

title Start Win10XPE_x64.ISO as RAMDISK UEFI boot from HD (no --top)
find --set-root /Isos/Win10XPE_x64.ISO
map --mem /Isos/Win10XPE_x64.ISO (0xff)
chainloader (0xff)

title Mini-10x64-2004-2.vhd (2360 MB) SVBus FILEDISK UEFI boot from HD
find --set-root /VHD/Mini-10x64-2004-2.vhd
map /VHD/Mini-10x64-2004-2.vhd (hd)
chainloader (hd-1)

title Mini-10x64-2004-2.vhd (2360 MB) SVBus RAMDISK UEFI boot from HD (no --top)
find --set-root /VHD/Mini-10x64-2004-2.vhd
map --mem /VHD/Mini-10x64-2004-2.vhd (hd)
chainloader (hd-1)

title Mini-10x64-2004-2.vhd (2360 MB) SVBus RAMDISK UEFI boot from HD
find --set-root /VHD/Mini-10x64-2004-2.vhd
map --mem --top /VHD/Mini-10x64-2004-2.vhd (hd)
chainloader (hd-1)

No more issues when using command map alone.

@yaya2007
Copy link
Collaborator

thank you

@Taviruni
Copy link
Author

Taviruni commented Dec 12, 2020

You are welcome my friend.

In this version, the floppy disk image is removed to see if it works properly.

Just some comments, related to the successful tests:

Since you released this UEFI version all my VHDs UEFI bootables are MBR formated (2 partitions), First 64 to 100 MB Primary FAT-32 Active Partition ID:0C + a Secondary NTFS partition, were the OS with SVBus driver installed resides.
The FAT-32 partition contains only boot files/folders to MBR/CSM and UEFI boot.

I think it was a good decision to remove the floppy disk image, and what about the CD image?, What is the use of it?

alacran (from reboot.pro)

@Taviruni
Copy link
Author

Taviruni commented Dec 17, 2020

Grub4dos-for_UEFI-2020-12-15 + ntfs_x64.efi Test Report

Mini-10x64-2004.vhd >>> Single partition NTFS Active booted very fine >>> 2004 - release.191206-1406

Booted very fine.

title Mini-10x64-2004.vhd (2360 MB only NTFS) as RAMDISK UEFI boot from HD
load /EFI/grub/ntfs_x64.efi
find --ignore-floppies --ignore-cd --set-root /Mini-10x64-2004.vhd
map --mem --top /Mini-10x64-2004.vhd (hd)
chainloader (hd-1)

alacran (reboot.pro)

@Taviruni
Copy link
Author

Grub4dos-for_UEFI-2020-12-15 + ntfs_x64.efi (from Rufus) Test Report

Mini-10x64-2004.vhd >>> Single partition NTFS Active booted very fine >>> 2004 - release.191206-1406

The VHD fail to boot:

title Mini-10x64-2004.vhd (2360 MB) Rufus as RAMDISK UEFI boot from HD NTFS
load /EFI/grub/ntfs_x64-R.efi
find --ignore-floppies --ignore-cd --set-root /Mini-10x64-2004.vhd
map --mem --top /Mini-10x64-2004.vhd (hd)
chainloader (hd-1)

Used ntfs_x64.efi from Akeo (Rufus) renamed to ntfs_x64-R.efi: https://efi.akeo.ie/...64/ntfs_x64.efi

The VHD was loaded to Ram but I got the message: Boot_image_handle not found, so this driver do not work fine with G4E

alacran (reboot.pro)

@a1ive
Copy link
Collaborator

a1ive commented Dec 17, 2020

Grub4dos-for_UEFI-2020-12-15 + ntfs_x64.efi (from Rufus) Test Report

Mini-10x64-2004.vhd >>> Single partition NTFS Active booted very fine >>> 2004 - release.191206-1406

The VHD fail to boot:

title Mini-10x64-2004.vhd (2360 MB) Rufus as RAMDISK UEFI boot from HD NTFS
load /EFI/grub/ntfs_x64-R.efi
find --ignore-floppies --ignore-cd --set-root /Mini-10x64-2004.vhd
map --mem --top /Mini-10x64-2004.vhd (hd)
chainloader (hd-1)

Used ntfs_x64.efi from Akeo (Rufus) renamed to ntfs_x64-R.efi: https://efi.akeo.ie/...64/ntfs_x64.efi

The VHD was loaded to Ram but I got the message: Boot_image_handle not found, so this driver do not work fine with G4E

alacran (reboot.pro)

this driver doesn't work on my PC, and is not caused by grub.

@Taviruni
Copy link
Author

@ aive

Thanks my friend, I was aware it didn't work on your PC, you commented it on a post on reboot.pro, just wanted to test the more recent version and this one also don't work, I don't mean the failure is caused by grub4dos, what I mean is it can't be used for our purpose.

alacran (reboot.pro)

@Taviruni
Copy link
Author

Taviruni commented Dec 25, 2020

From my post: http://reboot.pro/topic/22400-grub4dos-for-uefi/page-10#entry217481

I ran some tests to measure the times required to load to RAM a VHD:

UEFI grub4dos 2020-12-15 and last version of grub2 from a1ive were used on following tests.

Of course your times will be different as your VHDs will be different and your CPU, HD and Ram speeds will be also different but at least this can help to give an idea to what you can expect on your own tests.

My old Asus B75M-A MB 8 GB Ram DDR3 1333 MHz (AMI UEFI firmware) have several options to set the CSM: Auto, Enabled and Disabled. Also it allows to disable Secure Boot.

All tests are with the VHDs located into a partition on second HDD:

`Secure Boot = Disabled; CSM = Auto

Mini-10-UEFI-WB.vhd (FAT-32+NTFS) 1330 MB Grub2 >>> 28.94 sec.

Mini-10-UEFI-WB.vhd (FAT-32+NTFS) 1330 MB Grub2 + grub4dos >>> 15.08 sec.

Mini-10-UEFI-WB.vhd (FAT-32+NTFS) 1330 MB grub4dos >>> 15.00 sec.

Mini-10-UEFI-WB.vhd.lz4 (FAT-32+NTFS) 1330 MB Grub2 + grub4dos >>> 4.28 sec.

Mini-10-UEFI-WB.vhd.lz4 (FAT-32+NTFS) 1330 MB grub4dos >>> 4.31 sec.`

I will take both as 4.30 sec., as the difference may be just my finger response.

Here the lz4 compression helps a lot to improve the time as the Wimboot VHD has a lot of free space.

`Secure Boot = Disabled; CSM = Disabled

Mini-10-UEFI.vhd (NTFS) 2046 MB Grub2 >>> 44.53 sec.

Mini-10-UEFI.vhd (NTFS) 2046 MB Grub2 + grub4dos >>> Do not work

Mini-10-UEFI.vhd (NTFS) 2046 MB grub4dos >>> 22.76 sec.

Mini-10-UEFI.vhd.lz4 (NTFS) 2046 MB grub4dos >>> 21.16 sec.`

If booting from grub2 and chainloading to grub4dos the biggest VHD capable to boot is 1.3 GB as just tested too. There is something interfering to let boot bigger size VHDs in this case.

Here the lz4 compression do not help improve so much the time as the VHD has very small free space.

`Secure Boot = Disabled; CSM = Disabled

Mini-10x64-2004-2.vhd (FAT-32+NTFS) 2.3 GB Grub2 >>> 50.89 sec.

Mini-10x64-2004-2.vhd (FAT-32+NTFS) 2.3 GB Grub2 + grub4dos >>> Do not work

Mini-10x64-2004-2.vhd (FAT-32+NTFS) 2.3 GB grub4dos >>> 26.30 sec.

Mini-10x64-2004-2.vhd.lz4 (FAT-32+NTFS) 2.3 GB grub4dos >>> 22.85 sec.`

As on previous set of tests: If booting from grub2 and chainloading to grub4dos the biggest VHD capable to boot is 1.3 GB as just tested too. There is something interfering to let boot bigger size VHDs in this case.

Here the lz4 compression helps improve a few the time as the VHD has more free space than in previous set of tests.

Conclusions:

  1. The best times required to load to Ram a VHD are got using UEFI grub4dos and lz4 compression.
  2. The second best times to load to Ram a VHD are got booting with UEFI grub2 and chainloding to grub4dos and lz4 compression.
    
  3. **Having 8 GB of Ram and booting with UEFI grub2 and chainloading to grub4dos, do not work for files bigger than 1.3 GB.**
    
  4. Unfortunatelly booting directly to UEFI grub4dos is not possible on only UEFI new MBs, so the users are forced to boot to UEFI grub2 first.
    

I think our very appreciated friends a1ive and yaya should try to find the cause of this issue, as this is inconceivable those VHDs boot very fine when loaded without chainloading from UEFI grub2 to UEFI grub4dos, something on the code of one of them (or maybe both) loaders is creating this issue when chainloading to the second loader.

I'm aware of the hard work (for free) of both of you, and I appreciate it a lot, but this issue is making me crazy. Sorry if my words could sound as a demand, please don't take it that way, take it as a very respectful and kindly request.

alacran (reboot.pro)

@Taviruni
Copy link
Author

Taviruni commented Dec 26, 2020

Unfortunatelly booting directly to UEFI grub4dos is not possible on only UEFI new MBs, so the users are forced to boot to UEFI grub2 first.

I tried to say:

New PCs are only UEFI, and Secure Boot can't be dissabled, then we are forced to boot to UEFI grub2 and chainload to UEFI grub4dos.

And then the 2.30 GB VHDs can't be loaded to Ram, as you can see on my previous tests.

alacran (reboot.pro)

@yaya2007
Copy link
Collaborator

Please use the test version published in today's worry free forum to test.

@yaya2007
Copy link
Collaborator

yaya2007 commented Dec 26, 2020

After gtub2 goes to G4e, take a look at the number of $grub4dos between 0-0x9ffff.
Is svbus used?

@yaya2007
Copy link
Collaborator

yaya2007 commented Dec 29, 2020

Enter the grub2 command line, execute lsefimmap and take a screenshot.
grub2->g4e, execute displaymem, screenshot.
load 2.3Gb VHD.
Add suffix. Txt to the screenshot file name and send it. I can't open the image here.
I want to compare. Thank you for the test.

@Taviruni
Copy link
Author

Attached requested info, if you need more info just let me know.

Grub2.jpeg.txt
G4E.jpeg.txt
Loading-VHD.jpeg.txt

alacran (reboot.pro)

@yaya2007
Copy link
Collaborator

yaya2007 commented Dec 30, 2020

grub2->g4e, execute displaymem -a, screenshot.
load 2.3Gb VHD.
Add suffix. Txt to the screenshot file name and send it. I can't open the image here.
I want to compare. Thank you for the test.
BOOTX64.rar.txt

@Taviruni
Copy link
Author

This is the info you requested:

displaymem-a.jpeg.txt
There were also other 800000000000000f on other screens

Booting.jpeg.txt
The 2.3GB RamOS didn't boot but the message is different now.

alcran (reboot.pro)

@yaya2007
Copy link
Collaborator

I found the problem .
You can use this test .
I hope the problem can be solved to help you.
BOOTX64.rar.txt

@Taviruni
Copy link
Author

Please have a look at the file properties. Is it December 30, 19:04?

No, but just downloaded the December 30, 19:04

But I will try first the version on your last post and report back.

alcran (reboot.pro)

@Taviruni
Copy link
Author

Taviruni commented Dec 31, 2020

Sorry to say this but your very last version didn't work, now even the 1.30GB VHD is not loaded to Ram it remains forever with the message:
Read disk data to memory, please wait

1.3GBRamOS.jpeg.txt

And I had to reboot the PC.

About this:

No, but just downloaded the December 30, 19:04

I was wrong after cheking the date of the file just downloaded from your comment : https://github.com/chenall/grub4dos/issues/248#issuecomment-752368896 I can see now it is December 30, 18:53 (downloaded it twice to avoid mistakes)
It seems maybe you uploaded the wrong version date, so I can repit the test you asked my to do on the mentioned comment.

Please take your time my friend, there is no rush, enjoy the welcome to the new year (if you celebrate it) and we will talk next year.
I wish you and all your family and friends a new year full of success and health.

Also the same wishes for all members of wuyou.net/forum, especially to liuzhaoyzz and a1ive, that have being in direct contact with us on reboot.pro

Have a happy new year!

Your friend
alacran (reboot.pro)

@liuzhaoyzz
Copy link

liuzhaoyzz commented Dec 31, 2020

Happy new year,my friends!
@alacran @yaya2007 @a1ive ...

@yaya2007
Copy link
Collaborator

Alacrane, happy New Year! It doesn't have to be a crash. If it usually takes one minute to load 2.3GB, you should wait patiently for two minutes. It's best to use a stopwatch.

@Taviruni
Copy link
Author

Taviruni commented Dec 31, 2020

Sorry my friend, I tested again with the 2.3 GB VHD and this time I waited for 5 minutes and it never load, also there was not any flashing on HD led to let me know it was reading the HD, as it does usually.

From some of my tests: http://reboot.pro/topic/22400-grub4dos-for-uefi/page-10#entry217481

This are my times to load to Ram that VHD:

Secure Boot = Disabled CSM = Disabled

Mini-10x64-2004-2.vhd (FAT-32+NTFS) 2.3 GB Grub2 >>> 50.89 sec.

Mini-10x64-2004-2.vhd (FAT-32+NTFS) 2.3 GB Grub2 + grub4dos >>> Do not work

Mini-10x64-2004-2.vhd (FAT-32+NTFS) 2.3 GB grub4dos >>> 26.30 sec.

Mini-10x64-2004-2.vhd.lz4 (FAT-32+NTFS) 2.3 GB grub4dos >>> 22.85 sec.

But this time I decided to test something else and after booting to a1ive's grub2 and chainload to UEFI grub4dos, I ran following entry from my menu.lst (never tested before to load/boot this VHD lz4 compressed chainloading from grub2 to UEFI grub4dos on any other previous version of UEFI grub4dos, but I'll do it too, only case of a VHD lz4 compresed I tested this way before was a 1.3 GB VHD, and it booted fine):

title Boot /VHD/Mini-10x64-2004-2.vhd.lz4 (FAT+NTFS) UEFI RAMDISK 2.3 GB
find --set-root --ignore-floppies --ignore-cd /VHD/Mini-10x64-2004-2.vhd.lz4
map --mem --top /VHD/Mini-10x64-2004-2.vhd.lz4 (hd)
chainloader (hd-1)

And to my surprice the message: Read disk data to memory, please wait never appear and it started loading the Mini-10x64-2004-2.vhd.lz4 immediately and it booted very fine (Sorry I didn't take the time).

alacran (reboot.pro)

@Taviruni
Copy link
Author

Tested with 2020-12-29 09.53 version and the same 2.3 GB VHD lz4 compressed didn't load to Ram, same old message out of map memory: 8000000000000009 as I expected.

So far your very last version is the only one capable to load and boot the 2.3 GB VHD lz4 compressed, but not uncompressed, then this means your code is fine on some part but not in other part, but this also means you found the issue, and the way to avoid it, and you only need to fix the part that is not working fine, which I think is very close (or related) to the message: Read disk data to memory, please wait.

I'm very interested in try to help to solve this issue not only for me, but for all those users with limited memory and a only UEFI machine where they can't disable Secure Boot and will be forced to boot first to grub2 and chainload to UEFI grub4dos if they want to use this great tool you have developed.

But as I said before my friend there is no rush, take your time, and we can talk about this in a few days if you want.

Also you can be sure I'm ready to run all test you ask me, with no limit.

alacran (reboot.pro)

@yaya2007
Copy link
Collaborator

yaya2007 commented Jan 1, 2021

@alacran
Happy new year, my friend!
You are right. Version 2020-12-31 has solved the problem that VHD larger than 1.3GB cannot be loaded into memory.
Compression VHD, using read file mode, so normal.
Uncompressed VHD uses read sector mode, so it is abnormal.
But liuzhaoyzz test, 8GB uncompressed VHD, is very fast, only 41% of the time of the old version. I don't know why you failed here.

Now upload a all read file mode, you can do all kinds of tests, you can load compressed and uncompressed VHD larger than 1.3GB.
BOOTX64_ok.rar.txt

At the same time, uploading an uncompressed VHD is read sector mode.
You can test uncompressed VHD, Test only one step:
map --mem --top /VHD/Mini-10-UEFI-WB.vhd (hd)

BOOTX64-test.rar.txt

1 similar comment
@yaya2007
Copy link
Collaborator

yaya2007 commented Jan 1, 2021

@alacran
Happy new year, my friend!
You are right. Version 2020-12-31 has solved the problem that VHD larger than 1.3GB cannot be loaded into memory.
Compression VHD, using read file mode, so normal.
Uncompressed VHD uses read sector mode, so it is abnormal.
But liuzhaoyzz test, 8GB uncompressed VHD, is very fast, only 41% of the time of the old version. I don't know why you failed here.

Now upload a all read file mode, you can do all kinds of tests, you can load compressed and uncompressed VHD larger than 1.3GB.
BOOTX64_ok.rar.txt

At the same time, uploading an uncompressed VHD is read sector mode.
You can test uncompressed VHD, Test only one step:
map --mem --top /VHD/Mini-10-UEFI-WB.vhd (hd)

BOOTX64-test.rar.txt

@Taviruni
Copy link
Author

Taviruni commented Jan 1, 2021

Happy new year to you too!

Thanks for the explanation my friend, now it makes sense to me why with Version 2020-12-31 I got different results in my case depending if the VHD was compressed or uncompressed.

Just downloaded both files, I will test them and report back.

See you latter
alacran (reboot.pro)

@Taviruni
Copy link
Author

Taviruni commented Jan 1, 2021

Test report:

The "ok version" works fantastic, I was able to boot a RamOS from a uncompressed 2.5 GH VHD, NOTE: It is internally LZX compressed as it was installed in Compact mode.

Loading the VHD.jpeg.txt

The "test version" do not load even the 1.3 GB VHD, (where the OS is installed on Wimboot mode), I copied it to another location to type a few less in G4E command line.

If this approach that uses read sector mode is similar to a1ive's grub2 -l (or --blocklist), I have tested it extensively too, and it is not capable to load a internally compressed VHD or a VHD installed on Wimboot mode, In fact it is only only capable to boot a RamOS that was installed without any compression.

I tested it with Wimboot, Compact mode and also when using NTFS compression when installing the OS into the VHD, all not externally compressed, and it wasn't able to load any of them to Ram.

See this quote from a post of a1ive on reboot.pro (made a little late after all my tests):
From: http://reboot.pro/topic/22429-a1ives-grub-2/#entry217523

blocklist does not support any kind of compression, including all formats of compressed files and NTFS compression.
I don't know what vhd-wimboot or vhd-compact is, but the following two things must be met to convert to blocklist disks.

  1. the VHD file cannot be compressed.
  2. the file system where the VHD file is located cannot have any kind of compression.

If this is the case, best option could be, use the ok version approach as the standard and general use version, as it is capable to load and boot all kind of VHD, and you may include an additional optional parameter to let the user use the read sector mode, only to be used on the especific case of a non compressed (internally and/or externally) VHD.

But as I already know you are a very smart guy, I'm almost sure you already thought in doing it this way.

Your friend
alacran (reboot.pro)

@Taviruni
Copy link
Author

Taviruni commented Jan 1, 2021

Using your "OK VERSION" I was able to boot from a RamOS installed in "Compact LZX mode" on a two partitions 3GB VHD, booting first with a1ve's grub2 and chainloading to G4E to load and boot the VHD, it is working fantastic.

3GB-VHD.png.txt

Your very grateful friend
alacran (reboot.pro)

@yaya2007
Copy link
Collaborator

yaya2007 commented Jan 2, 2021

I want to know, on your computer with 8GB memory, is there any difference between loading 3gb VHD and using or not using the -- top parameter? Can it be started successfully?

@Taviruni
Copy link
Author

Taviruni commented Jan 2, 2021

Not with "OK VERSION", even the 2.3 GB don't load to Ram, I get the message:

out of map memory: 8000000000000009 and keyboard is dead and I have to reboot.

Tested chainloading from a1ive's grub2 to G4E and also booting directly to G4E and same thing in both cases.

But a previous version used on #248 (comment)
was able to boot the 2.3 GB VHD without --top if booting directly to G4E

title Mini-10x64-2004-2.vhd (2360 MB) SVBus RAMDISK UEFI boot from HD (no --top)
find --set-root /VHD/Mini-10x64-2004-2.vhd
map --mem /VHD/Mini-10x64-2004-2.vhd (hd)
chainloader (hd-1)

alacran (reboot.pro)

@yaya2007
Copy link
Collaborator

yaya2007 commented Jan 2, 2021

Oh, I see.
It has been said that UEFI firmware will not automatically allocate more than 4GB of memory.Must be forced to load to 4GB or more memory. It seems that this view is wrong .

@Taviruni
Copy link
Author

Taviruni commented Jan 2, 2021

Remember the firmware of my Asus MB has an option called Memory Remap Feature to allow to enable remapping the memory above 4GB and it is set by default as enabled. Maybe some other brand don't have it and then from there came that idea.

#248 (comment)

alacran (reboot.pro)

@Taviruni
Copy link
Author

Taviruni commented Jan 2, 2021

I decided to make a single partition VHD of 3 GB, to test the time required to load it in Ram, and booting first with a1ve's grub2 and chainloading to G4E to load and boot the VHD:

10x64-LZX-3.vhd (NTFS) 3GB Grub2 + grub4dos >>> 33.82 sec.

10x64-LZX-3.vhd.lz4 (NTFS) 3GB Grub2 + grub4dos >>> 25.43 sec.

NOTE: I used the "ok version"

alacran (reboot.pro)

@Taviruni
Copy link
Author

Taviruni commented Jan 2, 2021

title Boot /VHD/10x64-35.vhd - UEFI Grub4dos SVBus RAMDISK - 3.5 GB
find --set-root --ignore-floppies --ignore-cd /VHD/10x64-35.vhd
map --mem --top /VHD/10x64-35.vhd (hd)
chainloader (hd-1)

NOTE: Same WIM file applied on Compact LZX mode too, this time it is a 2 partitions VHD.

Also booted fine booting to G2 and chainloadig to G4E "ok version"

alacran (reboot.pro)

@liuzhaoyzz
Copy link

liuzhaoyzz commented Jan 3, 2021

BOOTX64_ok.rar.txt,without --top,I got the same error like alacran.Most of my vhds are larger than 4GB.The --top parameter is necessory.It's hard for g4e/grub2 to handle it automatically.We have proved this before.

out of map memory: 8000000000000009 and keyboard is dead and I have to reboot.

Tested chainloading from a1ive's grub2 to G4E and also booting directly to G4E and same thing in both cases.

But a previous version used on #248 (comment)
was able to boot the 2.3 GB VHD without --top if booting directly to G4E

title Mini-10x64-2004-2.vhd (2360 MB) SVBus RAMDISK UEFI boot from HD (no --top)
find --set-root /VHD/Mini-10x64-2004-2.vhd
map --mem /VHD/Mini-10x64-2004-2.vhd (hd)
chainloader (hd-1)

@Taviruni
Copy link
Author

Taviruni commented Jan 3, 2021

Test booting a RamOS on a 3.8 GB VHD (Not compressed):

I was able to Ramboot a 3.8 GB VHD NTFS single partition, on same PC with 8 GB of Ram, in this case this is a normal (uncompressed) installation, and it booted fine.

As in previous case booted first to a1ive's grub2 and chainloaded to UEFI grub4dos, "ok-version"

a1ive asked me to test a new version of his grub2 and this are the results:

please use this grubx64.efi to load vhd: https://drive.google.com/file/d/1ut-WZBz8ImZ8mwUy_XIIPLciCvY5V1U-/view?usp=sharing

Tested following menuentries:

menuentry "Boot /VHD/Mini-10-UEFI.vhd -l (NTFS) UEFI RAMDISK 2 GB" "/VHD/Mini-10-UEFI.vhd" {
efiload /EFI/grub/ntfs_x64.efi
search --no-floppy --set --file $2
map --mem --rt -l $2
}

menuentry "Boot /VHD/Mini-10-UEFI.vhd (NTFS) UEFI RAMDISK 2 GB" "/VHD/Mini-10-UEFI.vhd" {
efiload /EFI/grub/ntfs_x64.efi
search --no-floppy --set --file $2
map --mem --rt $2
}

First with -l do not work
Second without -l works fine

The VHD file has a single NTFS partition, and was installed on Compact mode (LZX compressed), and it is located on a standard MBR uncompressed NTFS partition.

Also tested following menuentries:

menuentry "Boot /VHD/10x64-38.vhd - UEFI Grub2 SVBus RAMDISK - 3.8 GB" {
efiload /EFI/grub/ntfs_x64.efi
search --file --set=vhd_drive --no-floppy /VHD/10x64-38.vhd
map --mem --rt ($vhd_drive)/VHD/10x64-38.vhd
boot
}

menuentry "Boot /VHD/10x64-38.vhd -l UEFI Grub2 SVBus RAMDISK - 3.8 GB" {
efiload /EFI/grub/ntfs_x64.efi
search --file --set=vhd_drive --no-floppy /VHD/10x64-38.vhd
map --mem --rt -l ($vhd_drive)/VHD/10x64-38.vhd
boot
}

In this case this is a normal (uncompressed) install on a 3.8 GB VHD, single NTFS partition VHD.

The first menuentry without -l booted fine.
The second menuentry with -l do not load to Ram.

alacran (reboot.pro)

@Taviruni
Copy link
Author

Taviruni commented Jan 3, 2021

Test booting a RamOS on a 4000 MB (3.9 GB) VHD (Not compressed):

Just to test the limits I ran this test.

I was able to Ramboot a 4000 MB VHD NTFS single partition, on same PC with 8 GB of Ram, in this case this is same normal (uncompressed) installation, and it booted fine.

To test this, I increased the VHD size in command line and latter just expanded the single NTFS partition and edited the menu.lst, the VHD was not fragmented when cheked with Winconting.

As in previous case, booted first to a1ive's grub2 and chainloaded to UEFI grub4dos.

I don't think it is a good idea to go further as the memory above 4 GB available on this PC, is 4 GB or 4096 MB.

Then I can conclude with this "ok-version" of UEFI grub4dos, we can make use of all the memory available on the PC above 4 GB, without any issue.

alacran (reboot.pro)

@yaya2007
Copy link
Collaborator

yaya2007 commented Jan 3, 2021

What good news!

@Taviruni
Copy link
Author

Taviruni commented Jan 11, 2021

Just to report grub4dos-for_UEFI-2021-01-10 new version is working fine here too.

No problem when loading to Ram and booting (externally uncompressed) VHDs installed on Wimboot or Compact mode.

Also if they are (externally) lz4 compressed they are loaded fine to Ram and boot fine too.

NOTE: As in previous case, booted first to a1ive's grub2 and chainloaded to UEFI grub4dos to load and boot the VHD

alacran (reboot.pro)

@Taviruni
Copy link
Author

Reporting grub4dos-for_UEFI-2021-01-12 new version is working fine here too. Tested with MBR single partition and MBR 2 partitions VHD.
No problem when loading to Ram and booting (externally uncompressed) VHDs installed on Wimboot or Compact mode.

Also if they are (externally) lz4 compressed they are loaded fine to Ram and boot fine.

As in all previous cases, booted first to a1ive's grub2 and chainloaded to grub4dos for UEFI to load and boot the VHD files.

alacran (reboot.pro)

@Taviruni
Copy link
Author

Reporting grub4dos-for_UEFI-2021-01-13 new version is also working fine here.

Tested with MBR single NTFS partition, and MBR 2 partitions (FAT-32+NTFS) VHDs.

No problem when loading to Ram and booting (externally uncompressed) VHDs installed on Wimboot or Compact mode.

Also if they are (externally) lz4 compressed they are loaded fine to Ram and boot fine.

As in all previous cases, booted first to a1ive's grub2 and chainloaded to grub4dos for UEFI to load and boot the VHD files.

alacran (reboot.pro)

@Taviruni Taviruni closed this as completed Sep 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants