[issue]: Instead of dm-patch, consider a more secure and upstreamable solution that does not do kernel taint #2234
Comments
hgkamath
changed the title
hgkamath
changed the title
|
Below log comes from a boot of a fedora workstation 37 live-iso They don't come from the fedora iso. |
|
The below logs come from a vdisk ventoy-boot of Fedora-37. |
|
In this comment, I document
Native bootingIn normal OS booting, the OS is directly installed to a partition, and the UEFI transfers control to a bootloader that loads the OS directly from files laid out in a filesystem of a partition. The usefulness of live iso/vdisk multibooting solutions. The reasons for using ventoyDualboot or multiboot is having multiple operating systems on a machine, and being able to boot into them Why not a VM?:VM-s are nice but a bit slower than nativeboot. Many real hardware requiring tasks cannot be done inside a VM. Such as, get full native experience of a live iso, GPU support etc. Some system level recovery and repair tools must be booted as native. How one arrives at a multiboot solution like ventoyIn the old days Harddisks, USB drives, iso files were small. At one point I considered having a small number of 8GB microsd cards to function Disadvantages of flashing USB drive
With multiboot solutions, Its much easier to copy in and out liveisos between I've have tried creating my own grub2 configurations to loop mount isos. Then I considered other multiboot solutions like Ventoy seems to best them, by
Ventoy seems to be fork of grub2 with some additional code to handle the above, and a lot of both grub-scripting and initramfs scripting. How vdisk based system can be better than partitioning or large single volume systemsLarge single volumes, have huge filesystems and can take a long time to repair if there is a disk corruption. In contrast, a vdisk based solution will have one large partition/volume per disk, which in turn contains many reasonably sized vdisks that can be moved about. The vdisks may be of the type that contain a bootble OS ot the type that contain ordinary data. Different-OSes have different space requirements which can be solved more easily by creating a suitably sized vdisk. On the other hand, repartitioning and transferring partitions is problematic. Another problem with over-partitioned systems for multi-OS machines is free space fragmentation. The limited disk space of a 512gb SSD drive gets broken and underutilized between partitions. This leaves less usable space in a home partition. Filesystems like btrfs allow having one big volume, installing an OS to subvolumes and allow booting from subvolumes. Thereby sharing unused space. One can then backup-up, offload and restoring subvolumes on need. But, this isn't cross platform. Unlike vdisk-files, subvolumes require more mental cognitive involvement. LVM and Proxmox type virtual hosting solutions also do something similar, space management and logistics. Except, only for cloud based virtual machines. Consider having a windows and linux dual boot sytem and a data partition. One might give 64 gb to each OS. That means when booted into any one OS, the space occupied by the other OS is a waste. Before you know it, there is very little space on the SSD due to all the OS partitions. Ventoy allows one to just keep only as much as needed 40gb vdisk files in the 512 gb partition, and keep the rest offloaded. The user can easily move all unused vdisk-file images between local disks and to external storage for later use. Why being able to mount the outside partition from inside the nativebooted OS is useful
There is danger in mounting the outside partition because of the potential to touch/modify the vdisk file in which the nativeboot-ed is running. This danger is mitigated by a self contract. The self-contract to be followed while nativebooted into a vdisk and mounting the outside partitionBooting into an OS in a vdisk is called nativeboot. If the nativebooted-os wants to mount the outside partition containing the vdisk, there is a little danger that must be mitigated, which is that the fs-entry, file-fragment-locations, filesize of the vdisk file should not be modified.
This is so that the the file entry in the outside fs remains untouched. This is usually done by an administrative privilege user (root), who knows what the implications are and can set access controls to prevent inadvertent access. So it is not so much of a problem. If one adheres to the above self-contract, the filesystem code for partitions inside the dm-device does not interfere with the filesystem code for the outside partition. Windows also does nativeboot and allows write access to outside partitionHere, I'm not giving a 'because windows does, linux should do so too', justification. But, windows does do nativeboot of vhdx now since Win10-1903. Only, to mention, that nativebooting vdisks is a useful enough thing, that Microsoft Keeping interoperability with windowsVdisks also lend themselves to be mounted inside vitual machines. |
THE ANTI-VENTOY IDEAHere-in, I document an idea that just came to me. I called it anti-ventoy because a second dm-device is made from all the remaining partition blocks that the dm-device ventoy is not (gravity anti-gravity, proton anti-proton). It's just to give it a name to refer it by in this page. You can call it whatever. The reason why the host partition can't be mounted is because the The idea is : What if, in addition to The below logs are collected after the fedora vdisk boots up. By side stepping the blocks that were allocated Long ago, I had defragmented the ntfs partition in win10. so the vdisk file is a single contiguous unfragmented sequence of blocks. It would have been more work if it were fragmented but still do-able. ventoy does not require the file to be contiguous. However, I think, contiguous dm-maps are more performant. Advantages of this method:
Disadvantages of this method:
These block boundary calculations need to be perfect, in order to present a filesystem as perfect as the original. The job of the antiventoy dm-map is to just substitute the vdisk file-blocks with zero-blocks, in order to avoid the contention that prevents mount. Some testing will help with assurance of safety. I mounted the fs "ro" to be safe. Don't know what tool I can use to check the sanity of the ntfs filesystem under Linux ntfsck from ntfsprogs was not helpful. It is an incomplete implementation, which, for the moment does nothing. |
|
The idea is just how Ventoy did to support persistence. The ISO file corresponds to |
possibly yes, except that this is the blocks of the whole remaining partition stitched together with zero-block-sections that take the place of the vdisk-blocks.
Basic pseudocode algorithm to generate dmtable for antiventoyTestingMuch testing will needed just to make sure that a filesystem made this way is safe.
I also wanted to mount the filesystem as read-write, but before that I wanted to be sure that my calculations are correct. In test like these, there is the possibility that the mount might will succeed, just because most of the filesystem metadata is near the start of the partition, away from the vdisk-file. So its necessary to check that the blocks zeroed are precisely only those that correspond to the vdisk-file. I don't know if there is way to do this in Linux. but if the vdisk file can be exclusively locked after mount, and be made to remain exclusively locked until shutdown, so that no entity can attempt to move/modify the vdisk-file, after mounting |
|
Try the latest CI release: |
hgkamath
changed the title
Official FAQ
Ventoy Version
1.0.88
What about latest release
Yes. I have tried the latest release, but the bug still exist.
Try alternative boot mode
Yes. I have tried them, but the bug still exist.
BIOS Mode
UEFI Mode
Partition Style
GPT
Disk Capacity
2TB
Disk Manufacturer
Seagate
Image file checksum (if applicable)
None
Image file download link (if applicable)
No response
What happened?
This thought came while trying to get vtoyboot/VTOY_LINUX_REMOUNT to work
In a few years, there may be 3 looming trouble areas on the time horizon.
Even now, there is trouble
If it were part of kernel, then at-least patching the dm would go away and it will be by default already be secureboot signed at distribution.
If the dm-patch were upstreamed as a feature of dm into the linux-kernel, then separate maintenance will get replaced with committed maintenance. One has to keep up with the kernel anyway, whether it be via patching or in-tree. The level of code quality as well as responsiveness to new-kernel readiness will need to meet kernel-code-standards, and kernel stewards will need a committed maintenance keeping up playing nicely with with new kernel features.
In order to upstream, kernel stewards would need convincing that multibooting iso-s and nativebooting vdisks is a common usecase, and that the code for the feature will be maintained.
A kernel parameter could then enable the feature in DM on kernel-boot-cmdline, for only those who use ventoy, thereby keeping those who don't use the feature safe from misuse.
Presently the way dm_patch is inserted into kernel, as an external non-tree module, it is considered as a kernel-taint. even though ventoy is an open-source GPL project. The module patch is being distributed in binary form. This not only taints the kernel, but can also create incompatibilities as it the kernel dm module itself changes through the kernel versions. so it will work with a narrow version-range of linux kernels.
Presently, if dm_mod patch fails, then the host partition on which the vdisk lies, cannot be mounted read-write and used like a normal partition.
The feature that dm_patch provides (its purpose) is to make the vtoy-vdisk hosting partition to re-appear to the OS as unused/not-busy, so that it allows the hosting-partition to again be mountable as read-write despite the fact that devmapper mapped blocks within the partition are being used. the boot kernel cmdline parameter, VTOY_LINUX_REMOUNT, renamed in accordance with kernel parameter conventions, can be used for that purpose.
Filing this issue, so that there is something to think over, plan for and set a milestone.
Feel free to close this issue if it is in-actionable.
log
The below are executed from inside a vtoy linux boot (m01_lnx.raw.img.vtoy on an ExFAT partition sda16)
The below are executed from different linux boot
Ref
dm_patch_64.kovtoyboot.sh, and various shell scripts that run during initramfs creation and during initramfs bootup are distributed inside vtoyboot-1.0.25.iso (why iso? maybe to prevent newbies from running accidentally on working systems. The iso needs to be loop-mounted, and the filevtoyboot-1.0.25.tar.xzneeds to be copied-off and extracted)The text was updated successfully, but these errors were encountered: