Allow copying games to fast storage

[labre]

Bug description

I’d like to synchronize game data to/from tmpfs with a prelaunch-script, but have noticed, that prelaunch() and configure_game() from game.py interfere with it through a folder and file existence check.

Having game data in ram could give huge performance boosts for HDD bound games. I have Path of Exile in mind, which loads game elements just in time to prevent information leaks through area load times. It does require lots of ram though, since most HDD bound games have a size of at least 20GB, so 64GB are a minimum. On the other hand, there might be similar use cases like copying game data from network storage and maybe more things, that I have not yet thought of.

I prepared a patch and a PR before having a look on the contributing guidelines. Since there is no milestone for this feature and I still would like to have this in a release in the future, I kindly ask you to put it on the 0.5.11 milestone. Please? I don’t mind rebasing and adopting my little patch 1 to that version, since it’s quite small. Without the option, it seems to allow wine prefix creation currently though, as opposed to unpatched. I’ll investigate that.

I have working prelaunch-script examples for the Wine runner. 2 Steam will be more complicated, since the library would have to be created on tmpfs and games copied there. This also has to happen before Steam starts. This will take a little bit more testing.

How to Reproduce

Steps to reproduce the behavior:

1. Set game executable and prefix to tmpfs
2. Configure a prelaunch-script, that would copy the game there

Expected behavior

– Not breaking when the executable is not found, but rather wait for the prelaunch-script to complete

Log output

2022-03-01 01:49:03,535: Package 'distro' unavailable. Unable to read Linux distribution
INFO     2022-03-01 01:49:03,822 [startup.init_lutris:164]:Starting Lutris 0.5.10
WARNING  2022-03-01 01:49:03,851 [libretro.get_libretro_cores:24]:No folder at /home/manu/.local/share/lutris/runners/retroarch/
DEBUG    2022-03-01 01:49:03,857 [xrandr._get_vidmodes:15]:Retrieving video modes from XrandR
INFO     2022-03-01 01:49:03,891 [startup.check_driver:65]:Running AMD Mesa driver 21.3.5 on AMD Radeon RX 5500 XT (NAVI14, DRM 3.42.0, 5.15.23-gentoo-x86_64, LLVM 13.0.0) (0x7340)
INFO     2022-03-01 01:49:03,891 [startup.check_driver:77]:GPU: 1002:7340 1462:3820 (amdgpu drivers)
INFO     2022-03-01 01:49:03,940 [startup.update_runtime:203]:Startup complete
DEBUG    2022-03-01 01:49:04,274 [lutriswindow.update_store:430]:Showing 40 games

Note: A message stating the missing executable is shown, but nothing is logged at that point.

System Information

[System]
OS:              u n k n o w n
Arch:            x86_64
Kernel:          5.15.23-gentoo-x86_64
Desktop:         Gnome
Display Server:  x11

[CPU]
Vendor:          AuthenticAMD
Model:           AMD Ryzen Threadripper 3990X 64-Core Processor
Physical cores:  64
Logical cores:   128

[Memory]
RAM:             125.7 GB
Swap:            256.0 GB

[Graphics]
Vendor:          AMD
OpenGL Renderer: AMD Radeon RX 5500 XT (NAVI14, DRM 3.42.0, 5.15.23-gentoo-x86_64, LLVM 13.0.0)
OpenGL Version:  4.6 (Compatibility Profile) Mesa 21.3.5
OpenGL Core:     4.6 (Core Profile) Mesa 21.3.5
OpenGL ES:       OpenGL ES 3.2 Mesa 21.3.5
Vulkan:          Supported

Media (optional)

No response

Checklist:

• I'm not asking for support with a game or the wine runner.
• I have followed the above mentioned guides and have all the graphics and wine dependencies installed.
• I have checked for existing issues that describe my problem prior to opening this one.
• I understand that improperly formatted bug reports may be closed without explanation.

[strycore]

I also want a mechanism to transfer from slow storage (NAS, HDD, SFTP, ...) to fast storage (SSD, NVMe, tmpfs)

In the meantime, if the prelaunch command needs to be fixed to allow copying to tmpfs, I'd rather do that than add another option that users might not understand the purpose of.

Does the "wait for pre-launch script completion" option not work?

[labre]

Sorry for the late answer, I was quite busy lately.

I also want a mechanism to transfer from slow storage (NAS, HDD, SFTP, ...) to fast storage (SSD, NVMe, tmpfs) In the meantime, if the prelaunch command needs to be fixed to allow copying to tmpfs, I'd rather do that than add another option that users might not understand the purpose of.

I tend to agree. Doing the right thing by default is the better solution.

Does the "wait for pre-launch script completion" option not work?

Nope, at least not for this use case. The reason for that is, that lutris prepares the game startup before executing the script, so the script is executed after the wineserver (for instance) is started. It *is* waited for its completion before the prepared lutris environment executes the game binary, however, if the game does not yet exist on the tmpfs game directory, the prefix is not found, so lutris will abort the startup. Besides that I successfully tested symlinks for game library directories with Steam. So we could eventually automate the whole game directory movement (and backsynchronizing) process for at least the Steam and Wine runners.

[teotikalki]

I'd really like both this and the opposite action.

I install games to my NVME, play them a bunch... and then they mostly just sit there. I sometimes go back and play a bit; maybe some new DLC came out and I return to an older game, but there are titles that I want to keep in my 'library' but don't need to have on NVME. I could happily move them to SSD or even spinning rust, and it would be nice to have an integrated option for that, with a corresponding option to move them back if/when I decide I want to play them again.

I think that a similar situation could occur for people with large ROM collections; keep the big collection available but not in premium storage and have it be easy to migrate individual titles back and forth.

I guess I'm suggesting that this feature request could be upgraded to "Integrated Tiered Storage Options"?

[strycore]

@teotikalki this feature should cover your use case.

I also want to add a use case for this feature that is similar. On the Steam Deck, you can swap a SD card with another one and even have 2 SD cards if you use a USB-C dock. This can cause Lutris to become confused and consider a bunch of games as missing until the SD card is restored to the original mount point the game was installed on.

Lutris should use relative paths whenever possible and have a more fine grained location for the root location. Currently it's located in the system options and defaults to ~/Games. We should aim for something that is similar to what Steam is doing (+ some additional backend for storage mediums that are purely for backup purposes (Samba, NFS, SSHFS, ....)