High CPU usage when transferring games

[Degru]

I've noticed that Game Pipe uses lots of CPU when transferring games, and my slow Core i3 seems to bottleneck the transfer speed to around 20MB/s, when my HDD can handle up to 80MB/s. This happens when going both to/from my HDD and SSD, and there is nothing else running when this is happening. Task Manager shows around 24MB/s for Game Pipe overall, and 14MB/s write on the destination disk (I was testing with a fairly small game, ETS2, that I transferred back and forth a couple times, so it was probably cached in RAM already)

I have no programming experience at all, but I think this happens because Game Pipe uses its own internal code for copying files, and not Windows native APIs, or something like that. Until this issue is fixed, my use for Game Pipe is limited to using it to quickly look up my game IDs when moving games back and forth manually. As it stands, the transfer speed is too slow for larger games.

[DjScribbles]

Thanks for your detailed and inciteful report here, I'm sorry I didn't respond to it sooner (working on Game Pipe became really difficult for a long time after being unceremoniously rejected from Greenlight).

I was able to identify an issue in the file transfer code, where each chunk (roughly 50KB) of data being copied was checking to see if Steam was running, which uses a time consuming call to Windows to see if the process is running. Fixing this drastically reduced CPU usage while bumping transfer speeds substantially (Seeing speeds of 120MB/s at times when copying to/from my SSD, or 60MB/s between HDDs). I also removed some validation code for local copies which didn't actually do much of anything useful, which helped reduce the CPU usage a bit more.

You are correct that Game Pipe uses it's own copy-code, this is to support the LAN feature, along with allowing you to pause, cancel, and re-prioritize transfers without waiting for the current file to finish transferring (which can become pretty annoying when some game files are several gigabytes).

These changes are incorporated into the latest release: https://github.com/DjScribbles/GamePipe/releases

Thank you for taking the time to submit this issue, the details here were helpful in comparing with my original and new results.

[Degru]

Wow, thanks!