New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Verify Integrity of Game Cache ignores filename capitalisation #577

Open
DarkStarSword opened this Issue Jan 7, 2013 · 11 comments

Comments

Projects
None yet
10 participants
@DarkStarSword

DarkStarSword commented Jan 7, 2013

Verify Integrity of Game Cache will claim that all files successfully verify even if some of the filenames have the incorrect case. This may happen under certain circumstances when copying the files off a filesystem that is not case sensitive, such as ntfs or vfat, or when those filesystems have been mounted with certain options (e.g. lowntfs).

For testing, this script can be used to recursively rename all files in a directory to lower case: https://raw.github.com/DarkStarSword/junk/master/steam/breakcase.py

I also have written a script that can rename all the files to the correct case listed in the .manifest files, announcement here: http://steamcommunity.com/app/221410/discussions/0/846941710185740762/

For example:

After running the above breakcase.py script on the Osmos directory, I get the following error when trying to launch it:

Game update: AppID 29180 "Osmos", ProcID 21884, IP 0.0.0.0:0
ERROR: ld.so: object 'gameoverlayrenderer.so' from LD_PRELOAD cannot be preloaded: ignored.
/bin/sh: 1: /home/ian/.local/share/Steam/SteamApps/common/Osmos/Osmos: not found
Game removed: AppID 29180 "Osmos", ProcID 21884 
saving roaming config store to 'sharedconfig.vdf'
roaming config store 2 saved successfully

Yet running verify integrity of game cache still reports "All files successfully validated"

If I then rename osmos to Osmos and osmos.bin32 to Osmos.bin32 and launch the game it does launch, but all the graphics are messed up - lots of quadrilateral shapes are drawn on the screen, but no text is drawn and it is impossible to work out what is going on - essentially Osmos can't find it's textures. Running my abovementioned check_acf.py script with --rename then allows Osmos to work properly again.

I would propose that the verify integrity of game cache feature is extended to rename files to correct any capitalisation issues in a similar manner to my check_acf.py script.

There may still be some edge cases if the filesystem that SteamApps is mounted on is case insensitive (such as using mount -t lowntfs instead of mount -t ntfs3g).

Processor Information:
    Vendor:  GenuineIntel
    Speed: 1600 Mhz
    8 logical processors
    4 physical processors
    HyperThreading:  Supported
    FCMOV:  Supported
    SSE2:  Supported
    SSE3:  Supported
    SSSE3:  Supported
    SSE4a:  Unsupported
    SSE41:  Supported
    SSE42:  Supported

Network Information:
    Network Speed:  

Operating System Version:
    Debian GNU/Linux 7.0 (wheezy) (64 bit)
    Kernel Name:  Linux
    Kernel Version:  3.7.0-rc6-ian+
    X Server vendor:  The X.Org Foundation
    X Server release:  11204000

Video Card:
    Driver:  NVIDIA Corporation Quadro FX 880M/PCIe/SSE2

    Driver Version:  3.3.0 NVIDIA 310.19
    Desktop Color Depth: 24 bits per pixel
    Monitor Refresh Rate: 60 Hz
    VendorID:  0x10de
    DeviceID:  0xa3c
    Number of Monitors:  2
    Number of Logical Video Cards:  1
    Primary Display Resolution:  1600 x 1200
    Desktop Resolution: 3200 x 1200
    Primary Display Size: 15.75" x 11.81"  (19.69" diag)
                                            40.0cm x 30.0cm  (50.0cm diag)
    Primary Bus: PCI Express 16x
    Primary VRAM: 1024 MB
    Supported MSAA Modes:  2x 4x 8x 16x 

Sound card:
    Audio device: Conexant CX20585

Memory:
    RAM:  3897 Mb

Miscellaneous:
    UI Language:  English
    LANG:  en_AU.UTF-8
    Microphone:  Not set
    Total Hard Disk Space Available:  413250 Mb
    Largest Free Hard Disk Block:  112581 Mb

Installed software:

Recent Failure Reports:
    Fri Jan  4 03:00:14 2013 GMT: file ''/tmp/dumps/assert_20130104140012_1.dmp'', upload no: ''HTTP response code said error''
    Mon Jan  7 03:30:40 2013 GMT: file ''/tmp/dumps/assert_20130107143038_1.dmp'', upload no: ''HTTP response code said error''

@johndrinkwater

This comment has been minimized.

Member

johndrinkwater commented Jan 9, 2013

Had two users in group chat have this problem so far after copying Team Fortress 2 from their Windows library to their Linux one, its a case issue so verification didn’t highlight the problem. Though, like Issue #123 it might just be labelled unsupported.

13:44 - Krywk: solved
13:44 - Krywk: my team fortress 2 folder was called 'team fortress 2'
13:44 - Krywk: Steam was looking for 'Team Fortress 2'
13:44 - Krywk: renamed the folder
13:44 - Krywk: there you go
@DarkStarSword

This comment has been minimized.

DarkStarSword commented Feb 6, 2013

I've noticed some oddities when downloading Half-Life (Beta) and Psychonauts in the Linux client where the case of some directories that Steam creates does not match that in the manifest files.

I'm not sure if this is a mismatch within the manifest files or something else, but it certainly makes things... interesting. Let me know if any of this belongs in a new bug report.

Left side is the filename on disk, right side is the capitalisation from 3840_3963299974783782956.manifest:

Psychonauts/documents - Documents
Psychonauts/workresource - WorkResources
Psychonauts/workresource/fonts - WorkResource/Fonts
Psychonauts/workresource/localization - WorkResource/Localization
Psychonauts/workresource/pclevelpackfiles - WorkResource/PCLevelPackFiles
Psychonauts/workresource/pixelshaders - WorkResource/PixelShaders
Psychonauts/workresource/savedgames - WorkResource/SAVEDGAMES
Psychonauts/workresource/scripts - WorkResource/Scripts
Psychonauts/workresource/sounds - WorkResource/Sounds
Psychonauts/workresource/textures - WorkResource/Textures
Psychonauts/workresource/textures/icons - WorkResource/Icons
Psychonauts/workresource/vertexshaders - WorkResource/VertexShaders
Psychonauts/workresource/vertexshaders/modules - WorkResource/Modules

Half Life is a bit more interesting since it has multiple manifest files with contradicting capitalisation:

Half-Life/platform/Resource
3_252432840369628186.manifest lists resource with a lower case r
8_3125558230937844363.manifest lists Resource with a capital R
Half-Life/platform/Steam
3_252432840369628186.manifest lists steam with a lower case s
8_3125558230937844363.manifest lists Steam with a capital S

My script isn't explicitly looking for these kind of contradictions, so there may be others that it didn't pick up.

@ghost ghost assigned martino-valve Nov 1, 2013

@Half-Shot

This comment has been minimized.

Half-Shot commented Mar 9, 2014

Bumping this as steam cloud also seems to remove capitalization which obviously on other operating systems doesn't matter. Cloud saving on Dust : An Elysian Tail is broken as a result.

@Cheeseness

This comment has been minimized.

Cheeseness commented Jan 29, 2015

This seems to also seems to be an issue when receiving updates to games. Within the past 6 months or so, I've had two games (Crypt of the NecroDancer and Grim Fandango Remastered) have filename case changes which weren't propagated to my Linux game cache. In both of these cases, it resulted in game breaking behaviour.

I see that this issue was marked as assigned over a year ago. Has there been any progress with getting it fixed in the client?

@CoolSilver

This comment has been minimized.

CoolSilver commented May 5, 2015

Filesystem space issues due to this bug. Dual booting and sharing game library folder creates duplicated files with different case in Linux even though they were updated in Windows. Windows sees only the one file but the file system knows the other case mismatched file exists. Storage space is being mishandled and used when it shouldn't be. Dual boot SteamOS likely will have similar issues.

@DarkStarSword

This comment has been minimized.

DarkStarSword commented May 5, 2015

@CoolSilver sharing a library between Windows and Linux is not recommended - for some games it works OK, but others had issues if the same files exist in both Windows and Linux depots with differing content causing them to require updates whenever switching OS.

It is reasonable to copy the Windows install to a separate Linux library (or vice versa) to reduce redownloading identical files though, with the caveat that the filenames from Windows may not have the correct case (hence my check_acf.py script to fix them).

@Cheeseness

This comment has been minimized.

Cheeseness commented Aug 17, 2016

It's a bit late to mention it, but this bug wreaked havoc with Firewatch's launch. It allowed a filename case change to the binary to slip under the radar, and resulted in users being unable to play for several hours while we tried to work out a nice solution that wouldn't trigger an entire re-download of the game for Linux users ( we couldn't find one ).

@Cheeseness

This comment has been minimized.

Cheeseness commented Nov 14, 2016

The Living Dungeon also hit this issue during Linux testing. The developer also claims that at some point, the case of files in the depot unexpectedly changed between build uploads.

@Plagman Plagman self-assigned this Nov 16, 2016

@Cheeseness

This comment has been minimized.

Cheeseness commented Mar 21, 2017

Is this addressed by yesterday's client update?

I don't have anything that I can specifically test to confirm with.

@Plagman

This comment has been minimized.

Member

Plagman commented Mar 22, 2017

That change only had to do with launch configurations, not SteamPipe depots, sorry.

@Plagman Plagman removed the Need Retest label Mar 22, 2017

@Cheeseness

This comment has been minimized.

Cheeseness commented Mar 25, 2017

No probs! Hope this can still get addressed at some point in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment