Support the new cover format #57

Lucki · 2019-09-22T20:47:37Z

This adds support for the new library format (600x900).
Google search is disabled because of the really bad results. We should add some additional sources like IGDB or SteamGridDB but this will need an api key as user input/parameter/config.
Apng gets preserved if no overlay is applied.

I've never done something in GO before so let me know if there's something to improve.

Fix #54

Lucki · 2019-09-23T14:15:55Z

It's a bit strange that the covers are cut off at the bottom. Here you can see my test overlay where the bottom outline isn't visible:

Also the images for custom games aren't visible anymore.

With the new formats there are now many possible variations available. With different stores and databases these covers vary in sizes so let's scale them to the overlay size as that's probably the correct format.

This allows to remove overlays, fixes boppreh#51 and the applied overlay counter.

Lucki · 2019-10-09T15:47:10Z

SteamGridDB has implemented the required api calls. Since this still hasn't been merged or received comments I have merged SteamGridDB and IGDB support into this PR.

You can optionally provide your api keys (SteamGridDB, IGDB) like this:

./steamgrid --steamgriddb "steamgriddb-api-key" --igdb "igdb-api-key"

strzlee · 2019-10-13T11:42:11Z

This doesn't work for "Non-Steam-Games". Seems the new cover-files for non-steam-games are named different than the old ones.

Example (GOG Version of the Witcher 3):
old grid image: 13431636140889210880.jpeg
new cover image : 3127296488p.jpg

Lucki · 2019-10-13T14:54:22Z

Steam refused to set a custom image for my added games. (nothing happens when I set a custom artwork)
Do they work for you when set from inside steam? (I guess this is a yes)
Do you know how the new name is known? The current/old way is here:

steamgrid/games.go

Lines 114 to 120 in eac4d3f

    
           gameName := gameGroups[1] 
        
           target := gameGroups[2] 
        
           uniqueName := bytes.Join([][]byte{target, gameName}, []byte("")) 
        
           // Does IEEE CRC32 of target concatenated with gameName, then convert 
        
           // to 64bit Steam ID. No idea why Steam chose this operation. 
        
           top := uint64(crc32.ChecksumIEEE(uniqueName)) | 0x80000000 
        
           gameID := strconv.FormatUint(top<<32|0x02000000, 10)

When comparing with my file I spotted some differences in my actual file:

- (?i)appname\x00(.+?)\x00\x01exe\x00(.+?)\x00\x01.+?\x00tags\x00(.*?)\x08\x08
+ (?i)\x00\x01appname\x00(.+?)\x00\x01exe\x00(.+?)\x00\x01.+?\x00tags\x00\x01(.*?)\x08\x08

boppreh · 2019-10-20T13:07:03Z

Call me crazy, but I still cannot find this new cover format used anywhere on Steam. Is this is an Open Beta UI kind of thing?

PoorPocketsMcNewHold · 2019-10-20T13:31:57Z

Call me crazy, but I still cannot find this new cover format used anywhere on Steam. Is this is an Open Beta UI kind of thing?

It is. Go opt-in for the Steam Open Beta to test it out.

strzlee · 2019-10-22T19:59:13Z

Steam refused to set a custom image for my added games. (nothing happens when I set a custom artwork)
Do they work for you when set from inside steam? (I guess this is a yes)
Do you know how the new name is known? The current/old way is here:

steamgrid/games.go

Lines 114 to 120 in eac4d3f

gameName := gameGroups[1]

target := gameGroups[2]

uniqueName := bytes.Join([][]byte{target, gameName}, []byte(""))

// Does IEEE CRC32 of target concatenated with gameName, then convert

// to 64bit Steam ID. No idea why Steam chose this operation.

top := uint64(crc32.ChecksumIEEE(uniqueName)) | 0x80000000

gameID := strconv.FormatUint(top<<32|0x02000000, 10)

When comparing with my file I spotted some differences in my actual file:
- (?i)appname\x00(.+?)\x00\x01exe\x00(.+?)\x00\x01.+?\x00tags\x00(.*?)\x08\x08
+ (?i)\x00\x01appname\x00(.+?)\x00\x01exe\x00(.+?)\x00\x01.+?\x00tags\x00\x01(.*?)\x08\x08

The filename for the new ui covers is what the operation at line 119 generates.

For my example "The Witcher 3 - Wild Hunt":

fmt.Println(top)

3127296488

fmt.Println(gameID)

13431636140889210880

image-files in ".../config/grid/":

3127296488p.jpg
13431636140889210880.jpeg

Lucki · 2019-10-23T09:41:24Z

Thanks @strzlee! While I'm still unable to set them manually through steam this worked:

Image

Lucki · 2019-10-25T09:56:30Z

Setting the hero image and the logo is now possible too, yet I'm still unable to set them here so I'd appreciate if someone can tell me if the naming scheme is somehow similar.

strzlee · 2019-10-25T15:18:21Z

Setting the hero image and the logo is now possible too, yet I'm still unable to set them here so I'd appreciate if someone can tell me if the naming scheme is somehow similar.

According to my previous example:

Cover: 3127296488p.jpg
Hero: 3127296488_hero.jpg
Logo: 3127296488_logo.jpg
Grid: 13431636140889210880.jpg

Lucki · 2019-10-25T16:17:30Z

Are you sure about that grid image?
The grid image I see is set through 3127296488.jpg (without an additional extension). It's the first shown image when hovering over a cover or can be seen on the last played slot.

Thanks!

strzlee · 2019-10-25T17:39:25Z

With "Grid" I refer to the classic format for "non-Beta-Clients" respectively the "Big-Picture-Mode".

downloads disabled for now until stemgriddb is ready

Lucki · 2019-11-09T02:25:08Z

Apngs are now supported but there are still some issues I have to figure out while sticking the overlay on top.
Also: They're getting HUGE and heroes are partially supported by steamgriddb.

Lucki · 2019-11-09T15:14:50Z

@Logerfo APNGs are working now. I advise to not use an overlay with something half transparent as that'll get darker over time.

strzlee · 2019-11-10T11:08:30Z

@Lucki

Sometimes I encounter timeouts and errors from the SteamGridDB-API requests. Maybe some sort of rate-limiting?! Reducing (or slowing down) the calls would perhaps help.

E.g.: For non-steam-games you call a GET request with the steam ID (which always fails),
then a GET /search/game-name & a GET to /game/SteamGridID. The first request, which always fails, could here be avoided.

Logerfo · 2019-11-10T15:55:12Z

@Lucki I think it would be very useful to have a filter that only looks for animated covers. I want animated covers for my games, but I have a lot of games in my library, so it would take a very long time to search for animated covers for all of them.

Lucki · 2019-11-10T15:56:16Z

@Logerfo --help lists all available options

boppreh · 2019-11-15T11:30:28Z

@Lucki, what's the current status of this PR? Is it ready to merge, do you want to add more stuff, do you need help fixing anything?

Lucki · 2019-11-15T11:44:03Z

@boppreh
I wanted to add logo download from steamgriddb but that doesn't seems to be that near:

I don't feel like touching the site until the Manager is fixed again.

So I think this is ready? Couldn't find any problem in my test runs. There aren't any remaining problems mentioned from @strzlee and @PoorPocketsMcNewHold , although I'm unsure about the timeout issue. While a lot of options are added the fire and forget design remained intact.
Since there wasn't much feedback I'm unsure if it's good but at least it'll work.

strzlee · 2019-11-15T20:10:56Z

@boppreh
I wanted to add logo download from steamgriddb but that doesn't seems to be that near:

I don't feel like touching the site until the Manager is fixed again.

So I think this is ready? Couldn't find any problem in my test runs. There aren't any remaining problems mentioned from @strzlee and @PoorPocketsMcNewHold , although I'm unsure about the timeout issue. While a lot of options are added the fire and forget design remained intact.
Since there wasn't much feedback I'm unsure if it's good but at least it'll work.

I tested the branch in his current state on several windows and linux machines with different configurations and stepped through a couple of debug sessions with "delve".
From my view it's stable and works well.

I only encountered some inexplicable results when querying the steamgriddb-api for banner with the "dimensions" parameter. But that's probably a server-side issue.

Edit: from the steamgriddb-api documentation

dimensions | Array of string | Items Enum:"460x215""920x430""600x900""342x482""legacy" | Filter results by image dimension. Multiple dimensions can be provided as comma seperated strings. Note: It is not suggested to use "460x215" "920x430", or "legacy" yet as they are unreliable. The "legacy" size WILL be depricated in the future.

Lucki · 2019-12-09T14:45:57Z

Since nothing's going on here or at steamgriddb I've uploaded prebuild binaries: https://github.com/Lucki/steamgrid/releases/tag/v3.0.0

cbec-dev · 2019-12-11T15:19:50Z

Since nothing's going on here or at steamgriddb I've uploaded prebuild binaries: https://github.com/Lucki/steamgrid/releases/tag/v3.0.0

Hey thanks for adding builds, I tried it on windows and it works but for some reason it ignores non steam games, not sure what the issue is.

Lucki · 2019-12-12T10:21:33Z

Please have a look into your /userdata/<id>/config/shortcuts.vdf with a hex editor and verify the sequence is similar to the mentioned above:

- (?i)appname\x00(.+?)\x00\x01exe\x00(.+?)\x00\x01.+?\x00tags\x00(.*?)\x08\x08
+ (?i)\x00\x01appname\x00(.+?)\x00\x01exe\x00(.+?)\x00\x01.+?\x00tags\x00\x01(.*?)\x08\x08

Perturbator666 · 2019-12-16T21:08:37Z

Since nothing's going on here or at steamgriddb I've uploaded prebuild binaries: https://github.com/Lucki/steamgrid/releases/tag/v3.0.0

Can't locate STEAMPATH

Lucki · 2019-12-17T10:11:16Z

I haven't changed anything substantial in the steam detection. Did it work for you with the old version?
Please try again by giving the steam path with --steamdir.

Perturbator666 · 2019-12-17T21:23:09Z

I haven't changed anything substantial in the steam detection. Did it work for you with the old version?
Please try again by giving the steam path with --steamdir.

That worked.

boppreh · 2019-12-18T02:58:45Z

steamgrid.go

+		// LogoLQ: 640 x 360
+		// LogoHQ: 1280 x 720
+		// artStyle: ["idExtension", "nameExtension", steamExtension, dimXHQ, dimYHQ, dimXLQ, dimYLQ]
+		"Banner": []string{"", ".banner", "header.jpg", "920", "430", "460", "215"},


Should probably use a record/struct value instead of an array of strings, so that accesses are clearer.

boppreh · 2019-12-18T03:00:35Z

README.md

+    * Add the extension `.cover`/`p` before the image extension for cover art: `Psychonauts.cover.png`, `3830p.png`
+    * Add the extension `.hero`/`_hero` before the image extension for hero art `Psychonauts.hero.png`, `3830_hero.png`
+    * Add the extension `.logo`/`_hero` before the image extension for logo art `Psychonauts.logo.png`, `3830_logo.png`
+4. *(optional)* Generate a some API Keys to enhance the automatic search:


Suggested change

4. *(optional)* Generate a some API Keys to enhance the automatic search:

4. *(optional)* Generate some API Keys to enhance the automatic search:

boppreh · 2019-12-18T03:04:22Z

Looks stable enough and also passed my tests.

That was an amazing tour de force. Thank you very much for this contribution, from the large parts (cover format, new APIs) to the small (more reliable parsing of steam files). Will be releasing a new version ASAP.

boppreh · 2019-12-18T03:16:27Z

New release! My superficial tests worked, but please let me know if you run into any problems or have further suggestions. Unfortunately I don't have a lot of spare time to dedicate to this tool, but I'm glad it's still helping people.

gitscosh · 2019-12-18T16:53:15Z

Thank you to everyone who made this happen!

Lucki · 2021-12-02T18:03:17Z

It's a bit strange that the covers are cut off at the bottom. Here you can see my test overlay where the bottom outline isn't visible:

Picture

Today's Steam Beta update fixed this issue:

Fix game images being truncated by a couple of pixels in shelves and collections.

Support the new cover format

a0b5aba

Lucki force-pushed the new_cover branch from 3975d7d to a0b5aba Compare September 25, 2019 13:23

Lucki added 4 commits September 25, 2019 15:37

Change the extension at download

6265db4

simplify, enhance

386f0b5

Scale!

a97057c

With the new formats there are now many possible variations available. With different stores and databases these covers vary in sizes so let's scale them to the overlay size as that's probably the correct format.

Treat exising images without backup as 'clean' instead of 'overlay'

8b554d8

This allows to remove overlays, fixes boppreh#51 and the applied overlay counter.

Lucki force-pushed the new_cover branch from 9a02c5c to 8b554d8 Compare September 27, 2019 13:27

Lucki added 9 commits September 27, 2019 15:35

Add support for SteamGridDB

371637e

Also try searching at SteamGridDB

39c2160

Implement command line parsing

8933e2f

Fix url overwrite

720e2c2

Add support for IGDB

e47b8e3

Catch false aspect ratios

c274f59

Fix url overwrite

fdce418

Merge branch 'steamgriddb' into new_cover

841a4eb

Merge branch 'igdb' into new_cover

f6bea28

Simplified naming

812d5bd

Support hero, logo and old naming

48a7b51

downloads disabled for now until stemgriddb is ready

strzlee and others added 3 commits November 9, 2019 02:03

flag added: search only for non-steam-games

9694539

Fully support apng images

e5766b1

be consistent with steamgriddb

0c7af6b

Fix blending issue

35a0324

Skip appID request for custom games

5203ae1

Fixed error check

e2e59b3

Also catch wrong token for custom ids

0d48652

boppreh reviewed Dec 18, 2019

View reviewed changes

boppreh approved these changes Dec 18, 2019

View reviewed changes

boppreh merged commit b833674 into boppreh:master Dec 18, 2019

Lucki mentioned this pull request Nov 2, 2021

Setting custom background and logos for Non-Grid-View #98

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support the new cover format #57

Support the new cover format #57

Lucki commented Sep 22, 2019

Lucki commented Sep 23, 2019 •

edited

Loading

Lucki commented Oct 9, 2019

strzlee commented Oct 13, 2019

Lucki commented Oct 13, 2019 •

edited

Loading

boppreh commented Oct 20, 2019

PoorPocketsMcNewHold commented Oct 20, 2019

strzlee commented Oct 22, 2019

Lucki commented Oct 23, 2019 •

edited

Loading

Lucki commented Oct 25, 2019

strzlee commented Oct 25, 2019 •

edited

Loading

Lucki commented Oct 25, 2019 •

edited

Loading

strzlee commented Oct 25, 2019

Lucki commented Nov 9, 2019 •

edited

Loading

Lucki commented Nov 9, 2019

strzlee commented Nov 10, 2019

Logerfo commented Nov 10, 2019

Lucki commented Nov 10, 2019

boppreh commented Nov 15, 2019

Lucki commented Nov 15, 2019

strzlee commented Nov 15, 2019 •

edited

Loading

Lucki commented Dec 9, 2019

cbec-dev commented Dec 11, 2019

Lucki commented Dec 12, 2019

Perturbator666 commented Dec 16, 2019

Lucki commented Dec 17, 2019

Perturbator666 commented Dec 17, 2019

boppreh Dec 18, 2019

boppreh Dec 18, 2019

boppreh commented Dec 18, 2019 •

edited

Loading

boppreh commented Dec 18, 2019

gitscosh commented Dec 18, 2019

Lucki commented Dec 2, 2021

	4. (optional) Generate a some API Keys to enhance the automatic search:
	4. (optional) Generate some API Keys to enhance the automatic search:

Support the new cover format #57

Support the new cover format #57

Conversation

Lucki commented Sep 22, 2019

Lucki commented Sep 23, 2019 • edited Loading

Lucki commented Oct 9, 2019

strzlee commented Oct 13, 2019

Lucki commented Oct 13, 2019 • edited Loading

boppreh commented Oct 20, 2019

PoorPocketsMcNewHold commented Oct 20, 2019

strzlee commented Oct 22, 2019

Lucki commented Oct 23, 2019 • edited Loading

Lucki commented Oct 25, 2019

strzlee commented Oct 25, 2019 • edited Loading

Lucki commented Oct 25, 2019 • edited Loading

strzlee commented Oct 25, 2019

Lucki commented Nov 9, 2019 • edited Loading

Lucki commented Nov 9, 2019

strzlee commented Nov 10, 2019

Logerfo commented Nov 10, 2019

Lucki commented Nov 10, 2019

boppreh commented Nov 15, 2019

Lucki commented Nov 15, 2019

strzlee commented Nov 15, 2019 • edited Loading

Lucki commented Dec 9, 2019

cbec-dev commented Dec 11, 2019

Lucki commented Dec 12, 2019

Perturbator666 commented Dec 16, 2019

Lucki commented Dec 17, 2019

Perturbator666 commented Dec 17, 2019

boppreh Dec 18, 2019

Choose a reason for hiding this comment

boppreh Dec 18, 2019

Choose a reason for hiding this comment

boppreh commented Dec 18, 2019 • edited Loading

boppreh commented Dec 18, 2019

gitscosh commented Dec 18, 2019

Lucki commented Dec 2, 2021

Lucki commented Sep 23, 2019 •

edited

Loading

Lucki commented Oct 13, 2019 •

edited

Loading

Lucki commented Oct 23, 2019 •

edited

Loading

strzlee commented Oct 25, 2019 •

edited

Loading

Lucki commented Oct 25, 2019 •

edited

Loading

Lucki commented Nov 9, 2019 •

edited

Loading

strzlee commented Nov 15, 2019 •

edited

Loading

boppreh commented Dec 18, 2019 •

edited

Loading