Automatic game hacks #1894
Comments
|
Gameindex is already a huge file to handle in GIT. If we really put a database, it would be better to store it in a more efficient format. But eventually a solution to handle per game setting would be nice. |
Indeed. But either we drop the "editable/readble by people" criteria, or otherwise I don't see how in the world a 5000 entries X 10 lines each X let's say 20 characters could fit in less than the actual ~1MB. Reorganizing it as I mentioned in #1893 could quite potentially fix some redundancy (who knows, perhaps in the same order of magnitude of the additions for hacks we'd have to make) then, but with time I'm sure it will still grow. |
|
Yeah, I know. But the fact is that last time I listed the big entry in |
|
Oh, in that sense, yeah I see it being a massive problem. |
|
Top 100 in reverse (bigger file at the bottom) Generated by |
It is more complicated :) |
Top 100 what? surely all the [edit] |
|
yeah there is something fishy. 100 * 1MB give us a size biger than |
|
ok I was wrong. You can forgot what I said. |
Wrong about what? Regardless, I think we should get back to square one and evaluate this thing from the beginning.
Once we have those, we'd have a better picture of the scope and usefulness of this thing, and then be able to make a better decision on whether we want it, and if yes, then we can discuss where to store the info etc. But for now, we're not there yet. So let's start by answering 1 and 2 and get a clearer picture first. This whole hoo-ha on |
|
The above command doesn't show the real size of the object inside the pack format. But instead the size when they're unpacked. I'm prefer to be cautious than sorry. Git history is nearly 100MB already. |
About what? I don't see (yet) anything which suggested we will end up doing big updates... |
|
I was afraid that any update on the db will end up to a big increase of the git history (due to the big size of the file). But I was misleading by some git command. |
|
Ah. I think that very roughly the extra size a commit adds to the repo is proportional to the patch size, possibly compressed. But then I'm not even sure it could be calculated exactly because git may repack objects to store them more efficiently. Specifically for the games db updates, I doubt very highly that "normal" games db commits add anything meaningful to the repo size (the initial commits could have been big, because a lot of data, but from then it's incremental). But then again, we're not there yet. I'm still at "what the heck is this issue about?" part. |
|
The main point is that gsdx has a lot of settungs changing the output. The amount of settings and their labels/descriptions are highly confusing for new users. However with the recent development these settings improve the output accuracy significantly so that default settings are far from optimal settings. The question is whether it is worth to create a new database to store optimal settings for each game similiarly as it is currently done with the gameindex.dbf. gsdx so far handled all these game specific settings in a hard coded way as crc hacks. However I'd believe not all settings can be handled by the crc hack system and doing it in a hard coded way in the binary might not be the best idea. IMO settings are currently changing too fast. A new setting appears every second week and old settings are reworked or eliminated. Creating a game specific database makes currently no sense as it would've been outdated after a couple of days. One first needs to wait till gregory finishes his master plan for gsdx accuracy vs hacks and then one could consider it. |
First link in the OP.
Is there a single ps2 game that hasn't "rough spots"? :s
Gsdx hacks are the first thing that comes to my mind, but we may as well throw in just anything else that is hackable Then, on the specific [gsdx] implementation, I'm not sure. Arguably some hacks become "positive" only once certain others ones are enabled (e.g. sprite hacks after increase of resolution), so some way to conditionally enable them would be required. Performance would be at stake in some cases, absolutely true and arguable on forums. As I was saying in the OP though, there are many, many other cases where enabling a hack is a no brainer.
You might have a point, but anyway this doesn't block the "backend" to be developed in the meantime.
I never mentioned a new database. It seems a waste tbh considering:
|
|
So, so basically this issue is about gsdx game fixes per game, and storing them in a db, while possibly moving also some/all the crc hacks to that db? Or is mostly about just moving the crc hacks from the gsdx source files to a separate db file which can be edited without compiling gsdx? Specifically WRT the crc hacks, this might not be trivial because many of them involve actual logic, i.e. programming, which would be hard to support with a "db" file, IMHO. |
Not trivial but not impossible either, and I even created a proof of concept exactly for developing CRC hacks without recompiling gsdx, and even edit them while the game is running to see their effect in real time. It's at https://github.com/PCSX2/pcsx2/tree/master/tools/dynacrchack and I haven't tried it for a while. In a nutshell, you run PCSX2, and then:
All while pcsx2 keeps running. |
And even with this extremely ugly (though potentially useful) hack, it can only do the "normal" crc hacks. There are many more lines of code scattered inside GSdx where it checks if the CRC is whatever and does something a little differently as a result. These would most likely be very difficult to move to an external file. In general, the only way we can do this IMO for gsdx is:
So my guess is that this won't happen. But it's still possible. If someone wants to consider this, tcc (the compiler i used for that hack) is ~300k for windows including the include files, etc, and it should not be hard to either distribute it with pcsx2 (windwos only) or use it as a library which pcsx2 uses to compile the crc hacks in runtime. Another option is a scripting language like lua or javascript, in which the hacks will be programmed. There are good small libraries which pcsx2 can use, but it'd likely be a bit slower than compiling c code (however, the normal crc hacks are not hot code, so the perf impact could be quite small). But my guess - not gonna happen. |
|
I think the current implementation of crc hacks is just fine. Using github, necessary updates are really easy to implement. It is more about all the I think reusing the gamedbf for a plugin is not a good idea. Especially considering that a user can have/use several versions of the same plugin while the gamedbf disregards plugin versioning. It also sounds inconsequent to implement something in the core db for a plugin. IMO the only option would be to have a tickbox in gsdx plugin settings 'Use automatic HW/SW hacks' that most other settings disable/grey out. Then upon gsdx load the gsdx version and game specific game db (of type key=>value) overwrites all settings of type Implementation should be farely easy. Creating and maintaining the knowledge/data base too much effort for such a small team. |
That's why I asked to be more specific about what's this about. Sure, specific config options per game to improve the user experience is doable. Regardless, implementing JS scripting as live crc hacks code would have been fun :p |
|
I'm not sure what's the problem with CRC hacks where they already are.
But gamedbf is only shipped together with X gsdx version.
The "hack implementation" itself may be of course plugin specific, I see it. And even though gsdx is de facto the only one, "coherency" is still a good point. In other words, I mean, if the game was originally programmed to have screen divided in tiles, with borders not visible on native resolution just by miracle, and then you proceed upsampling/scaling.. If any I'd see the point in using "neutral" "tags" in the database, like "has shifted pixels", or "requires flushing".
If nobody creates it.. There won't be any difference from now. If somebody put its effort into it instead, profit.
|
|
CRC hacks mostly depends on Dx state, mostly the depth emulation. (and likely some bug in GL depth too). I think the main issue (for an user point of view) are "upscaling" hacks. |
|
Many issues arise from up scaling hence I don't think it would be a good idea to have such list. Some could be useful tho. However the one option that seems mandatory is HW mipmapping. Since all renderers share the same issue for specific games. Adding another option to the mipmapping list would be great in combination to the above changes. Automatic (Default) Basically it will work in a similar fashion the crc option does. |
... And that's why having the required quirks baked/listed in helps?
The only thing that might tamper with the plan, is when even fixes have problems, and there's some tradeoff of sort (say, accuracy for performance)
Note that we are starting to basically have an automatic setting for just about anything. |
For those issues there needs to be different configs for native and for upscaled.
It's a good thing imo. People won't have to spend time tweaking settings. Making things easier. |
Of course?
Indeed that's the very point of this issue! |
|
I could maybe do something like this for HW hacks ( bow down to my magnificent paint skills ) If checked then automatic hw hacks is disabled and you can toggle like you normally do. Not promising but I can look in to it 😛 |
|
That checkbox there is as magnificently well-fitting as it is half-assed ... :s Which is not meant to any particular c++ complex crc hacks logic/programming (I think we all agreed they are good where they are) My first idea was to add this to gameindex (since.. well, it already lists all games) - but I can see why one would be baffled from adding conditionals and more complexity to an already massive file. |
|
I think this will just end up creating more work for everyone and future contributors. I don't like adding things to their own list that then have to be actively maintained and kept up to date while still tracking regressions. This is too risky IMO, and there's just too many games to worry about. |
No maintenance, worst case... means we are simply back to current situation. Already discussed here.
Seriously, market this feature well for 1.6, and it's going to be the greatest thing invented since sliced bread. |
|
It might be easier to implement a per game config setting like in rpcs3. Although I'm not sure how it'd fit in the pcsx2 GUI. The most frustrating thing about the current model is having to switch settings every time you play a different game (unless you spend a bunch of time linking game ini's to shortcuts). |
|
Yes, per game config is the point of this? I'm not sure where the news would be. Anyway, after the discussions in #2265... I'm wondering how many hacks would require a particular "logic", if we can hardcode in gsdx "don't use this or that on native resolutions"? |
|
I personally think that not including any more per game config settings would alleviate the large git concerns. So if there was some way to easily edit in the gui that might require way less space and therefore minimize increase in git size since that seems to concern the devs. |
|
Git size concerns were a dupe, leave those alone. |
|
While good in theory, realistically this will be a nightmare to maintain. Almost every game needs some kind of game hack. Maintaining GameDB is already difficult as it is. Per game configuration Spectabis is a better alternative for now. A real solution would be improving the accuracy, detection .. etc without the need of hacks.
|
|
Good luck improving somehow upscaling? And Spectabis has no general databse that I know of. |
|
While it may seem good in theory, reality is it's a nightmare for maintenance. |
Yes, I know, people wisdom and will is crap.
And yes, I realize in some cases we are in situations comparable to a triangle, with speed, accuracy and "neatness" on the opposite vertices.
But it's also undeniable that in others, enabling them (for as much "dirty as mud") it's a no brainer. And really, it starts to feel like a joke for people to wander forums and do the same guesswork over and over again separately.
And looking at how nicely #1874 has been executed, I was dreaming a similar "automatic preset" for hacks too (handled in gameindex together with gamefixes perhaps?)
Or at least some way, link, note, whatever for ANY average joe to at least take notice he can do something.
It's depressing to see how many people just crank up resolution, and then complains even it the issue had been worked around ages before.
EDIT: supposedly Dolphin calls this "Game INIs"
The text was updated successfully, but these errors were encountered: