Fix 404/500 issues on tools/scummvm for both systems and images #2
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issues Fixed
Root Cause and Logic to Address Issues
- Fix image logic to not issue
500
error on file size if file doesn't exit (gamedata.xml references missing file). I just ran into this fixing image upload and seemed like a good idea.ftfy
fix_text was causing the utf-8 normalization differences with the rom name. This post-processing didn't matter for the cleaned up text per say (it looks fine), but when using the file it appeared to not exist unless it's an exact binary match.find_normalized
methods on the rom name.Fix Overview
Most of the above fixes are simple to implement. However, getting the system roms directory from es_systems.cfg is not always obvious as it is often needed on a per game basis in contexts where es_systems.cfg has not been parsed. Parsing it for each game (for example) would be a huge overhead.
The proposed solution is to lazily cache the systems.cfg object in memory and use it across all APIs. This is the most controversial part of this PR as there is no existing pattern to follow for this in the code. A few thoughts as to why this is ok:
NOTE on memory: Caching the es_systems.cfg object does potentially use a bit more memory, but it should be relatively small. If this were to be an issue, parsing and keeping a dictionary with only the system paths would be a good option.
Testing
tools
system does not display a 4042021
screenshots
system does not display a 404 (might make sense to hide/enhance in the future)scummvm
system does not display a 404scummvm
game can be launched vialaunch
button.Quick memory usage check:
Nothing fancy - just check total memory used reported by
top
before and after killing python server.210.4
-194.4
=16MB
209.5
-194.0
=15.5MB