Python 3 and Chunkloading Yaml to prevent memory crashes #9
Conversation
|
I've tested both loading into Sqlite and MySQL/MariaDB and they are working fine, however haven't tested others. |
… issues on small servers
Caffe1neAdd1ct
force-pushed
the
master
branch
from
726399e
to
5490663
Compare
Conflicts: README.md tableloader/tableFunctions/blueprints.py tableloader/tableFunctions/bsdTables.py tableloader/tableFunctions/categories.py tableloader/tableFunctions/certificates.py tableloader/tableFunctions/graphics.py tableloader/tableFunctions/groups.py tableloader/tableFunctions/icons.py tableloader/tableFunctions/skins.py tableloader/tableFunctions/types.py tableloader/tableFunctions/universe.py
|
Merged and updated changes |
…t if this is needed in python 3
|
I've merged all the latest changes, fixed any new code to work with Python 3 This needs a tidy up at some point but works perfectly for now 👍 |
files to initially expected location, added connection exception handling and db engine option check
|
Updated again and had a bit of a tidy up... App config loader added Might be worth pulling this into a python3 branch? |
|
@fuzzysteve would you be interested in this pull request if i updated? |
Disclaimer: I'm no python developer
But needed to use your yaml loader for an app i've started building in PHP with the eve APIs, however needed the item data as well.
I've managed to get this running on an 8GB of RAM server, however needed to implement "chunk loading" of the yaml files to stop the whole files going into RAM (resulting in system crashes).
Also done an upgrade to python 3, however i completely understand if you don't want to pull these changes in. There is also a fix for UTF-8 handling for yaml files as well.
I would appreciate the pulling / implementing of the chunk loading into your master as this would help out a few of us with lower spec machines, see what you think 👍