New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Starting the game server takes too long #10

Closed
sven-n opened this Issue Jan 4, 2018 · 4 comments

Comments

Projects
None yet
1 participant
@sven-n
Copy link
Member

sven-n commented Jan 4, 2018

After extending some game map initialization data where I added NPCs and Monsters and their spawn points with 792148e, the startup time has gone up from around 15 seconds to over a minute. This is caused by loading the GameConfiguration which includes all this data.
I think when adding the rest of the data to the configuration it could take several minutes to load the GameConfiguration - not acceptable.

Maybe it's time to investigate how to integrate a document database (or a similar approach), where we have the GameConfiguration in just one document.

@sven-n

This comment has been minimized.

Copy link
Member Author

sven-n commented Jan 4, 2018

I have another idea as well:
We could build a view (or a very big sql query ;)) to load the whole GameConfiguration object at once - as json or xml. We would save a lot of database roundtrips this way and still keep all benefits of a relational database.

sven-n added a commit that referenced this issue Jan 7, 2018

Added a query builder which creates a query to return a whole object …
…graph as json by using postgres json functions.

Part of a proof-of-concept for #10 - not complete or polished yet. The
usage of the query and parsing of the json result is not implemented
yet. Also many-to-many relationships are not considered yet.
@sven-n

This comment has been minimized.

Copy link
Member Author

sven-n commented Jan 8, 2018

I added a query builder yesterday. Today, I refined it a bit and it seems to be a bit tougher task to deserialize it to an object - my brain is actually hurting a bit ;-) Just a small list of problems I need to solve next:

  • We need to eliminate some shadow properties which EF Core depend on - they are not assigned by the json parser, of course.
  • To reference other objects, I use $ref-objects (newtonsoft.json style). Some objects are referenced before they actually occure, so they can't get resolved. I naively solved it by parsing the json twice but that leads to other problems:
    • it's not good for performance, obviously
    • we have multiple objects with the same id (from the first read and the second one) - EF Core change tracker doesn't like it. Ofc, we could deactivate tracking, but it's not our goal ;)

The good news is, it actually seems to work. In my first tests the loading time went down to 10 seconds, with much room for improvement :)

sven-n added a commit that referenced this issue Jan 10, 2018

Changed loading of GameConfiguration objects to use queries which ret…
…urn them as json for better performance

See also #10.
Also prepared to use it for accounts - we still have some minor issues
there. For example an ItemStorage loads it's items, and items load the
item storage which tries to load it's items again... and so on. So the
JsonObjectLoader needs some adjustments to handle such cases.
Another limitation of the new approach for the GameConfiguration is,
that the DbContext can't track the changes anymore. It contains circular
references, so we load this json twice, whereas the first run is just
there to fill the cache of the IdReferenceResolver - leading to
different (but same) instances with the same Id - the change tracker is
not afraid about that ;)
@sven-n

This comment has been minimized.

Copy link
Member Author

sven-n commented Jan 10, 2018

I got it now working completely (as far as I can see...). The server is now using json queries to retrieve the GameConfiguration. It takes now ~5 seconds to load it - not so bad :)
The double parsing takes about 600ms, so it's negligible - but change tracking needs to stay deactivated for now, as we have some circular references which are not resolveable.
I'm also trying to get it working for accounts, as their structure can get very complex, too. Imagine an account filled full of items which have a lot of item options...
This also works but has some issues. For example an ItemStorage loads it's items, and items load the item storage which tries to load it's items again... and so on. So the JsonObjectLoader needs some adjustments to handle such cases.

@sven-n

This comment has been minimized.

Copy link
Member Author

sven-n commented Jan 10, 2018

Created issue #11 for the account loading. As this issue is resolved for now, I'm closing it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment