Datomic support #15
Merged
Datomic support #15
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Just going through the firt tutorial. No real code changes. Maybe I'll reuse some of it for real, maybe not.
This is my first datomic schema so it's probably quite naive but I think it's enough for storing my game data. I will add more attributes to it later actually. Also includes commented out example of how to add and read data using this schema.
The DB interceptor is now saving data to datomic db. Reading from DB is done as well but the current game model is a bit tricky to convert the db model to because of non-flat structure. I might need to finally refactor the game model to be a flat map of maps. That way I can do a simple map over the data to convert to and from db schema. I've commented out all the service_test.clj file since I will need to find a proper way to mock the datomic db. It was simple to do with in-memory object database I should be able to do something similar for datomic. Anyway, this commit will result in a non-working game but it has a bunch of working code nonetheless.
Datomic finally persists the game data. However, the commit has all the debugging stuff and inline pieces of code that I was playing with. Also it's slow. Super slow. Slow as a high priority bug slow. I will need to debug, read more about datomic and improve this. The commit is only a stepping stone to make sure I have it. Something that works but not supposed to be seen by good people of... anywhere!
So the performance issue that I thought came from adding datomic to the flow actually came from my initial fill flood implementation. It seems that the new board data structure (flat map vs nested vectors) allowed my initial 4-way recursive naive implementation to run wild and not be stopped by the base case (since the base case happened in one branch, other 3 didn't know about it). Plus I always wanted to change that algorigh since always increasing the recursion tree by four branches seemed like super duper overkill. The new algorithm (hiding under fill-flood-loop) is a simple refactoring of the fill flood to use q and a single trunk recursion without any branches. We just keep adding the new fillable cells to the q until we run out of new cells to fill or we reach a border. I've also made small code tweaks here and there and updated the board tests. As before version was updated manually to bust the cache. I've removed the notes file cause I don't use it at all. db_util.clj is still dirty (has commented chunks of repl code) and interceptors tests are still in need to be updated to cover for datomic usage.
Since there's no chance that the size of the board changes during game I can safely memoize these two functions since they will always return the same value and the value now is calculated. It used to be a const but I want to make the engine more flexible so I always calculate the size of the board. But once the calculation is done we can cache it.
Had to shuffle some things around. Didn't manage to do the start-dev/restart cycle through integrant only, but the production system is completely setup from the system.edn file. Next step would be to move the production credentials outside the edn file to env variables for example. That way I can still commit the config but do not have to share any important info with the world.
I'm using wallmart's dyn-end package to inject dynamic properties into system config file. That way I can populate critical props with env variables and not have them stored in the source countrol. I'm also providing an example of how to start-up the server and populate all the needed props. The example is currently a PowerShell script since I'm using windows 10 as my main dev env at the moment. I will add example for linux later when I finally make myself create proper tools for production release. I've also bumped up the version of the game and cleaned-up some unused code and change the target uberjar filer name to be simply zots.jar - don't want version number in the name - that makes future CI/CD scripting more difficult (without any additional value, since the versioning should be done somewhere else - not in the file name).
The Run book is the important part of the commit. It includes all the commands and steps I've used to configure datomic and run the game connecting to an AWS transactor. Also this commit means that I've setup the AWS datomic system (storage + transactor) and the game is currently sending the data and reading the data from a remote transactor. So the games are now persistent.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a huuuge pull request. Both in terms of code changes and system changes. In short it includes:
In the future I do need to create a better deployment and start-up system, maybe use supervisor correctly, maybe get a separate instance for the peer server. I need to make the server side logic testable again (now it's coupled with interceptors and the db). I need to clean-up and update the runbook most probably.