Finished games is a small personal project whose purposes are:
1) To provide an easy to use videogames catalog tracking, both for your owned titles (and videogame platforms), which ones have you finished or are currently playing, and to keep a wishlist of interesting games you desire. I'm going to use it frequently and wanted to build it my way.
2) To practice fully developing a website with Django 2.x. On my daily work I don't get to touch all the pieces that the framework provides, so this project is a good way to fix that.
Near the end of this README you will find detailed lists of libraries and django features used.
You can see a live demo of the project at https://finishedgames.kartones.net (user accounts restricted to only friends, not for public use, sorry!).
Also, navigating to the help quickly explains the meaning of the action buttons.
The Docker Python image is already compatible with variable type hints (>= 3.6), but as many OS still come with as much as 3.5, like Ubuntu, keeping variable type annotations as comments for now so they run with Python 3.5.
This project requires
Docker Compose to be built (only Python to run). Python required version is minimum
3.5, but the Docker container is built with
3.7. Django version is
2.2 so anyway you need Python >= 3.5.
Python package requirements are specified in the corresponding
requirements-dev.txt files. These packages are installed inside the containers in development and testing.
Most requirements are not version-pinned on purpose, if a build fails due to some new version breaking change it will be triaged and updated accordingly.
To run pending migrations (both initial setup and after an update that brings new ones):
make shell python manage.py migrate
To create an administrator/super-user (you'll need one to then create normal users from the Admin):
make shell python manage.py createsuperuser
To launch the website in development, just run:
Once loaded the site will be available from http://0.0.0.0:5000/.
Admin site is accessible from http://0.0.0.0:5000/admin/.
NOTE: Some data creation and management is done from the Admin site. One of the principles of this project is not repeating work already done or more easily done from the Django Admin.
Fetching and importing from external sources
You can manually add
Platforms to your catalog, but this can become a titanic task. The included
catalogsources django app provides an extensible adapter-based system to fetch data from external sources that provide APIs. This repo includes connectors for GiantBomb and MobyGames, but you must always request your own API keys and setup them (see
finishedgames/finishedgames/settings/prod.py.sample for expected configuration format).
Once you have added your key(s) to the settings, you can invoke fetching of platforms using a Django command:
# Command accepts one or more source_ids python3 manage.py fetch_platforms <source_id_1> [<source_id_2> ...]
Or, once you have fetched platforms from a given source, fetch games of one or more fetched platform ids:
# Command accepts one source_id but multiple platform_ids python3 manage.py fetch_games <source_id> <platform_id_1> [<platform_id_2> ...]
Now you will contain
Fetched Platforms and
Fetched Games, which you need to add to the main catalog (so they appear on the website), or hide them if you do not wish to import. This allows to make changes, handle conflicts, and the like without editing live data.
When using the Admin site to browse fetched items, there are custom convenience Actions available, like marking as hidden or importing to the main catalog.
There's also a handy and self-descriptive command:
# import up to 500 new games; exclude_unreleased skips those with date `1970` (unknown or, typically, not yet released) python3 manage.py import_fetched_games_without_fg_game 500 --exclude_unreleased
Then, when a game becomes updated, either from a fetch (e.g. gets released on new platforms) or from a manual edit in the admin interface, its
sync state will be
false. You can filter to see only un-synced games, and sync them from the web, but again there's a command that will do smart sync:
# sync up to 500 games out of sync python3 manage.py sync_games 500
The command will sync:
- Game cover
- Publish date (if greater than imported game's publish date)
- New platforms (platforms not already present in the imported game). Note that it won't remove platforms.
Fetch and import flow
+----------------+ +--------------+ +------------------+ | django command | | django admin | | main catalog/web | +--------+ |----------------| |--------------| |------------------| | | | | | | | | | source +-->| fetch_xxx +-->| import xxx +-->| <item available> | | | | | | | | | +--------+ +----------------+ +--------------+ +------------------+ ^ ^ | | | | v v +-----------------+ +----------+ | | | | | FetchedGame | | Game | | FetchedPlatform | | Platform | | | | | +-----------------+ +----------+
Running tests (including type hint checking with
To obtain the code coverage:
Note: If not run under Linux, will fail after generating the coverage as will try to open the file
cov_html/index.html in the default browser with
To create a new migration, edit the models at
core and then:
make shell python manage.py makemigrations core
Or if you need a data migration (remember to add your operation using
make shell python manage.py makemigrations --empty core
To run a Django shell if you need it:
make shell python manage.py shell
Commiting code and Code Formatting
You must install
pre-commit to run the formatters and some linters upon commiting code:
python3 -m pip install pre-commit --user pre-commit install
Demo of all NES.css available components: https://nostalgic-css.github.io/NES.css/ . Note that this website uses a project fork, as they dropped (at least for now) support for Firefox while older versions worked fine.
Personal recommendation of IDE for SQLite browsing: DB Browser for SQLite
To see the SQL query of an ORM query, use the
- To check if there are new versions of the dependencies
make shell python3 -m pip list --outdated
To setup the production settings, copy
finishedgames/finishedgames/settings/prod.py and setup your secrets (secret key, database credentials, etc.). You should never remove the
prod.py file from
.gitignored list as you could commit your production credentials.
Also remember that you need to setup the statics for production when going live, for development it works out of the box. To prepare the statics, run
Libraries and tools
Docker: for development and testing, partially done for production
requests: for fetching APIs and game covers
pillow: to manipulate and convert game covers
pytest: configured to run django
TestCasetests, including code coverage generation and
pytest-randomlyto shuffle test suites order
mypy: Fully typed code, including django classes
- Rate-limiting (custom implementation based on a token bucket algorithm)
mypymandatory linter test (any broken rule fails tests run)
blackauto-formatters upon commit (automatically reformat files)
- configuration hierarchy (
base -> dev/prod/test -> local), including different default SQLite DB
- Django >= 2.2
- Extensive ORM usage, including
prefetch_related(), query optimizations (
- custom model managers
Paginatorcomponent for pagination
- custom management commands
- custom template tags (with and without rendering templates) and filters
- admin custom ordering, filters, fieldsets, readonly fields, search fields...
- custom admin actions, decorators, filters, forms, form fields (a handy non-pg dependant
SimpleArrayField) and views
django-debug-toolbar: Disabled on production for security
Select2based configurable autofill both for the admin and for normal views
ASCII diagram built with ASCIIFlow.