Web Services for games developers on Google App Engine
Python HTML CSS
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.idea
babel
cloudstorage
enki
locale
markdown2
passlib
pygments
pytz
static
templates
test
.gitignore
app.yaml
babel.cfg
babel.py
cron.yaml
deploy.py
dos.yaml
example_secrets.txt
index.yaml
licence.txt
main.py
readme.md
settings.py

readme.md

enkiWS

Web Services for independent games developers on Python Google App Engine

A permissively licensed Python web service for independent games developers. enkiWS is a library for setting up a website and ancillary services for games on Google App Engine.

Online demo - may be out of sync with the source code

Status

This is a work in progress and not yet ready for production use.

[ NEW in v0.16 ] Added extension email newsletter subscriptions and batch email sending; enforced Youtube privacy enhanced mode in forum posts; fixed anchor links to video in forum posts; fixed Store and Friends extensions breaking profile and admin pages.
[ v0.15 ] Added forums responsive images and video embedding and code syntax highlighting (pygment); added media gallery page; added custom 404 page not found.
[ v0.14 ] Added stay logged in, fixed localised links in store emulator.
[ v0.13 ] Added user roles; reworked the personal user profile page; moved user's licences library and sessions management to dedicated pages; updated French translation; improved icons accessibility features; refactoring: moved library functions into related model classes.
[ v0.12 ] Sticky threads and posts, Forum events email notifications to admin
[ v0.11 ] Added Admin tools page with free licence key generator, reporting cron, gcloud deployment script
[ v0.10 ] Added canonical host url, event counters for download and purchase, Store fixes and escaping, prevent remove auth method if user would only have email without pw left, email validation detects empty
[ v0.9 ] Security fixes and improvements

Functionality

Current

Base

  • User Accounts - demo
    • Display name
    • Email
    • Password change and recovery
    • Login with email/password or with OAuth & OpenID providers - Facebook, Google, Twitter, Github, Steam
    • Stay logged in
    • Manage sessions
    • Delete account
    • Extensions:
      • Library: manage licence keys
      • Manage email newsletter subscriptions
      • Manage friends
  • Security and privacy - demo
    • Backoff timer
    • User enumeration prevention
    • Account recovery via email (if account was breached and email changed by a third party)
    • Minimum user personal information stored: user email and login credentials
    • OAuth: minimum user info requested - user email and unique Id with the auth provider
    • Passwords encrypted using PassLib scheme pbkdf2_sha512
    • User display name (alias)
    • User display name can be changed but the old display name(s) remain public
    • User roles (Admin)
  • Media - demo
    • Gallery of images and videos
    • Enlarge and browse images - demo
    • Data-Driven contents and layout using JSON

Extensions

  • Online store - demo
    • Payment provider FastSpring
    • Licence key generation and activation
    • Store emulator
  • Email newsletter subscription - demo
    • Subscription with double opt-in
    • Unsubscribe links
    • Batch email sending (Admin)
  • Friends - demo
    • Search by display name and invite
    • Message alert for friend invite
  • Forums - demo
    • Posts can be edited or deleted by author
    • Sticky threads and posts
    • Posts formatting using Markdown2, including:
      • video embedding: Youtube, Vimeo, mp4, using the image syntax ![alt text](url)
      • code syntax highlighting using Pygments
  • REST API
    • Authentication (account and game key)
    • Friends list
    • Data Store

Admin

  • Admin tools
    • Reporting
    • Forum events email notifications
    • Extensions:
      • Free licence keys generator
      • Batch email sending
      • Apps management
  • Localisation - English & French - demo
  • Custom 404 Page not Found - demo

Intended for release 1.0.0

  • Installation and usage documentation
  • REST API improvements:
    • datastore limits
    • online datastore explorer
    • authentication timeout control
    • datastore object modifcation time and lifetime controls

Intended for release 1.x.x

Instructions

Running the enkiWS website locally

You can run enkiWS on your machine using the Google App Engine Launcher:

  1. Download & extract enkiWS
  2. Download & install Google App Engine with python 2.7
  3. Run GoogleAppEngineLauncher:
    1. Choose File > Add Existing Application.
    2. Set the Application Path to the directory enkiWS was extracted to (where the app.yaml file resides)
    3. Select Add - enkiWS is added to the list of project.
  4. In the GAE Launcher select enkiWS, press Run, then press Browse - the enkiWS site opens in your browser.

Debugging enkiWS locally using PyCharm CE

A .idea directory is included in the project. It is preconfigured to enable the use of the free PyCharm Community Edition as an IDE for debugging python GAE code, with one modification to make manually. Note: if you'd prefer to configure PyCharm CE yourself see the detailed tutorial. Otherwise follow the simplified instructions below:

  1. Ensure you have python 2.7 and Google app Engine installed. To check it works, try running the enkiWS website locally.
  2. Download and install Pycharm CE
  3. Start Pycharm and open the project - set the project location to the directory enkiWS was extracted to (the parent folder of the .idea directory).
  4. A Load error: undefined path variables, GAE_PATH is undefined warning is displayed. To fix it see the PyCharm tutorial Method A step 3.c. onwards.
  5. Note: if you get a message stating No Python interpreter configured for the project, go to File > Settings > Project:enkiWS > Project Interpreter and set the project interpreter to point to the location of python.exe on your computer (..\Python27\python.exe).
  6. Restart PyCharm
  7. You can now run / debug the project from PyCharm using one of the configurations provided (e.g. GAE_config).

Enabling OAuth login with Google, Facebook, Twitter, Github

To set up Open Authentication, you need to configure secrets.py:

  1. Follow the instructions in example_secrets.txt
  2. Go to the login page: you will see the login buttons for the providers you've set up. Clicking on those buttons creates an account &/or logs you into enkiWS using OAuth.

Notes:

  • Valve's Steam is always available since it doesn't require a client Id nor secret.
  • When you navigate the enkiWS site you will no longer see the warning message stating that the setup is incomplete.

REST API

Overview

WARNING: The API is in flux until v1.0

The rest api provides a mechanism for developers to create games, apps and websites which interact with users data.

Administration of the Apps and app_secret required for access to the Game API use Google user account login which requires a Google App Engine admin account for the enkiWS GAE install. To access the admin page go to /admin/apps

  • Protocol: HTTPS
  • Request method: POST
  • Request and response format: JSON
  • Request and result parameters format: String unless specified otherwise

The REST API security mechanism is to use HTTPS for as the protocol combined with user authentication (detailed later). Currently we do not implement a client secret or other application verification mechanism since any global key available on a client machine can be stolen - thus the REST API is deliberatly limited in the scope of the changes it can make.

User Authentication via a Connect Code

EnkiWS encourages the use of OAUTH so users may not have a password, and having users type their password into an unknown application is potentially risky. So we've developed an approach which allows users to authenticate an app by getting a temporary short code which they use to login, and the app exchanges this for a long lasting authentication token. Users can remove authentication priviledges using their profile page.

  1. User goes to their profile page and requests a 'Connect Code'
  2. EnkiWS displays the code e.g. 'Q354D'
  3. User types their full displayname and connect code into the app login screen
  4. App uses the /api/v1/connect API to login
  5. App receives an auth_token and user_id, which it can use for further API requests. This can be stored on disk and re-used if required

The datastore

The datastore provides a named JSON object store for users with private, friends, and public read access control. The Google App Engine backend limits the per-object size to around 1 megabyte, however we intend to add per user limits with product based increases (for example you could configure enkiWS to give all registered users a small amount of storage, but users with a given product several megabytes).

Example REST API uses

Once an app has authenticated the user, it can use the auth_token and user_id to perform further API queries, and use the datastore to store user data.

  • Check if a user has purchased a game
    1. Use /api/v1/ownsproducts and request for your game
  • Store and find out if friends are online
    1. Get the list of friends with /api/v1/friends
    2. Use the datastore /api/v1/datastore/set to store a JSON structure containing the details you need for friend status (online, ingame, IP address and ports for chat or game connect etc.). Make sure to have "read_access" : "friends"
    3. Use /api/v1/datastore/getlist with "read_access" : "friends" to get a list of the status for each user_id
  • Invite a friend to play a game
    1. We discover the friend status as above, ensuring that the datastore entry has an IP address and port for messages to be passed. The game can then connect via this address and send an invite
  • Get a list of open servers
    1. Again using the datastore we store the details required (server name, IP address, port, game details) to connect to the game with "read_access" : "public"
    2. Clients can pull this list using /api/v1/datastore/getlist with "read_access" : "public", and ping the servers for online status

API Functionality table

URL Functionality Request Parameters Request example Response Parameters Response example (success)
/api/v1/
connect
User connect displayname,
code,
app_id,
app_secret
{"displayname":"Silvia#2702",
"code":"Q354D",
"app_id":"5141470990303232",
"app_secret":"0ZYWOl..Y9Xq"}
user_id,
auth_token,
success, error
{"user_id":"5066549580791808",
"auth_token":"kDfFg1..dw3S",
"success":true, "error":""}
/api/v1/
logout
User logout user_id,
auth_token,
app_secret
{"user_id":"5066549580791808",
"auth_token":"kDfFg1..dw3S",
"app_secret":"0ZYWOl..Y9Xq"}
success, error {"success":true, "error":""}
/api/v1/
authvalidate
Validate user user_id,
auth_token,
app_secret
{"user_id":"5066549580791808",
"auth_token":"kDfFg1..dw3S",
"app_secret":"0ZYWOl..Y9Xq"}
user_displayname,
success, error
{"user_displayname":"Silvia#2702",
"success":true,"error":""}
/api/v1/
ownsproducts
List products activated by user user_id,
auth_token,
app_secret
{"user_id":"5066549580791808",
"auth_token":"kDfFg1..dw3S",
"app_secret":"0ZYWOl..Y9Xq"}
products_owned (list of strings),
success, error
{"products_owned":["product_a","product_b"],
"success":true,"error":""}
/api/v1/
ownsproducts
List confirming products activated by user user_id,
auth_token,
app_secret,
products (list of strings)
{"user_id":"5066549580791808",
"auth_token":"kDfFg1..dw3S",
"app_secret":"0ZYWOl..Y9Xq",
"products":["product_b","product_c"]}
products_owned (list of strings),
success, error
{"products_owned":["product_b"],
"success":true,"error":""}
/api/v1/
friends
List user's friends user_id,
auth_token,
app_secret
{"user_id":"5066549580791808",
"auth_token":"kDfFg1..dw3S",
"app_secret":"0ZYWOl..Y9Xq"}
friends user_id
and displayname
(list of dictionaries of strings),
success, error
{"friends":[
{"user_id":"4677872220372992",
"displayname":"Toto#2929"},
{"user_id":"6454683010859008",
"displayname":"Ann#1234"}],
"success":true,"error":""}
/api/v1/
datastore/
set
Create / update user's data filtered by app id, data type and data id user_id,
auth_token,
app_secret,
data_type,
data_id,
data_payload (JSON,
inc. optional
calc_ip_addr),
time_expires (int)
read_access
{"user_id":"5066549580791808",
"auth_token":"kDfFg1..dw3S",
"app_secret":"0ZYWOl..Y9Xq",
"data_type":"settings",
"data_id":"s42",
"data_payload":
"{"colour":"green","size":"0.5",
"calc_ip_addr":""}",
"time_expires":3600,
"read_access":"friends"}
success, error {"success":true,"error":""}
/api/v1/
datastore/
get
Get user's data filtered by app id, data type and data id user_id,
auth_token,
app_secret,
data_type,
data_id
{"user_id":"5066549580791808",
"auth_token":"kDfFg1..dw3S",
"app_secret":"0ZYWOl..Y9Xq",
"data_type":"settings",
"data_id":"s42"}
data_payload (JSON),
time_expires (int),
read_access,
server_time (int),
success, error
{"data_payload":[
{"colour":"green","size":"0.5","calc_ip_addr":"127.0.0.1"}],
"time_expires":1458074738000,
"read_access":"friends"
"server_time":1458071138,
"success":true,"error":""}
/api/v1/
datastore/
getlist
Get list of users' data filtered by app id, data type and read access. If read_access is
- "public": return all users public data.
- "friends": return user's friends' data that have read_access set to "friends".
- "private": return the user's private data.
user_id,
auth_token,
app_secret,
data_type,
read_access ("public", "friends", "private")
{"user_id":"5066549580791808",
"auth_token":"kDfFg1..dw3S",
"app_secret":"0ZYWOl..Y9Xq",
"data_type":"settings",
"read_access":"friends"}
data_payloads
(list of dictionaries
(user_id, data_id,
data_payload (JSON),
time_expires (int))),
server_time (int),
success, error
{"data_payloads":[
{"user_id":"4677872220372992","data_id":"s42",
"data_payload":{"colour":"blue","size":"0.8","calc_ip_addr":"127.0.0.4"},
"time_expires":1457777535},
{"user_id":"6454683010859008","data_id":"s15",
"data_payload":{"colour":"red","size":"0.4","calc_ip_addr":"127.0.0.3"},
"time_expires":1458223683}],
{"user_id":"6454683010859008","data_id":"s39",
"data_payload":{"colour":"white","size":"0.9","calc_ip_addr":"127.0.0.3"},
"time_expires":1458329792}],
"server_time":1458071139,
"success":true,"error":""}
/api/v1/
datastore/
del
Delete user's data filtered by app id, data type and data id user_id,
auth_token,
app_secret,
data_type,
data_id
{"user_id":"5066549580791808",
"auth_token":"kDfFg1..dw3S",
"app_secret":"0ZYWOl..Y9Xq",
"data_type":"settings",
"data_id":"s42"}
success, error {"success":true,"error":""}

API Errors

Error messages Description Response example (failure)
Invalid request Invalid or missing request parameters {"success":false,"error":"Invalid request"}
Unauthorised app App not registered or invalid secret.
- Connect request: app_id/app_secret invalid.
- Other requests: app_secret invalid.
{"success":false,"error":"Unauthorised app"}
Unauthorised user User could not be authenticated.
- Connect request: user_displayname/code invalid.
- Other requests: user_id/auth_token invalid.
{"success":false,"error":"Unauthorised user"}
Not Found No data found or data expired {"success":false,"error":"Not found"}

Third Party dependencies

Most of the third party libraries, code and tools used in this project are included in the GitHub repository. The others are installed with Google App Engine and Python 2.7 or linked to.

Demo of enkiWS

Online demo - may be out of sync with the source code

Projects using enkiWS

Our website enkisoftware.com uses enkiWS, with the addition of static pages and a custom blog.

FAQ

Why use Google App Engine?

Small games developers like ourselves typically have very irregular backend requirements - website and service traffic are typically relatively low, but spike when there's a new release or if some content goes viral. Google App Engine (GAE) provides a low cost scalable solution for this scenario. For more information see our article on Implementing a static website in Google App Engine or Wolfire's article on GAE for indie developers as well as Wolfire's article on hosting the Humble Indie Bundle.
Note that if you don't want to use Google App Engine, you can use the open source AppScale environment to run this code on other platforms.

Why Python?

Python is sufficiently popular and easy to use that it made a convenient choice of language from those available on Google App Engine. We considered Google's Go language, but although it has many benefits we thought it would be less widely known in the game development community.

does enkiWS comply with the EU Cookie law?

Cookies used in enkiWS are exempt from consent according to EU legislation.

Credits

Implementation - Juliette Foucaut - @juliettef
Architecture and implementation - Doug Binks - @dougbinks
Testing - Andy Binks
Testing - Sven Bentlage - @sbe-dev
Localisation - Charlotte Foucaut - @charlf

Licence

zlib - see licence.txt