Permalink
Browse files

Merge branch 'release/r0.1.1'

Conflicts:
	TODO
	skeleton/templates/layout/base.html

Signed-off-by: Sean Chittenden <sean@chittenden.org>
  • Loading branch information...
sean- committed Jun 10, 2011
2 parents d6e2b69 + c11aa90 commit b8667301e5b15c9d546c1bd55127ed7e5b3c80be
Showing with 1,706 additions and 727 deletions.
  1. +8 −0 .enter.local.tcsh.dist
  2. +3 −2 .enter.tcsh → .enter.tcsh.dist
  3. +3 −0 .gitignore
  4. +80 −40 INSTALL
  5. +62 −31 TODO
  6. +1 −1 default_settings.py
  7. +4 −3 doc/COOKIES.txt
  8. +6 −1 requirements.txt
  9. +30 −24 skeleton/__init__.py
  10. +1 −0 skeleton/models/__init__.py
  11. +8 −0 skeleton/models/timezone.py
  12. +10 −0 skeleton/modules/aaa/__init__.py
  13. +24 −3 skeleton/modules/aaa/forms.py
  14. +4 −3 skeleton/modules/aaa/models/__init__.py
  15. +11 −5 skeleton/modules/aaa/models/user.py
  16. +10 −0 skeleton/modules/aaa/models/user_info.py
  17. +22 −0 skeleton/modules/aaa/templates/profile.html
  18. +1 −0 skeleton/modules/aaa/templates/register.html
  19. +36 −0 skeleton/modules/aaa/user.py
  20. +63 −11 skeleton/modules/aaa/views.py
  21. +2 −2 skeleton/modules/home/models/__init__.py
  22. +1 −0 skeleton/modules/home/views.py
  23. +1 −1 skeleton/modules/mod1/models/__init__.py
  24. +3 −2 skeleton/modules/mod1/views.py
  25. +2 −1 skeleton/modules/mod2/views.py
  26. +1 −0 skeleton/modules/mod3/forms.py
  27. +3 −3 skeleton/modules/mod3/models/__init__.py
  28. +2 −1 skeleton/modules/mod3/models/page.py
  29. +2 −0 skeleton/modules/mod3/templates/root.html
  30. +19 −6 skeleton/modules/mod3/views.py
  31. BIN skeleton/static/flask-powered.png
  32. BIN skeleton/static/powered-by-flask-s.png
  33. +1 −1 skeleton/templates/_formhelpers.html
  34. +3 −2 skeleton/templates/layout/base.html
  35. +0 −1 sql/.gitignore
  36. +0 −14 sql/create_roles.sql
  37. +0 −7 sql/fixup_func_owner.sql
  38. +0 −279 sql/funcs.sql.in
  39. +1 −0 sql/initialize/.gitignore
  40. +27 −0 sql/initialize/100_create_roles.sql
  41. +3 −0 sql/initialize/130_create_database.sql
  42. +2 −0 sql/initialize/140_alter_public_schema.sql
  43. +7 −0 sql/initialize/200_schema.sql
  44. +50 −0 sql/initialize/205_integrated_definitions.sql
  45. +122 −0 sql/initialize/210_tables.sql
  46. +47 −0 sql/initialize/280_views.sql
  47. +430 −0 sql/initialize/300_funcs.sql.in
  48. +9 −0 sql/initialize/400_triggers.sql
  49. +9 −0 sql/initialize/500_fixup_func_owner.sql
  50. +38 −0 sql/initialize/600_create_indexes.sql
  51. +22 −0 sql/initialize/700_foreign_keys.sql
  52. +2 −2 sql/{initial_data.sql → initialize/800_initial_data.sql}
  53. +395 −0 sql/initialize/801_timezone_data.sql
  54. +6 −0 sql/initialize/900_cleanup_users.sql
  55. +109 −0 sql/maintenance/100_perms.sql
  56. +0 −93 sql/perms.sql
  57. +0 −183 sql/schema.sql
  58. +0 −5 sql/triggers.sql
View
@@ -0,0 +1,8 @@
+if ( -d /opt/local/lib/postgresql91/bin ) then
+ setenv PATH /opt/local/lib/postgresql91/bin:${PATH}
+endif
+
+setenv PGDATABASE skeleton
+setenv PGUSER skeleton_${USER}
+setenv PGUSER_RW skeleton_rw_${USER}
+setenv PGUSER_ROOT pgsql
@@ -1,9 +1,10 @@
-# To activate this file: chmod 600 .enter.tcsh
-# To deactivate this file: chmod 640 .enter.tcsh
# To turn off the virtualenv, type: deactivate
# Don't do anything if a virtual environment is already loaded
if ( ${?VIRTUAL_ENV} == "1" ) exit 0
+# Load a local .tcsh file
+if (-o .enter.local.tcsh && -P22: .enter.local.tcsh == "0") source .enter.local.tcsh
+
printf 'Activating the "%s" virtual environment.\n' `basename $PWD`
source bin/activate.csh >& /dev/null
View
@@ -6,6 +6,9 @@ local_settings.py
bin/
include/
lib/
+src/
+.enter.tcsh
+.enter.local.tcsh
# Don't loose ssl.key and ssl.cert... don't add them either.
#ssl.cert
View
120 INSTALL
@@ -14,6 +14,9 @@
$ less ~/.cshrc # _Always_ review changes to your shell config
$ exec /bin/tcsh # Start using tcsh(1)
% chsh /bin/tcsh # Change your default shell once you feel comfortable
+ % mv .enter.tcsh.dist .enter.tcsh; mv .enter.local.tcsh.dist .enter.local.tcsh
+ % ${EDITOR} .enter.tcsh .enter.local.tcsh
+ % cd $PWD # Run this after you finish installing PostgreSQL
0b) Download and install python 2.7.1 (NOTE: python 3.X is a no-go with
@@ -30,13 +33,18 @@
0d) Download and install PostgreSQL. Similar to the above, but since this is
- a database and not a random library, you're on your own for not being a
- nub:
+ a database and not a random library, you're on your own for not being a
+ nub:
- % sudo port install postgresql91-server
+ % sudo port install postgresql91-server
+ % sudo mkdir -p /opt/local/var/db/postgresql91/defaultdb
+ % sudo chown postgres:postgres /opt/local/var/db/postgresql91/defaultdb
- # Not required, but man pages are handy!
- % sudo port install postgresql91-doc
+ # Run this in a different Terminal window
+ % sudo su postgres -c '/opt/local/lib/postgresql91/bin/initdb -D /opt/local/var/db/postgresql91/defaultdb'
+
+ # Not required, but man pages are handy!
+ % sudo port install postgresql91-doc
1) Rename from 'skeleton' to 'myapp':
@@ -52,9 +60,12 @@
2) Start the database:
+ # This should be set by your .enter.local.tcsh file.
+ #setenv PGUSER_ROOT pgsql
+
# MacPorts installed version of PostgreSQL 9.1, I run this in a different
# terminal when developing so I can see what's going on.
- sudo su - pgsql -c '/opt/local/lib/postgresql91/bin/postgres -D /opt/local/var/db/postgresql91/defaultdb'
+ sudo su - ${PGUSER_ROOT} -c '/opt/local/lib/postgresql91/bin/postgres -D /opt/local/var/db/postgresql91/defaultdb'
3) Create a virualenv for the skeleton:
@@ -73,21 +84,18 @@
source bin/activate.csh
-5) Pull in the required packages. This step is a bit goofy due to PostgreSQL's
- pg_config being hidden away out of the normal path.
-
- # Tweak your path to where pg_config(1) is hidden
- setenv PATH /opt/local/lib/postgresql91/bin/:${PATH}
+5) Pull in the required packages.
- # And tweak the build environment slightly for python-libmemcached and
- # M2Crypto
+ # Tweak the build environment slightly for python-libmemcached and
+ # M2Crypto.
setenv CPPFLAGS '-I/opt/local/include -I/opt/local/include/openssl'
# Break the installation down in to two steps: download and install (if
# something fails, you don't want to start over again from scratch). ~/tmp
# is created by .cshrc.
- pip install -U --download-cache=~/tmp -I -r requirements.txt --no-install
- pip install -U --download-cache=~/tmp -I -r requirements.txt --no-download
+ mkdir ~/tmp/pip
+ pip install -U --download-cache=~/tmp/pip -I -r requirements.txt --no-install
+ pip install -U --download-cache=~/tmp/pip -I -r requirements.txt --no-download
# Make sure everything is there (or newer):
pip freeze | sort > requirements.txt
@@ -99,6 +107,7 @@
# applying the following patch to your install (versions 0.6.2 & 0.7.0):
% patch -d lib/python2.7/site-packages/ -p0 < patches/werkzeug::contrib::cache_452e8402d056.patch-0.6.2
+
6) Setup debugging:
echo 'DEBUG = True' >> local_settings.py
@@ -110,22 +119,40 @@
7) Initialize the database (these steps should be moved in to a script for
easy unit testing):
- # l2dba 101
+ # l2dba 101.
+
+ # Environment variables that should be set by your .enter.local.tcsh:
+ # PGUSER_RW, PGUSER and PGDATABASE.
+ setenv PGUSER_DBA skeleton_root
+ setenv PGUSER_ROOT pgsql
+
+ # Create the base roles
+ psql -d template1 -U ${PGUSER_ROOT} -1f sql/initialize/100_create_roles.sql
- # Create the admin ROLE and web user
- psql template1 pgsql -1f sql/create_roles.sql
+ # Create your user roles (use two, one for read-only and the other for
+ # read-write activities)
+ psql -d template1 -U ${PGUSER_ROOT} -c "CREATE ROLE ${PGUSER} CONNECTION LIMIT 2 LOGIN;"
+ psql -d template1 -U ${PGUSER_ROOT} -c "CREATE ROLE ${PGUSER_RW} CONNECTION LIMIT 1 LOGIN;"
- # Create the database. Don't pull a MySQL and use ISO-8895-1
- psql template1 pgsql -c "CREATE DATABASE skeleton OWNER skeleton_admin ENCODING='UTF8' LC_COLLATE='en_US.UTF-8' LC_CTYPE='en_US.UTF-8'"
+ # Add the ROLEs to their respective DBA ROLE/GROUPs
+ psql -d template1 -U ${PGUSER_ROOT} -c "ALTER GROUP skeleton_dba ADD USER ${PGUSER};"
+ psql -d template1 -U ${PGUSER_ROOT} -c "ALTER GROUP skeleton_root ADD USER ${PGUSER_RW};"
+
+ # Create the rest of the DB objects
+ psql -d template1 -U ${PGUSER_ROOT} -f sql/initialize/130_create_database.sql
+ psql -U ${PGUSER_ROOT} -1f sql/initialize/140_alter_public_schema.sql
# Load the pgcrypto and uuid functions in to skeleton
cat /opt/local/share/postgresql91/extension/pgcrypto--1.0.sql | sed -e 's#MODULE_PATHNAME#pgcrypto#' > ~/tmp/pgcrypto.sql
cat /opt/local/share/postgresql91/extension/uuid-ossp--1.0.sql | sed -e 's#MODULE_PATHNAME#uuid-ossp#' > ~/tmp/uuid-ossp.sql
- psql skeleton pgsql -1f ~/tmp/pgcrypto.sql
- psql skeleton pgsql -1f ~/tmp/uuid-ossp.sql
+ psql -U ${PGUSER_ROOT} -1f ~/tmp/pgcrypto.sql
+ psql -U ${PGUSER_ROOT} -1f ~/tmp/uuid-ossp.sql
# Load the schema
- psql skeleton skeleton_admin -1 -f sql/schema.sql
+ psql -U ${PGUSER_DBA} -1f sql/initialize/200_schema.sql
+ psql -U ${PGUSER_DBA} -1f sql/initialize/205_integrated_definitions.sql
+ psql -U ${PGUSER_DBA} -1f sql/initialize/210_tables.sql
+ psql -U ${PGUSER_DBA} -1f sql/initialize/280_views.sql
# Load the functions. This loads the functions and changes their
# permissions. Change DATABASE_PASSWORD_HASH to a random string of bytes
@@ -134,42 +161,48 @@
# on your database servers and is not present (in any capacity) along side
# with your webserver password keys.
pwgen -acn 32 1 > sql/db_password.hash
- psql -At skeleton skeleton -c 'SELECT uuid_generate_v4();' > sql/db_email.uuid
+ psql -At -U ${PGUSER_DBA} -c 'SELECT uuid_generate_v4();' > sql/db_email.uuid
+ chmod 600 sql/db_password.hash sql/db_email.uuid
sed -e "s#DATABASE_PASSWORD_HASH#`cat sql/db_password.hash`#" \
- -e "s#DATABASE_EMAIL_UUID#`cat sql/db_email.uuid`#" sql/funcs.sql.in > sql/funcs.sql
- psql skeleton skeleton_admin -1f sql/funcs.sql
+ -e "s#DATABASE_EMAIL_UUID#`cat sql/db_email.uuid`#" \
+ sql/initialize/300_funcs.sql.in > sql/initialize/300_funcs.sql
+ psql -U ${PGUSER_DBA} -1f sql/initialize/300_funcs.sql
# CREATE TRIGGER commands are run outside of funcs.sql because there is no
# "CREATE OR REPLACE TRIGGER" syntax.
- psql skeleton skeleton_admin -1f sql/triggers.sql
+ psql -U ${PGUSER_DBA} -1f sql/initialize/400_triggers.sql
- # ALTER FUNCTION ... OWNER TO ... commands must be run by the pgsql user.
- psql skeleton pgsql -1f sql/fixup_func_owner.sql
+ # ALTER FUNCTION ... OWNER TO ... commands must be run by the -U ${PGUSER_ROOT} user.
+ psql -U ${PGUSER_ROOT} -1f sql/initialize/500_fixup_func_owner.sql
# Add aaa and mod1 to the skeleton_www user's default search_path. This
# only needs to be done when you create a new user.
- psql skeleton pgsql -c 'ALTER ROLE skeleton_www SET search_path = aaa, mod1, public'
+ psql -U ${PGUSER_ROOT} -c "ALTER ROLE skeleton_www IN DATABASE ${PGDATABASE} SET search_path = aaa, mod1, public"
# Populate some initial data
- psql skeleton skeleton_admin -1f sql/initial_data.sql
+ psql -U ${PGUSER_DBA} -1f sql/initialize/800_initial_data.sql
+ psql -U ${PGUSER_DBA} -1f sql/initialize/801_timezone_data.sql
# Setup the GRANTS. Be sure to audit the database permissions after the
# fact via the psql(1) commands: \dp, \ddp and \dn+
- psql skeleton skeleton_admin -1 -f sql/perms.sql
+ psql -U ${PGUSER_DBA} -1f sql/maintenance/100_perms.sql
+
+ # Neuter the DBA user and use your own personal DBA account.
+ psql -U ${PGUSER_ROOT} -1f sql/initialize/900_cleanup_users.sql
# Other misc PostgreSQL tips:
#
# 1) Look in to using the contrib'ed auto-explain module:
# http://www.postgresql.org/docs/current/static/auto-explain.html
#
# 2) A few suggested development tunables for postgresql.conf:
- # shared_preload_libraries = 'pgcrypto,uuid-ossp'
- # synchronous_commit = off
- # log_connections = on
- # log_disconnections = on
- # log_duration = on
- # log_statement = 'all'
- # timezone = 'UTC'
+ # shared_preload_libraries = 'pgcrypto,uuid-ossp'
+ # synchronous_commit = off
+ # log_connections = on
+ # log_disconnections = on
+ # log_duration = on
+ # log_statement = 'all'
+ # timezone = 'UTC'
#
# 3) Learn how to use dblink (and get creative with its use in functions
# and views):
@@ -220,7 +253,13 @@
# production, but it's really convenient to test in an SSL environment
# from the beginning.
+ a) Register:
+
+ https://127.0.0.1:5000/register
+ Then update shadow.aaa_email_confirmation_log:
+
+ skeleton=> UPDATE shadow.aaa_email_confirmation_log SET confirmed = TRUE WHERE id = 1;
9) In production, you will need to combine all of your static assets
together and probably have them served by nginx. Something like:
@@ -234,7 +273,8 @@
# Be sure to change your CANONICAL_NAME and CANONICAL_PORT settings once
# you move in to production.
-10) Feel free to re-run the sql/perms.sql script as many times as you'd
+
+10) Feel free to re-run the sql/maintenance/100_perms.sql script as many times as you'd
like. \dp, \ddp and \dn+ are your friends.
View
93 TODO
@@ -1,53 +1,84 @@
-See below for a list of completed items. Things still in progress:
+See below for a list of completed items. Things still in progress (roughly in
+order of priority):
-*) Add per-user timezone support
*) Cache a user-object upon login in memcache
-*) Beef up the @logged_in decorator so that it:
- 1) checks memcache for a matching session
- 2) Logs a user out if their session has expired (or updates it accordingly)
- 3) Populates memcache with a user object upon cache miss
-*) Add authorization decorators (maybe)
+*) Begin using the g request global where possible
+*) Scripting via Flask-Script (both shell setup and cron-like jobs)
+*) Change the User model so that specific attributes call the appropriate SQL
+ to update the parameters in the database. Most of User is populated via a
+ VIEW.
+*) Support gettext()
+*) Add a file that handles all of the neuances of integrating Babel and accounts
*) Unit testing framework
*) Migrate to using setup.py instead of providing a fixed requirements.txt
-*) Scripting via Flask-Script (both shell setup and cron-like jobs)
-*) Localization
*) Services API example (XML, maybe protobuf)
-*) Add support for insecure and secure cookies
-*) Logout = kill session in memcache and the database
*) pgmemcache
+*) Example of sending mail
+*) Remaining session tedium:
+ *) Beef up the @logged_in decorator so that it:
+ a) checks memcache for a matching session
+ b) Logs a user out if their session has expired (or updates it accordingly)
+ c) Populates memcache with a user object upon cache miss
+ *) Loop detection for clients that have cookies disabled
+ *) Add no cookie page
+ *) Add support for insecure and secure cookies
+ *) Reissue cookie id's older than 24hrs
+ *) Cookies can have their freshness reset after 10min of life. A cookie
+ looses its fresh status after 20min of total life if not
+ refreshed. Each cookie has a timestamp that it was issued, a min
+ renewal time and a max freshness life.
+ *) Include the level of strength of the authenticated session (password,
+ old/renewed token, or 2FA auth'ed token)
+ *) Integrate/use Flask-Login where possible?
+ *) Add authorization decorators. Each session id needs to be given an
+ authorization token that gets refreshed every 300sec. User
+ automatically gets redirected from the decorator with the missing or
+ expired token and requests an authorization token that lasts for a
+ given app for 300sec.
+ *) Logout = kill session in memcache
-Things that are demonstrated well enough (alpha sorted list):
+Alpha sorted list of demonstrated components (some better than others):
AAA (Access, Authentication, Authorization):
- *) Login (via pl functions)
- *) Logout
- *) Registration
+ *) Login (via pl functions)
+ *) Logout
+ *) Registration
Application:
- *) Integration with other WSGI Middleware's
- *) Modularized development (filesystem layout)
- *) Session management (secure cookie handling)
- *) Static assets management
+ *) Integration with other WSGI Middleware's
+ *) Modularized development (filesystem layout)
+ *) Basic profile management
+ *) Session management (secure cookie handling)
+ *) Static assets management
Database (PostgreSQL):
- *) Database ROLEs and permissions
- *) ORM Layer and examples
- *) PostgreSQL pl functions
- *) PostgreSQL schema
+ *) Give each "application class" different database users to connect as
+ *) Ordered list of .sql files to execute in order to recreate (and
+ maintain) the database.
+ *) ORM Layer and examples
+ *) PostgreSQL pl functions
+ *) Use schemas as a management tool for setting correct permissions
+ *) Use a "DBA" role for owning objects and a DBA user for per-user
+ connections
+ *) Support two DBA roles per user, a read-only acount that lets a DBA see
+ the entire database (but not make changes), and a read-write account
+ that gives the user write privileges. Think of it like being an "admin"
+ and then having to "sudo to the root UID" to complete any real work.
Caching (memcached):
- *) Objects
- *) Views
- *) memoized functions
+ *) memoized functions
+ *) Objects
+ *) Views
Development:
- *) Application profiling
- *) Debugging toolbar
+ *) Application profiling
+ *) Debugging toolbar
ORM (SQLAlchemy):
- *) Use of PostgreSQL functions with SQLAlchemy
+ *) Declarative table use
+ *) Use of PostgreSQL functions with SQLAlchemy
Templating (Jinja2):
- *) Template filters
- *) Template layout
+ *) Template filters
+ *) Template layout
View
@@ -22,7 +22,7 @@
DB_PASS = ''
DB_PORT = '5432'
DB_SCHEMA = 'skeleton_schema'
-DB_ADMIN = 'skeleton_admin'
+DB_ADMIN = 'skeleton_dba'
DB_USER = 'skeleton_www'
DEBUG = False
DEBUG_TOOLBAR = False
View
@@ -1,8 +1,9 @@
An explanation of every cookie:
b
- Randomly generated Browser ID
+ Randomly generated Browser ID
skeleton_session
- i = Session ID
- li = Logged In
+ dsturl = URL to be redirected to after a page completes handling a reque
+ i = Session ID
+ li = Logged In
Oops, something went wrong.

0 comments on commit b866730

Please sign in to comment.