Skip to content
This repository
Browse code

Adding a doc on upgrading

  • Loading branch information...
commit d9c6371c9cfea7ab5e88d3df6894a9ae8a880bec 1 parent ab1579a
Charles Leifer authored October 05, 2012
3  docs/index.rst
Source Rendered
@@ -15,6 +15,8 @@ peewee
15 15
 For flask integration, including an admin interface and RESTful API, check
16 16
 out `flask-peewee <https://github.com/coleifer/flask-peewee/>`_.
17 17
 
  18
+See notes on :ref:`notes on upgrading and changes from 1.0 <upgrading>`
  19
+
18 20
 Contents:
19 21
 ---------
20 22
 
@@ -24,6 +26,7 @@ Contents:
24 26
 
25 27
    peewee/overview
26 28
    peewee/installation
  29
+   peewee/upgrading
27 30
    peewee/cookbook
28 31
    peewee/example
29 32
    peewee/models
224  docs/peewee/upgrading.rst
Source Rendered
... ...
@@ -0,0 +1,224 @@
  1
+.. _upgrading:
  2
+
  3
+Upgrading peewee
  4
+================
  5
+
  6
+Peewee went from 2319 SLOC to 1666.
  7
+
  8
+Goals for the new API
  9
+---------------------
  10
+
  11
+* consistent: there is one way of doing things
  12
+* expressive: things can be done that I never thought of
  13
+
  14
+
  15
+.. _changes:
  16
+
  17
+Changes from version 1.0
  18
+------------------------
  19
+
  20
+The biggest changes between 1.0 and 2.0 are in the syntax used for
  21
+constructing queries.  The first iteration of peewee I threw up on github
  22
+was about 600 lines.  I was passing around strings and dictionaries and
  23
+as time went on and I added features, those strings turned into tuples and
  24
+objects.  This meant, though, that I needed code to handle all the possible
  25
+ways of expressing something.  Look at the code for `parse_select <https://gist.github.com/a957dbbff0310fd88d5c>`_.
  26
+
  27
+I learned a valuable lesson:  keep data in datastructures until the
  28
+*absolute* last second.
  29
+
  30
+With the benefit of hindsight and experience, I decided to rewrite and unify
  31
+the API a bit.  The result is a tradeoff.  The newer syntax may be a bit more 
  32
+verbose at times, but at least it will be consistent.
  33
+
  34
+Since seeing is believing, I will show some side-by-side comparisons.  Let's
  35
+pretend we're using the models from the cookbook, good ol' user and tweet:
  36
+
  37
+.. code-block:: python
  38
+
  39
+    class User(Model):
  40
+        username = CharField()
  41
+
  42
+    class Tweet(Model):
  43
+        user = ForeignKeyField(User, related_name='tweets')
  44
+        message = TextField()
  45
+        created_date = DateTimeField(default=datetime.datetime.now)
  46
+        is_published = BooleanField(default=True)
  47
+
  48
+
  49
+Get me a list of all tweets by a user named "charlie":
  50
+
  51
+.. code-block:: python
  52
+
  53
+    # 1.0
  54
+    Tweet.select().join(User).where(username='charlie') 
  55
+
  56
+    # 2.0
  57
+    Tweet.select().join(User).where(User.username == 'charlie')
  58
+
  59
+Get me a list of tweets ordered by the authors username, then newest to oldest:
  60
+
  61
+.. code-block:: python
  62
+
  63
+    # 1.0
  64
+    Tweet.select().order_by(('created_date', 'desc')).join(User).order_by('username')
  65
+
  66
+    # 2.0
  67
+    Tweet.select().join(User).order_by(User.username, Tweet.created_date.desc())
  68
+
  69
+Get me a list of tweets created by users named "charlie" or "peewee herman", and
  70
+which were created in the last week.
  71
+
  72
+.. code-block:: python
  73
+
  74
+    last_week = datetime.datetime.now() - datetime.timedelta(days=7)
  75
+
  76
+    # 1.0
  77
+    Tweet.select().where(created_date__gt=last_week).join(User).where(
  78
+        Q(username='charlie') | Q(username='peewee herman')
  79
+    )
  80
+
  81
+    # 2.0
  82
+    Tweet.select().join(User).where(Tweet.created_date > last_week & (
  83
+        (User.username == 'charlie') | (User.username == 'peewee herman')
  84
+    ))
  85
+
  86
+Get me a list of users and when they last tweeted (if ever):
  87
+
  88
+.. code-block:: python
  89
+
  90
+    # 1.0
  91
+    User.select({
  92
+        User: ['*'], 
  93
+        Tweet: [Max('created_date', 'last_date')]
  94
+    }).join(Tweet, 'LEFT OUTER').group_by(User)
  95
+
  96
+    # 2.0
  97
+    User.select(
  98
+        User, fn.Max(Tweet.created_date).alias('last_date')
  99
+    ).join(Tweet, JOIN_LEFT_OUTER).group_by(User)
  100
+
  101
+Lets do an atomic update on a counter model (you'll have to user your imagination):
  102
+
  103
+.. code-block:: python
  104
+
  105
+    # 1.0
  106
+    Counter.update(count=F('count') + 1).where(url=request.url)
  107
+
  108
+    # 2.0
  109
+    Counter.update(count=Counter.count + 1).where(Counter.url == request.url)
  110
+
  111
+Let's find all the users whose username starts with 'a' or 'A':
  112
+
  113
+.. code-block:: python
  114
+
  115
+    # 1.0
  116
+    User.select().where(R('LOWER(SUBSTR(username, 1, 1)) = %s', 'a'))
  117
+
  118
+    # 2.0
  119
+    User.select().where(fn.Lower(fn.Substr(User.username, 1, 1)) == 'a')
  120
+
  121
+
  122
+I hope a couple things jump out at you from these examples.  What I see is
  123
+that the 1.0 API is sometimes a bit less verbose, but it relies on strings in 
  124
+many places (which may be fields, aliases, selections, join types, functions, etc).  In the 
  125
+where clause stuff gets crazy as there are args being combined with bitwise 
  126
+operators ("Q" expressions) and also kwargs being used with django-style "double-underscore" 
  127
+lookups. The crazy thing is, there are so many different ways I could have expressed
  128
+some of the above queries using peewee 1.0 that I had a hard time deciding which 
  129
+to even write.
  130
+
  131
+The 2.0 API is hopefully more consistent.  Selections, groupings, functions, joins
  132
+and orderings all pretty much conform to the same API.  Likewise, where and having
  133
+clauses are handled the same way (in 1.0 the where clause is simply a raw string).
  134
+
  135
+
  136
+Changes in fields and columns
  137
+-----------------------------
  138
+
  139
+Well, for one, columns are gone.  They were a shim that I used to hack in non-integer
  140
+primary keys.  I always thought the field SQL generation was one of the grosser
  141
+parts of the module and even worse was the back-and-forth that happened between the
  142
+field and column classes.  So, columns are gone - its just fields - and they're
  143
+hopefully a bit smaller and saner.  I also cleaned up the primary key business.
  144
+Basically it works like this:
  145
+
  146
+* if you don't specify a primary key, one will be created named "id"
  147
+* if you do specify a primary key and it is a PrimaryKeyField (or subclass),
  148
+  it will be an automatically incrementing integer
  149
+* if you specify a primary key and it is anything else peewee assumes you are
  150
+  in control and will stay out of the way.
  151
+
  152
+The API for specifying a non-auto-incrementing primary key changed:
  153
+
  154
+.. code-block:: python
  155
+
  156
+    # 1.0
  157
+    class OldSchool(Model):
  158
+        uuid = PrimaryKeyField(column_class=VarCharColumn)
  159
+
  160
+    # 2.0
  161
+    class NewSchool(Model):
  162
+        uuid = CharField(primary_key=True)
  163
+
  164
+The kwargs for the Field constructor changed slightly, the biggest probably
  165
+being that ``db_index`` was renamed to ``index``.
  166
+
  167
+
  168
+Changes in database and adapter
  169
+-------------------------------
  170
+
  171
+In peewee 1.0 there were two classes that controlled access to the database --
  172
+the Database subclass and an Adapter.  The adapter's job was to say what features
  173
+a database backend provided, what operations were valid, what column types were
  174
+supported, and how to open a connection.  The database was a bit higher-level and
  175
+its main job was to execute queries and provide metadata about the database, like
  176
+lists of tables, last insert id, etc.
  177
+
  178
+I chose to consolidate these two classes, since inevitably they always went in
  179
+pairs (e.g. SqliteDatabase/SqliteAdapter).  The database class now encapsulates
  180
+all this functionality.
  181
+
  182
+
  183
+How the SQL gets made
  184
+---------------------
  185
+
  186
+The first thing I started with is the QueryCompiler and the data structures it
  187
+uses.  It takes the data structures from peewee and spits out SQL.  It works 
  188
+recursively and knows about a few types of expressions:
  189
+
  190
+* the query tree
  191
+* comparison statements like '==', 'IN', 'LIKE' which comprise the leaves of the tree
  192
+* expressions like addition, substraction, bitwise operations
  193
+* sql functions like ``substr`` and ``lower``
  194
+* aggregate functions like ``count`` and ``max``
  195
+* columns, which may be selected, joined on, grouped by, ordered by, used as parameters
  196
+  for functions and aggregates, etc.
  197
+* python objects to use as query parameters
  198
+
  199
+At the heart of it is the ``Expr`` object, which is for "expression".  It
  200
+can be anything that can validly be translated into part of a SQL query.
  201
+
  202
+Expressions can be nested, giving way to interesting possibilities like
  203
+the following example I love which selects users whose username starts with "a":
  204
+
  205
+.. code-block:: python
  206
+
  207
+    User.select().where(fn.Substr(fn.Lower(User.username, 1, 1)) == 'a')
  208
+
  209
+The "where" clause now contains a tree with one leaf.  The leaf represents the
  210
+nested function expression on the left-hand-side and the scalar value 'a' on the
  211
+right hand side.  Peewee will recursively evaluate the expressions on either side
  212
+of the operation and generate the correct SQL.
  213
+
  214
+Another aspect is that :py:class:`Field` objects are also expressions, which
  215
+makes it possible to write things like:
  216
+
  217
+.. code-block:: python
  218
+
  219
+    Employee.select().where(Employee.salary < (Employee.tenure * 1000) + 40000)
  220
+
  221
+.. note:: I totally went crazy with operator overloading.
  222
+
  223
+If you're interested in looking, the ``QueryCompiler.parse_expr`` method is where
  224
+the bulk of the code lives.

0 notes on commit d9c6371

Please sign in to comment.
Something went wrong with that request. Please try again.