[1.7.x] Improve migrations/schema docs

docs/ref/django-admin.txt

@@ -1106,6 +1106,25 @@ of sync with its automatically incremented field data.
 The :djadminopt:`--database` option can be used to specify the database for
 which to print the SQL.
 
+squashmigrations <app_label> <migration_name>
+---------------------------------------------
+
+.. django-admin:: squashmigrations
+
+Squashes the migrations for ``app_label`` up to and including ``migration_name``
+down into fewer migrations, if possible. The resulting squashed migrations
+can live alongside the unsquashed ones safely. For more information,
+please read :ref:`migration-squashing`.
+
+.. django-admin-option:: --no-optimize
+
+By default, Django will try to optimize the operations in your migrations
+to reduce the size of the resulting file. Pass ``--no-optimize`` if this
+process is failing for you or creating incorrect migrations, though please
+also file a Django bug report about the behaviour, as optimization is meant
+to be safe.
+
+
 startapp <app_label> [destination]
 ----------------------------------
 

docs/ref/migration-operations.txt

@@ -211,7 +211,46 @@ match the operation's place in the project history, and the second is an
 instance of SchemaEditor.
 
 You are advised to write the code as a separate function above the ``Migration``
-class in the migration file, and just pass it to ``RunPython``.
+class in the migration file, and just pass it to ``RunPython``. Here's an
+example of using RunPython to create some initial objects on a Country model::
+
+    # encoding: utf8
+    from django.db import models, migrations
+
+    def forwards_func(apps, schema_editor):
+        # We get the model from the versioned app registry;
+        # if we directly import it, it'll be the wrong version
+        Country = apps.get_model("myapp", "Country")
+        Country.objects.create(name="USA", code="us")
+        Country.objects.create(name="France", code="fr")
+
+    class Migration(migrations.Migration):
+
+        dependencies = []
+
+        operations = [
+            migrations.RunPython(
+                forwards_func,
+            ),
+        ]
+
+This is generally the operation you would use to create
+:ref:`data migrations <data-migrations>`, run
+custom data updates and alterations, and anything else you need access to an
+ORM and/or python code for.
+
+If you're upgrading from South, this is basically the South pattern as an
+operation - one or two methods for forwards and backwards, with an ORM and
+schema operations available. You should be able to translate the ``orm.Model``
+or ``orm["appname", "Model"]`` references from South directly into
+``apps.get_model("appname", "Model")`` references here and leave most of the
+rest of the code unchanged for data migrations.
+
+Much like ``RunSQL``, ensure that if you change schema inside here you're
+either doing it outside the scope of the Django model system (e.g. triggers)
+or that you use ``SeparateDatabaseAndState`` to add in operations that will
+reflect your changes to the model state - otherwise, the versioned ORM and
+the autodetector will stop working correctly.
 
 
 SeparateDatabaseAndState

docs/ref/schema-editor.txt

@@ -26,6 +26,15 @@ the order you wish changes to be applied. Some possible operations or types
 of change are not possible on all databases - for example, MyISAM does not
 support foreign key constraints.
 
+If you are writing or maintaining a third-party database backend for Django,
+you will need to provide a SchemaEditor implementation in order to work with
+1.7's migration functionality - however, as long as your database is relatively
+standard in its use of SQL and relational design, you should be able to
+subclass one of the built-in Django SchemaEditor classes and just tweak the
+syntax a little. Also note that there are a few new database features that
+migrations will look for: ``can_rollback_ddl`` and
+``supports_combined_alters`` are the most important.
+
 Methods
 =======
 
@@ -47,6 +56,9 @@ create_model
 
     create_model(model)
 
+Creates a new table in the database for the provided model, along with any
+unique constraints or indexes it requires.
+
 
 delete_model
 ------------
@@ -55,6 +67,9 @@ delete_model
 
     delete_model(model)
 
+Drops the model's table in the database along with any unique constraints
+or indexes it has.
+
 
 alter_unique_together
 ---------------------
@@ -63,6 +78,9 @@ alter_unique_together
 
     alter_unique_together(model, old_unique_together, new_unique_together)
 
+Changes a model's unique_together value; this will add or remove unique
+constraints from the model's table until they match the new value.
+
 
 alter_index_together
 --------------------
@@ -71,6 +89,9 @@ alter_index_together
 
     alter_index_together(model, old_index_together, new_index_together)
 
+Changes a model's index_together value; this will add or remove indexes
+from the model's table until they match the new value.
+
 
 alter_db_table
 --------------
@@ -79,6 +100,8 @@ alter_db_table
 
     alter_db_table(model, old_db_table, new_db_table)
 
+Renames the model's table from ``old_db_table`` to ``new_db_table``.
+
 
 alter_db_tablespace
 -------------------
@@ -87,6 +110,8 @@ alter_db_tablespace
 
     alter_db_tablespace(model, old_db_tablespace, new_db_tablespace)
 
+Moves the model's table from one tablespace to another.
+
 
 add_field
 ---------
@@ -95,6 +120,17 @@ add_field
 
     add_field(model, field)
 
+Adds a column (or sometimes multiple) to the model's table to represent the
+field. This will also add indexes or a unique constraint
+if the field has ``db_index=True`` or ``unique=True``.
+
+If the field is a ManyToManyField without a value for ``through``, instead of
+creating a column, it will make a table to represent the relationship. If
+``through`` is provided, it is a no-op.
+
+If the field is a ``ForeignKey``, this will also add the foreign key
+constraint to the column.
+
 
 remove_field
 ------------
@@ -103,10 +139,35 @@ remove_field
 
     remove_field(model, field)
 
+Removes the column(s) representing the field from the model's table, along
+with any unique constraints, foreign key constraints, or indexes caused by
+that field.
+
+If the field is a ManyToManyField without a value for ``through``, it will
+remove the table created to track the relationship. If
+``through`` is provided, it is a no-op.
+
 
 alter_field
 ------------
 
 ::
 
     alter_field(model, old_field, new_field, strict=False)
+
+This transforms the field on the model from the old field to the new one. This
+includes changing the name of the column (the ``db_column`` attribute),
+changing the type of the field (if the field class changes), changing
+the ``NULL`` status of the field, adding or removing field-only unique
+constraints and indexes, changing primary key, and changing the destination
+of ForeignKey constraints.
+
+The most common transformation this cannot do is transforming a
+ManyToManyField into a normal Field or vice-versa; Django cannot do this
+without losing data, and so it will refuse to do it. Instead, ``remove_field``
+and ``add_field`` should be called separately.
+
+If the database has the ``supports_combined_alters``, Django will try and
+do as many of these in a single database call as possible; otherwise, it will
+issue a separate ALTER statement for each change, but will not issue ALTERs
+where no change is required (as South often did).

docs/topics/migrations.txt

@@ -392,6 +392,69 @@ If you're interested in the more advanced migration operations, or want
 to be able to write your own, see our
 :doc:`migration operations reference </ref/migration-operations>`.
 
+.. _migration-squashing:
+
+Squashing migrations
+--------------------
+
+You are encouraged to make migrations freely and not worry about how many you
+have; the migration code is optimised to deal with hundreds at a time without
+much slowdown. However, eventually you will want to move back from having
+several hundred migrations to just a few, and that's where squashing comes in.
+
+Squashing is the act of reducing an existing set of many migrations down to
+one (or sometimes a few) migrations which still represent the same changes.
+
+Django does this by taking all of your existing migrations, extracting their
+Operations and putting them all in sequence, and then running an optimizer
+over them to try and reduce the length of the list - for example, it knows
+that ``CreateModel`` and ``DeleteModel`` cancel each other out, and it knows
+that ``AddColumn`` can be rolled into ``CreateModel``.
+
+Once the operation sequence has been reduced as much as possible - the amount
+possible depends on how closely intertwined your models are and if you have
+any RunSQL or RunPython operations (which can't be optimized through) - Django
+will them write it back out into a new set of initial migration files.
+
+These files are marked to say they replace the previously-squashed migrations,
+so they can coexist with the old migration files, and Django will intelligently
+switch between them depending where you are in the history. If you're still
+part-way through the set of migrations that you squashed, it will keep using
+them until it hits the end and then switch to the squashed history, while new
+installs will just use the new squashed migration and skip all the old ones.
+
+This enables you to squash and not mess up systems currently in production
+that aren't fully up-to-date yet. The recommended process is to squash, keeping
+the old files, commit and release, wait until all systems are upgraded with
+the new release (or if you're a third-party project, just ensure your users
+upgrade releases in order without skipping any), and then remove the old files,
+commit and do a second release.
+
+The command that backs all this is :djadmin:`squashmigrations` - just pass
+it the app label and migration name you want to squash up to, and it'll get to
+work::
+
+  $ ./manage.py squashmigrations myapp 0004
+  Will squash the following migrations:
+   - 0001_initial
+   - 0002_some_change
+   - 0003_another_change
+   - 0004_undo_something
+  Do you wish to proceed? [yN] y
+  Optimizing...
+    Optimized from 12 operations to 7 operations.
+  Created new squashed migration /home/andrew/Programs/DjangoTest/test/migrations/0001_squashed_0004_undo_somthing.py
+    You should commit this migration but leave the old ones in place;
+    the new migration will be used for new installs. Once you are sure
+    all instances of the codebase have applied the migrations you squashed,
+    you can delete them.
+
+Note that model interdependencies in Django can get very complex, and squashing
+may occasionally result in an optimized migration that doesn't work or is
+impossible to run. When this occurs, you can re-try with ``--no-optimize``, but
+please file a bug report either way detailing the models and their
+relationships so we can improve the optimizer to handle your case.
+
 
 .. _migration-serializing: