extensible autogen compare

[sqlalchemy-bot]

Migrated issue, originally created by Michael Bayer (@zzzeek)

1. replace the whole "diffs" list with a system that generates the Ops directives directly from autogenerate.render. Instead of diffs.append(('add_index', Index)) it's ops.append(AddIndex.from_index(index)). This removes the need for compose._produce_adddrop_command, compose._produce_modify_command, and compose._produce_command.

2. the production of the list here will now do the "compose" in place; e.g. when we do compare_tables, we first produce an ops.ModifyTableOps, and pass that into sub-operations. This removes the need for compose._group_diffs_by_table..

3. Most ops will now have a reverse() function. CreateIndex reverses to DropIndex and vice versa, UpgradeOps reverses to DowngradeOps(ops=list(reversed([o.reverse() for o in upgrade.ops]))) and vice versa, etc. This removes the need for compose._to_updown_op and related functions.

4. with #3, we now just produce an UpgradeOps from alembic.compare, and the DowngradeOps is just upgrade_ops.reverse(). this removes the need for compose._to_migration_script.

5. the use case of the legacy "diffs" being needed is now met by a new method on each Ops object, to_simple_diff(), which produces that old tuple; the autogenerate.api.compare_metadata function will call upon produce_migrations, then run through the whole UpgradeOps list, flattening it and running to_simple_diff(). This is also how user-defined operations can participate in the old-school "diffs" API. The current test suite that runs compare_metadata can continue to use this function.

6. the compare system needs to break into dispatch hooks, e.g. util.Dispatch, but a new list-version util.ListDispatch. Targets will be specific to UpgradeOps representing schema-wide changes, and ModifyTableOps for table-specific changes, which will invoke in the same place that foreign key / unique constraints and indexes are run, e.g. in between column adds and drops. We can also add AlterColumnOp as a dispatch target, where column-specific migrates can be detected, the current example being MySQL autoincrement. Each hook will have a different callable style that is specific to the type of diff we are doing.

7. the "include object" hook needs to be passed to the dispatchers we are doing in #6 as well in some way that isn't going to break; need to decide if the user-defined comparison function calls upon include object itself or if this is done in some automated way.

8. as always, the internal system needs to use these hooks. so the AlterColumnOp dispatcher itself is consulted within the dispatch for ModifyTableOps, the ModifyTableOps dispatcher itself is consulted within the dispatch for UpgradeOps.

[sqlalchemy-bot]

Michael Bayer (@zzzeek) wrote:

• rework all of autogenerate to build directly on alembic.operations.ops
  objects; the "diffs" is now a legacy system that is exported from
  the ops. A new model of comparison/rendering/ upgrade/downgrade
  composition that is cleaner and much more extensible is introduced.
• autogenerate is now extensible as far as database objects compared
  and rendered into scripts; any new operation directive can also be
  registered into a series of hooks that allow custom database/model
  comparison functions to run as well as to render new operation
  directives into autogenerate scripts.
• write all new docs for the new system
  fixes #306

→ c5be317

[sqlalchemy-bot]

Changes by Michael Bayer (@zzzeek):

• changed status to closed