Is it possible to tweak behavior of migration?

[axzxc1236]

I am slowly trying to move an APP from 0.x to 1.x, which includes switching from aerich to native migration tool.

I have encountered a problem that I am not sure how to solve

trying to add a JSON field that defaults to empty list or dict

For example I have this model

class Example(Model):
    name = fields.CharField(max_length=255)
    value = fields.CharField(max_length=255)

I want to add another field to this model

data = fields.JSONField(default=dict)

tortoise makemigrations happily generates a migration file, but when I run it I encountered tortoise.exceptions.OperationalError: Cannot add a NOT NULL column with default value NULL

Even if I have set a default of dict or list, it still tries to create a field that defaults to NULL.

I can mitigate by setting null=True, but that is not what I want because when I migrate with null=True set, data field on existing entries will be NULL (new entries work with behavior I want but not on existing entries), which requires more maintenance than just running migration, with aerich I just tweak raw SQL statement by adding default '{}' in generated statement.

[vsdudakov]

default=dict is a Python-side default — it's applied when the ORM creates objects,
so the generated DDL has no DEFAULT clause at all, and SQLite refuses to ADD a NOT
NULL column without one. Two ways out:

1. Edit the generated migration and add the DB-level default to the ADD COLUMN
   statement (... NOT NULL DEFAULT '{}') — same tweak you were making under aerich.
2. Three-step: add the column with null=True, backfill
   (UPDATE example SET data = '{}' WHERE data IS NULL), then tighten to NOT NULL —
   more ceremony on SQLite since it rebuilds the table for that alteration.

(1) is less maintenance and matches what you already did with aerich.

[axzxc1236]

Edit the generated migration

The new built in migration generate a Migration class like this, so I am not sure how to add a db level default, it is possible to give another hint?

class Migration(migrations.Migration):
    dependencies = [('models', '0001_initial')]

    initial = False

    operations = [
        ops.AddField(
            model_name='Example',
            name='data',
            field=fields.JSONField(default=dict, encoder=functools.partial(dumps, separators=(',', ':')), decoder=loads),
        ),
    ]

[axzxc1236]

Thank you @vsdudakov, I have now read the documentation further and figure out how to do it

There are two more documentation that I needed:

1. tortoise sqlmigrate Print the SQL for a migration without executing it. https://tortoise.github.io/migration.html#sqlmigrate
2. (ops.)RunSQL which lets you customize which sql statement to run https://tortoise.github.io/migration.html#runsql

I tweaked the statement printed by tortoise sqlmigrate and then write backward statement myself (because "--backwards" give a long list of SQL statement that looks like doing things in slower way)

I ended up with

class Migration(migrations.Migration):
    dependencies = [('models', '0001_initial')]

    initial = False

    operations = [
        ops.RunSQL(
            sql="ALTER TABLE 'example' ADD COLUMN 'data' JSON NOT NULL default '{}'",
            reverse_sql="ALTER TABLE 'Example' DROP COLUMN 'data'",
        ),
    ]

which does what I want.

I wish tortoise would add something like tortoise makemigrations --raw to make this easier to do.