Django App Fails to Migrate with Global AUTO_INCREMENT Keys

[andy-wm-arthur]

nautobot-plugin-version-control can successfully run Django migrations with Dolt v0.40.26, but fails on v0.40.27 with error ValueError: The database backend does not accept 0 as a value for AutoField

[zachmu]

Thinking auto increment might be a red herring here. This error appears to arise when inserting a key value of 0:

https://groups.google.com/g/django-users/c/bdC1EuvZzq4

So far I've run all the queries in the migration successfully against Dolt locally, so something stranger is happening.

[zachmu]

That is: something with a foreign key constraint on a table with an auto increment key is getting a value of 0 for that ID. It's getting rejected by the ORM before it even hits the server.

Does that ring any bells?

[andy-wm-arthur]

django stack trace of the migration failure:

Traceback (most recent call last):
  File "/usr/local/bin/nautobot-server", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/nautobot/core/cli.py", line 61, in main
    initializer=_configure_settings,  # Called after defaults
  File "/usr/local/lib/python3.7/site-packages/nautobot/core/runner/runner.py", line 266, in run_app
    management.execute_from_command_line([runner_name, command] + command_args)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 330, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 371, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 85, in wrapped
    res = handle_func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 245, in handle
    fake_initial=fake_initial,
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/executor.py", line 117, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/executor.py", line 227, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/migration.py", line 121, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/operations/special.py", line 190, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "/usr/local/lib/python3.7/site-packages/nautobot/extras/management/__init__.py", line 60, in populate_status_choices
    create_custom_statuses(app_config, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/nautobot/extras/management/__init__.py", line 172, in create_custom_statuses
    obj.content_types.add(content_type)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/fields/related_descriptors.py", line 952, in add
    through_defaults=through_defaults,
  File "/usr/local/lib/python3.7/site-packages/django/db/models/fields/related_descriptors.py", line 1156, in _add_items
    ], ignore_conflicts=can_ignore_conflicts)
  File "/usr/local/lib/python3.7/site-packages/cacheops/query.py", line 374, in bulk_create
    objs = self._no_monkey.bulk_create(self, objs, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 507, in bulk_create
    objs_without_pk, fields, batch_size, ignore_conflicts=ignore_conflicts,
  File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 1277, in _batched_insert
    self._insert(item, fields=fields, using=self.db, ignore_conflicts=ignore_conflicts)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 1254, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1396, in execute_sql
    for sql, params in self.as_sql():
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1341, in as_sql
    for obj in self.query.objs
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1341, in <listcomp>
    for obj in self.query.objs
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1340, in <listcomp>
    [self.prepare_value(field, self.pre_save_val(field, obj)) for field in fields]
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1281, in prepare_value
    value = field.get_db_prep_save(value, connection=self.connection)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/fields/related.py", line 971, in get_db_prep_save
    return self.target_field.get_db_prep_save(value, connection=connection)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/fields/__init__.py", line 823, in get_db_prep_save
    return self.get_db_prep_value(value, connection=connection, prepared=False)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/fields/__init__.py", line 2389, in get_db_prep_value
    value = connection.ops.validate_autopk_value(value)
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/mysql/operations.py", line 236, in validate_autopk_value
    raise ValueError('The database backend does not accept 0 as a '
ValueError: The database backend does not accept 0 as a value for AutoField.
~/NTC/nautobot-plugin-version-control % 

[andy-wm-arthur]

the failing migration is populate_default_status_records

This migration stage is responsible for populating the extras_status table which has a many-to-many relationship with django_content_type table via the extras_status_content_types mapping table. Its schema is:

CREATE TABLE `extras_status_content_types` (
  `id` int NOT NULL AUTO_INCREMENT,
  `status_id` char(32) NOT NULL,
  `contenttype_id` int NOT NULL,
  PRIMARY KEY (`id`),
  KEY `extras_status_content_types_contenttype_id_38503e74` (`contenttype_id`),
  KEY `extras_status_content_types_status_id_27c665c7` (`status_id`),
  UNIQUE KEY `status_idcontenttype_id` (`status_id`,`contenttype_id`),
  CONSTRAINT `extras_status_conten_contenttype_id_38503e74_fk_django_co` FOREIGN KEY (`contenttype_id`) REFERENCES `django_content_type` (`id`),
  CONSTRAINT `extras_status_conten_status_id_27c665c7_fk_extras_st` FOREIGN KEY (`status_id`) REFERENCES `extras_status` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_bin

[andy-wm-arthur]

Fixed by #4229