0144 migration fails with 'column "organization_id" contains null values'

[iurisilvio]

Looking at my sentry_project table, the first user Sentry (Internal) does not have an organization. I update the record with an organization and the migration worked fine.

 id |       name        | public |          date_added           | status |      slug       | team_id |  platform  | organization_id 
----+-------------------+--------+-------------------------------+--------+-----------------+---------+------------+-----------------
  1 | Sentry (Internal) | f      | 2014-07-31 16:48:50.563241-04 |      0 | sentry          |         | django     |                

Running migrations for sentry:
 - Migrating forwards to 0167_auto__add_field_authprovider_flags.
 > sentry:0144_auto__chg_field_project_organization
FATAL ERROR - The following SQL query failed: ALTER TABLE "sentry_project" ALTER COLUMN "organization_id" TYPE integer, ALTER COLUMN "organization_id" SET NOT NULL, ALTER COLUMN "organization_id" DROP DEFAULT;
The error was: column "organization_id" contains null values

Error in migration: sentry:0144_auto__chg_field_project_organization
Traceback (most recent call last):
  File "/opt/sentry/bin/sentry", line 9, in <module>
    load_entry_point('sentry==7.5.0.dev0', 'console_scripts', 'sentry')()
  File "/opt/sentry/local/lib/python2.7/site-packages/sentry-7.5.0.dev0-py2.7.egg/sentry/utils/runner.py", line 410, in main
    initializer=initialize_app,
  File "/opt/sentry/local/lib/python2.7/site-packages/logan/runner.py", line 169, in run_app
    management.execute_from_command_line([runner_name, command] + command_args)
  File "/opt/sentry/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/opt/sentry/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/sentry/local/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/opt/sentry/local/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/opt/sentry/local/lib/python2.7/site-packages/sentry-7.5.0.dev0-py2.7.egg/sentry/management/commands/upgrade.py", line 34, in handle
    verbosity=options['verbosity'],
  File "/opt/sentry/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 159, in call_command
    return klass.execute(*args, **defaults)
  File "/opt/sentry/local/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/opt/sentry/local/lib/python2.7/site-packages/django/core/management/base.py", line 415, in handle
    return self.handle_noargs(**options)
  File "/opt/sentry/local/lib/python2.7/site-packages/south/management/commands/syncdb.py", line 103, in handle_noargs
    management.call_command('migrate', **options)
  File "/opt/sentry/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 159, in call_command
    return klass.execute(*args, **defaults)
  File "/opt/sentry/local/lib/python2.7/site-packages/raven/contrib/django/management/__init__.py", line 37, in new_execute
    return original_func(self, *args, **kwargs)
  File "/opt/sentry/local/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/opt/sentry/local/lib/python2.7/site-packages/south/management/commands/migrate.py", line 111, in handle
    ignore_ghosts = ignore_ghosts,
  File "/opt/sentry/local/lib/python2.7/site-packages/south/migration/__init__.py", line 220, in migrate_app
    success = migrator.migrate_many(target, workplan, database)
  File "/opt/sentry/local/lib/python2.7/site-packages/south/migration/migrators.py", line 256, in migrate_many
    result = migrator.__class__.migrate_many(migrator, target, migrations, database)
  File "/opt/sentry/local/lib/python2.7/site-packages/south/migration/migrators.py", line 331, in migrate_many
    result = self.migrate(migration, database)
  File "/opt/sentry/local/lib/python2.7/site-packages/south/migration/migrators.py", line 133, in migrate
    result = self.run(migration, database)
  File "/opt/sentry/local/lib/python2.7/site-packages/south/migration/migrators.py", line 114, in run
    return self.run_migration(migration, database)
  File "/opt/sentry/local/lib/python2.7/site-packages/south/migration/migrators.py", line 84, in run_migration
    migration_function()
  File "/opt/sentry/local/lib/python2.7/site-packages/south/migration/migrators.py", line 60, in <lambda>
    return (lambda: direction(orm))
  File "/opt/sentry/local/lib/python2.7/site-packages/sentry-7.5.0.dev0-py2.7.egg/sentry/migrations/0144_auto__chg_field_project_organization.py", line 13, in forwards
    db.alter_column('sentry_project', 'organization_id', self.gf('sentry.db.models.fields.FlexibleForeignKey')(to=orm['sentry.Organization']))
  File "/opt/sentry/local/lib/python2.7/site-packages/south/db/generic.py", line 47, in _cache_clear
    return func(self, table, *args, **opts)
  File "/opt/sentry/local/lib/python2.7/site-packages/south/db/generic.py", line 535, in alter_column
    flatten(values),
  File "/opt/sentry/local/lib/python2.7/site-packages/south/db/generic.py", line 282, in execute
    cursor.execute(sql, params)
  File "/opt/sentry/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
    return self.cursor.execute(sql, params)
  File "/opt/sentry/local/lib/python2.7/site-packages/django/db/utils.py", line 99, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/opt/sentry/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: column "organization_id" contains null values

[dcramer]

What version were you upgrading from?

[iurisilvio]

It was 6.4.4. I installed it some time ago and don't really care about the data there.

Before this bug, I hit the #1386 bug and fixed with @meverone solution (#1386 (comment)).

All other migrations were ok.

[dcramer]

I'm wondering if the case that happened here was a project was missing a team (or said team had no organization) thus the query to backfill orgs failed.

I'm considering this low priority and have no intention to fix it at this time, but we'll keep it open as a point of reference.

[iurisilvio]

I created this sentry instance just to test how it works. I probably forgot some important configurations. I did no backup before the upgrade, I don't have the data to help you.

[ryanbagwell]

I have been struggling with an upgrade problem that I believe is related to this. In my case, the migrations erred at migration no. 143 due to non-unique combinations of slugs and organizations in the project table. I fixed it by ensuring that all of my project slugs were unique by manually editing them.

[BYK]

Closing this issue due to staleness. Feel free to comment here if you think we should still work on this but it is very unlikely that we would as this migration doesn't even exist anymore.