Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: MySQL/MariaDB error with typeclasses tag migration 0007 #1184

Closed
vincent-lg opened this issue Jan 31, 2017 · 16 comments
Closed

Bug: MySQL/MariaDB error with typeclasses tag migration 0007 #1184

vincent-lg opened this issue Jan 31, 2017 · 16 comments

Comments

@vincent-lg
Copy link
Contributor

@vincent-lg vincent-lg commented Jan 31, 2017

Brief summary of issue / Description of requested feature:

Since the new migration, 'evennia migrate' hangs. See traceback below.

  Applying typeclasses.0007_tag_migrations_may_be_slow...Traceback (most recent call last):
  File "c:\users\vincent\pyenv\Scripts\evennia_launcher.py", line 6, in <module>
    exec(compile(open(__file__).read(), __file__, 'exec'))
  File "C:\Users\Vincent\evennia\bin\windows\evennia_launcher.py", line 16, in <module>
    main()
  File "c:\users\vincent\evennia\evennia\server\evennia_launcher.py", line 1360, in main
    django.core.management.call_command(*args, **kwargs)
  File "c:\users\vincent\pyenv\Lib\site-packages\django\core\management\__init__.py", line 119, in call_command
    return command.execute(*args, **defaults)
  File "c:\users\vincent\pyenv\Lib\site-packages\django\core\management\base.py", line 399, in execute
    output = self.handle(*args, **options)
  File "c:\users\vincent\pyenv\Lib\site-packages\django\core\management\commands\migrate.py", line 200, in handle
    executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
  File "c:\users\vincent\pyenv\Lib\site-packages\django\db\migrations\executor.py", line 92, in migrate
    self._migrate_all_forwards(plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "c:\users\vincent\pyenv\Lib\site-packages\django\db\migrations\executor.py", line 121, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "c:\users\vincent\pyenv\Lib\site-packages\django\db\migrations\executor.py", line 198, in apply_migration
    state = migration.apply(state, schema_editor)
  File "c:\users\vincent\pyenv\Lib\site-packages\django\db\migrations\migration.py", line 123, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "c:\users\vincent\pyenv\Lib\site-packages\django\db\migrations\operations\models.py", line 359, in database_forwards
    getattr(new_model._meta, self.option_name, set()),
  File "c:\users\vincent\pyenv\Lib\site-packages\django\db\backends\base\schema.py", line 318, in alter_unique_together
    self._delete_composed_index(model, fields, {'unique': True}, self.sql_delete_unique)
  File "c:\users\vincent\pyenv\Lib\site-packages\django\db\backends\mysql\schema.py", line 87, in _delete_composed_index
    return super(DatabaseSchemaEditor, self)._delete_composed_index(model, fields, *args)
  File "c:\users\vincent\pyenv\Lib\site-packages\django\db\backends\base\schema.py", line 347, in _delete_composed_index
    ", ".join(columns),
ValueError: Found wrong number (0) of constraints for typeclasses_tag(db_key, db_category, db_tagtype)

Steps to reproduce the issue / Reasons for adding feature:

I am using MySQL to host the evennia database. Everything worked until I pulled and installed a new, clean database.

  1. I git pull-ed the Evennia repository.
  2. I drop my old MySQL database after dumping it, and created a new one.
  3. I deleted the auto-generated file in evennia typeclasses/migrations/008...
  4. Back in my MYD directory, I 'evennia makemigrations'
  5. And 'evennia migrate' which gave me the traceback.
@Griatch
Copy link
Member

@Griatch Griatch commented Jan 31, 2017

evennia makemigrations should not be used except on one's own models. The MySQL wiki page was updated. Will have to test with mysql to see if I can reproduce.

@Griatch Griatch added the bug label Jan 31, 2017
@vincent-lg
Copy link
Contributor Author

@vincent-lg vincent-lg commented Jan 31, 2017

@Griatch Griatch changed the title Error with the new migration (tags migration may be slow) Bug: MySQL error with typeclasses tag migration 0007 Feb 4, 2017
@vincent-lg
Copy link
Contributor Author

@vincent-lg vincent-lg commented May 3, 2017

I am not sure this is related, but a similar error is shown on a Stack overflow post. If I'm following well, the constraint that is sought doesn't match the one that was created. Perhaps quick editing in the migration file would allow to fix it. I'll check.

@vincent-lg
Copy link
Contributor Author

@vincent-lg vincent-lg commented May 3, 2017

I still don't get the error. Someone with a better understanding of Django migrations might though. Here's what I've gathered. After migration 6, the table typeclasses_tag looks like this:

Table	Create Table
typeclasses_tag
CREATE TABLE `typeclasses_tag` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `db_key` varchar(255) DEFAULT NULL,
  `db_category` varchar(64) DEFAULT NULL,
  `db_data` longtext,
  `db_model` varchar(32) DEFAULT NULL,
  `db_tagtype` varchar(16) DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `typeclasses_tag_db_key_31dd2997_uniq` (`db_key`,`db_category`,`db_tagtype`),
  KEY `typeclasses_tag_db_key_31dd2997_idx` (`db_key`,`db_category`,`db_tagtype`),
  KEY `typeclasses_tag_f765d495` (`db_key`),
  KEY `typeclasses_tag_65fe7e54` (`db_category`),
  KEY `typeclasses_tag_36df3c33` (`db_model`),
  KEY `typeclasses_tag_cd1c31bb` (`db_tagtype`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8

The constraint that causes the error contains only three fields (db_key, db_category and db_tagtype). It looks like it could be ``typeclasses_tag_db_key_31dd2997_uniqortypeclasses_tag_db_key_31dd2997_idx`. In both cases, it seems the order of fields is correct. On the other hand, I haven't been able to locate in the 007 migration, the line that modifies these constraints (the traceback isn't detailing this issue at all).

I hope this helps in debugging the error. I have a working MySQL server (MariaDB on Windows), don't hesitate if there's something to test.

@Griatch
Copy link
Member

@Griatch Griatch commented May 3, 2017

I just ran this on mysql from a fresh install and didn't get any traceback at migration. I take it you are testing it with a pre-populated database?

@vincent-lg
Copy link
Contributor Author

@vincent-lg vincent-lg commented May 3, 2017

No, it was a newly-created database as described on the wiki. I followed the instructions step by step. But I'm running MariaDB on Windows, it might make a difference.

@Griatch
Copy link
Member

@Griatch Griatch commented May 3, 2017

As far as I can tell at this time, this appears to be a MariaDB error, not a MySQL error. I get errors (not this one though) with MariaDB while it works with MySQL. Maybe someone else can try.

@depwl9992
Copy link

@depwl9992 depwl9992 commented May 13, 2017

Received the same errors when trying to migrate, as @Griatch put it, a pre-populated database into MySQL (God and tutorial_world were both created, which looks to be a decent amount of data).

  1. Deleted everything in the top-level directory since it's a new world, and I have nothing to lose...
  2. Restarted from the beginning of Getting Started
  3. Pulled commit 6db118a
  4. Dropped all tables in the MySQL db.
  5. Before the initial 'evennia migrate', I edited settings.py to MySQL, matching the structure of settings-default.py with db values baked in rather than loaded from the separate ini-format file that MySQL Setup recommends.

    Probably better mentioned in a different issue, but a note to pip install MySQL-python would be helpful in the MySQL Setup wiki page...

At this point, I ran the initial migrate and everything completed without incident. Server initialized and started correctly and the God character has been properly created. I can log in and install tutorial_world, and start playing around with it. Also can confirm that data is being generated correctly in MySQL.

@Griatch
Copy link
Member

@Griatch Griatch commented Jun 4, 2017

@depwl9992 Good to hear it works for you, this appears to be a MariaDB error rather than a MySQL error. Oh, and I added a note about the need for a Python bridge to the mysql doc thanks. :)

@Griatch Griatch changed the title Bug: MySQL error with typeclasses tag migration 0007 Bug: MySQL (MariaDB) error with typeclasses tag migration 0007 Jun 4, 2017
@Griatch Griatch changed the title Bug: MySQL (MariaDB) error with typeclasses tag migration 0007 Bug: MariaDB error with typeclasses tag migration 0007 Jun 4, 2017
@depwl9992
Copy link

@depwl9992 depwl9992 commented Jun 4, 2017

@Griatch Alright. I just figured I'd throw in my 2 cents, since I was getting the same behavior with MySQL as well. But what you're saying is the symptoms were maybe caused by a slightly different thing then? Actually, come to think of it, I'm not even sure what Maria is. Does this run in tandem with MySQL, or as an alternative?

@Griatch Griatch changed the title Bug: MariaDB error with typeclasses tag migration 0007 Bug: MySQL/MariaDB error with typeclasses tag migration 0007 Jul 13, 2017
@Tirit
Copy link

@Tirit Tirit commented Jul 14, 2017

I've been getting this similar error; I've been getting it when I start with an existing sqllite db migrating to a new MySQL db, and when I've just started the first evennia migrate into a new MySQL db.

Did a whole new clone of Evennia with the latest commit, did evennia --init , setup it up for sqllite and did a migrate. Started the game with just the Superuser and limbo. Setup up settings.py for mysql, and did a migrate in my folder only. Gave me the traceback.

Did a whole new clone of Evennia with the latest commit, did evennia --init , setup it up for mysql and did a migrate into a clean db. Gave me the traceback.

Applying typeclasses.0007_tag_migrations_may_be_slow...Traceback (most recent call last):
 File "/home/evennia/pyenv/bin/evennia", line 6, in <module>
    exec(compile(open(__file__).read(), __file__, 'exec'))
  File "/home/evennia/evennia/bin/unix/evennia", line 10, in <module>
    main()
  File "/home/evennia/evennia/evennia/server/evennia_launcher.py", line 1333, in main
    django.core.management.call_command(*args, **kwargs)
  File "/home/evennia/pyenv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 119, in call_command
    return command.execute(*args, **defaults)
  File "/home/evennia/pyenv/local/lib/python2.7/site-packages/django/core/management/base.py", line 399, in execute
    output = self.handle(*args, **options)
  File "/home/evennia/pyenv/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 200, in handle
    executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
  File "/home/evennia/pyenv/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 92, in migrate
    self._migrate_all_forwards(plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/home/evennia/pyenv/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 121, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/home/evennia/pyenv/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 198, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/home/evennia/pyenv/local/lib/python2.7/site-packages/django/db/migrations/migration.py", line 123, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/home/evennia/pyenv/local/lib/python2.7/site-packages/django/db/migrations/operations/models.py", line 359, in database_forwards
    getattr(new_model._meta, self.option_name, set()),
  File "/home/evennia/pyenv/local/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 318, in alter_unique_together
    self._delete_composed_index(model, fields, {'unique': True}, self.sql_delete_unique)
  File "/home/evennia/pyenv/local/lib/python2.7/site-packages/django/db/backends/mysql/schema.py", line 87, in _delete_composed_index
    return super(DatabaseSchemaEditor, self)._delete_composed_index(model, fields, *args)
  File "/home/evennia/pyenv/local/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 347, in _delete_composed_index
    ", ".join(columns),
ValueError: Found wrong number (0) of constraints for typeclasses_tag(db_key, db_category, db_tagtype) 
 Evennia 0.6.0 (rev f7830a5) (rev f7830a5)
    OS: posix
    Python: 2.7.12
    Twisted: 16.4.0
    Django: 1.9.9
    Evennia MUD/MUX/MU* development system
 select version();
+-------------------------+
| version()               |
+-------------------------+
| 5.7.18-0ubuntu0.16.04.1 |
+-------------------------+
1 row in set (0.00 sec)
@chainsol
Copy link
Member

@chainsol chainsol commented Jul 14, 2017

Just to add something to this, I don't have any problems with MariaDB. Migrations ran perfectly fine.

Server is running Ubuntu Server 16.04
MySQL:
mysql Ver 15.1 Distrib 10.0.29-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2

Evennia:
Evennia 0.6.0 (rev 88574c9) (rev 88574c9)
OS: posix
Python: 2.7.12
Twisted: 17.1.0
Django: 1.9.13

@ghost
Copy link

@ghost ghost commented Jul 15, 2017

SOLVED:

Somewhere on the Evennia wiki (and elsewhere in Django documentation) users are directed to set up their database by using something like:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'OPTIONS': {
            'read_default_file': 'server/conf/db.cnf',
        },
    }
}

If you did this, then you experienced this error. You may have experienced this error for other reasons, but if you did this, then you experienced this error. We know for sure that this affects MySQL and MariaDB, but it probably affects every other database Django interacts with too.

If you use the external db.cnf then when django.db.backends.mysql.introspection tries to do a query, it pulls the database name for that query from self.connection.settings_dict which is totally unpopulated. There's no username, no password, no db address, no database name. So the query always always always returns NOTHING and that's why you experienced this error.

Your solution is to put the following directly into your Evennia configuration settings file. If you want everybody on your git to know your db information (not a big deal if you did a proper setup, with a firewall blocking external access to database ports on top of mysql allowing access to your user only from localhost or only from whatever static IP your Evennia will be connecting from), you'd use settings.py. If you're a little less up to speed on firewalls and database user configurations then you DEFINITELY want to put this into secret_settings.py which Evennia has kindly slipped into .gitignore for you by default so anyone on your game's git team won't be seeing it. If you aren't sure which one to use, use secret_settings.py

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'myDBSchema',
        'USER': 'myDBUserName',
        'PASSWORD': 'myDBUserPassword',
        'HOST': 'localhost',  # Or an IP Address that your DB is hosted on
        'PORT': '3306',
    }
}
@vincent-lg
Copy link
Contributor Author

@vincent-lg vincent-lg commented Jul 16, 2017

@tekmunkey thanks so much. I tested it and it worked fine out of the box. I guess we could close this issue, since the wiki has been updated too.

@vincent-lg vincent-lg closed this Jul 16, 2017
@Griatch
Copy link
Member

@Griatch Griatch commented Jul 17, 2017

@tekmunkey Excellent, thanks for the explanation and solution. That's a weird error.

1 similar comment
@Griatch
Copy link
Member

@Griatch Griatch commented Jul 17, 2017

@tekmunkey Excellent, thanks for the explanation and solution. That's a weird error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.