Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Translated models can't be used in datamigration #157

Closed
vstoykov opened this issue Sep 12, 2016 · 18 comments
Closed

Translated models can't be used in datamigration #157

vstoykov opened this issue Sep 12, 2016 · 18 comments
Assignees

Comments

@vstoykov
Copy link
Contributor

Probably similar to #100

When I try to use translated models in data migration I have an error when I try to create new instance and save it. The error is:

File "/path/to/virtualenv/lib/python3.4/site-packages/django/db/models/query.py", line 346, in create
    obj = self.model(**kwargs)
File "/path/to/virtualenv/lib/python3.4/site-packages/parler/models.py", line 236, in __init__
    for field in self._parler_meta.get_all_fields():
AttributeError: 'NoneType' object has no attribute 'get_all_fields'

Also when I try to query translated models with specific methods to parler's TranslatableManager it raises AttributeError.

In the initial migration when model is created in migrations.CreateModel step I see that bases are defined like that:

bases=(parler.models.TranslatableModelMixin, models.Model),
@gatsinski
Copy link

gatsinski commented Sep 17, 2016

I have the same issue, I also can't use it in data migrations and I have the following traceback:

  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/path/to/virtualenv/lib/python3.5/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
    utility.execute()
  File "/path/to/virtualenv/lib/python3.5/site-packages/django/core/management/__init__.py", line 346, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/path/to/virtualenv/lib/python3.5/site-packages/django/core/management/base.py", line 394, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/path/to/virtualenv/lib/python3.5/site-packages/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/path/to/virtualenv/lib/python3.5/site-packages/django/core/management/commands/migrate.py", line 222, in handle
    executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
  File "/path/to/virtualenv/lib/python3.5/site-packages/django/db/migrations/executor.py", line 110, in migrate
    self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
  File "/path/to/virtualenv/lib/python3.5/site-packages/django/db/migrations/executor.py", line 148, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/path/to/virtualenv/lib/python3.5/site-packages/django/db/migrations/migration.py", line 115, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/path/to/virtualenv/lib/python3.5/site-packages/django/db/migrations/operations/special.py", line 183, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "/path/to/my_app/migrations/0003_auto_20160917_1003.py", line 15, in forwards_func
    award_type = TimelineType.objects.create()
  File "/path/to/virtualenv/lib/python3.5/site-packages/django/db/models/manager.py", line 127, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/path/to/virtualenv/lib/python3.5/site-packages/django/db/models/query.py", line 348, in create
    obj.save(force_insert=True, using=self.db)
  File "/path/to/virtualenv/lib/python3.5/site-packages/parler/models.py", line 604, in save
    self.save_translations(*args, **kwargs)
  File "/path/to/virtualenv/lib/python3.5/site-packages/parler/models.py", line 646, in save_translations
    for meta in self._parler_meta:
TypeError: 'NoneType' object is not iterable

@vdboor
Copy link
Contributor

vdboor commented Nov 5, 2016

I'm quite surprised to see bases=(parler.models.TranslatableModelMixin, models.Model),.

This doesn't activate our metaclass, so _parler_meta will not be initialized.
If you use bases=(parler.models.TranslatableModel,), this issue probably disappears.

Which Django version have you used?
If you must, you access your translated fields model directly (the only that inherits TranslatedFieldsModel in the migration).

@vstoykov
Copy link
Contributor Author

vstoykov commented Nov 7, 2016

I'm using Django 1.8

Migrations where created automatically. And this is what I have. When I change it as you said to use
Probably Django incorrectly determine which classes to put in bases or django-parler need to be parler.models.TranslatableModel the result is the same. I have the same error.

The only workaround is to remove bases from migration and operate as normal model without any translatable features (which is not a problem).

@x603
Copy link

x603 commented Nov 11, 2016

Faced the same issue when trying to create default objects during migration and also having the same bases. Im using Django 1.10.

@yakky
Copy link
Member

yakky commented Nov 11, 2016

I'm working a solution which involves refactoring the base class. Can't promise any ETA, though, but I'll keep you updated

@yakky yakky self-assigned this Nov 11, 2016
@mrgix
Copy link
Contributor

mrgix commented Feb 28, 2017

Same problem using 1.10

@aethemba
Copy link

aethemba commented Mar 7, 2017

I run into the same issue, also using Django 1.10. @yakky , any news on that ETA ;)?

@JacoboCarbo
Copy link

JacoboCarbo commented Jul 18, 2017

I experienced the exact same problem.
I managed to make it work by setting model bases inside the RunPython script during runtime like this:

MyModel.__bases__ = (models.Model,)

Here is the full code:

def my_script(apps, schema_editor):
    MyModel = apps.get_model("my_app", "MyModel")
    MyModel.__bases__ = (models.Model,)

    MyModelTranslation = apps.get_model("my_app", "MyModelTranslation")
    ...

class Migration(migrations.Migration):
    dependencies = [
        ('my_app', '0001_initial'),
    ]

    operations = [
        migrations.CreateModel(
            name='MyModel',
            fields=[...],
            options={...},
            bases=(parler.models.TranslatableModelMixin, models.Model),
        ),
        ... ,
        migrations.RunPython(my_script),
    ]

This way you will not need to modify any migration files.

@spnshguy
Copy link

@JacoboCarbo is your model the same as the ModelTranslation?

knbk added a commit to knbk/django-parler that referenced this issue Apr 26, 2018
knbk added a commit to knbk/django-parler that referenced this issue Apr 26, 2018
knbk added a commit to knbk/django-parler that referenced this issue Apr 26, 2018
knbk added a commit to knbk/django-parler that referenced this issue May 6, 2018
…ions.

Adds a TranslationsForeignKey field type that contributes translations
to the target model.
@knbk
Copy link
Contributor

knbk commented May 6, 2018

So someone in #django encountered this issue and I decided to dive in. The core of the issue is that Django does not add abstract base models to the bases in migrations, but rather copies the fields, managers and Meta options to its concrete base classes. This also sidesteps the metaclass that adds the translations to the model.

I've played with a few solutions, and by far the cleanest is to add a new field type subclassing ForeignKey to handle the link between the translatable model and the translations model. Since this is a concrete field (backed by a db column), the field is added to the migrations by Django, allowing it to run contribute_translations() even when the migration models are rendered.

One issue is that migrations do not have any operation to change the bases of a model. To add support for translations in data migrations, the CreateModel operation for the translations model has to be edited manually to add TranslatedFieldsModelMixin to its bases. For new migrations that's not an issue: I've factored out the methods into a mixin not inheriting from Model, which does get picked up by migrations automatically.

knbk added a commit to knbk/django-parler that referenced this issue May 6, 2018
…ions.

Adds a TranslationsForeignKey field type that contributes translations
to the target model.
@vdboor
Copy link
Contributor

vdboor commented May 16, 2018

nice work @knbk! I've seen your PR for your commit in #225 - sadly something is off for Django 1.8 but maybe I'll find some time to fix it.

@vdboor vdboor closed this as completed in 10479b0 Aug 27, 2018
@mkoistinen
Copy link
Contributor

mkoistinen commented Feb 6, 2019

Any chance we can get a release that includes this fix?

@yanickdi
Copy link

yanickdi commented Sep 24, 2019

same issue using
django-parler==2.0
django==2.1.7

used python manage.py makemigrations
when migrating from my old model:

from django.db import models
from parler.models import TranslatableModel, TranslatedFields

from webapp.core.models.base_model import BaseModel


class MyModel(models.Model):
    class Meta:
        verbose_name = 'My Model'
        verbose_name_plural = 'My Model'

    title = models.CharField(max_length=50)

to new Model:

from django.db import models
from parler.models import TranslatableModel, TranslatedFields

from webapp.core.models.base_model import BaseModel


class MyModel(TranslatableModel):
    class Meta:
        verbose_name = 'My Model'
        verbose_name_plural = 'My Model'

    translations = TranslatedFields(
        title=models.CharField("Title", max_length=50)
    )
Traceback (most recent call last):
  File "manage.py", line 32, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 316, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 353, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 83, in wrapped
    res = handle_func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 203, in handle
    fake_initial=fake_initial,
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/executor.py", line 117, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/executor.py", line 244, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/migration.py", line 114, in apply
    operation.state_forwards(self.app_label, project_state)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/operations/fields.py", line 73, in state_forwards
    state.reload_model(app_label, self.model_name_lower, delay=delay)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/state.py", line 158, in reload_model
    self._reload(related_models)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/state.py", line 191, in _reload
    self.apps.render_multiple(states_to_be_rendered)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/state.py", line 306, in render_multiple
    model.render(self)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/state.py", line 574, in render
    return type(self.name, bases, body)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/base.py", line 299, in __new__
    new_class._meta.apps.register_model(new_class._meta.app_label, new_class)
  File "/usr/local/lib/python3.7/site-packages/django/db/migrations/state.py", line 337, in register_model
    self.do_pending_operations(model)
  File "/usr/local/lib/python3.7/site-packages/django/apps/registry.py", line 420, in do_pending_operations
    function(model)
  File "/usr/local/lib/python3.7/site-packages/django/apps/registry.py", line 399, in apply_next_model
    self.lazy_model_operation(next_function, *more_models)
  File "/usr/local/lib/python3.7/site-packages/django/apps/registry.py", line 411, in lazy_model_operation
    apply_next_model(model_class)
  File "/usr/local/lib/python3.7/site-packages/django/apps/registry.py", line 399, in apply_next_model
    self.lazy_model_operation(next_function, *more_models)
  File "/usr/local/lib/python3.7/site-packages/django/apps/registry.py", line 385, in lazy_model_operation
    function()
  File "/usr/local/lib/python3.7/site-packages/django/db/models/fields/related.py", line 316, in resolve_related_class
    field.do_related_class(related, model)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/fields/related.py", line 387, in do_related_class
    self.contribute_to_related_class(other, self.remote_field)
  File "/usr/local/lib/python3.7/site-packages/parler/fields.py", line 54, in contribute_to_related_class
    self.model.contribute_translations(cls)
  File "/usr/local/lib/python3.7/site-packages/parler/models.py", line 939, in contribute_translations
    base = shared_model._parler_meta
AttributeError: type object 'MyModel' has no attribute '_parler_meta'

by manually changing the migration File from:

bases=(parler.models.TranslatedFieldsModelMixin, models.Model),

to

bases=(parler.models.TranslatableModel, models.Model),

migrate is doing it's job.

@pbeyeler
Copy link

I was able to make this work by replacing
base = shared_model._parler_meta in models.py:939
with
base = getattr(shared_model, '_parler_meta', None)
But I'm not sure if this will cause problems in other situations.

@vdboor
Copy link
Contributor

vdboor commented Jan 2, 2020

@pbeyeler
base = getattr(shared_model, '_parler_meta', None)
But I'm not sure if this will cause problems in other situations.

I also wonder about that. This really needs a good test to sort that out, or a migration file that demos the problem. This could become part of the test suite too.

For v2.0.1 I've only changed the AttributeError into a TypeError that describes the problem. Anything that inherits from TranslatableModelMixin should get the _parler_meta attribute after all. So if the shared model doesn't have it, something seems to be inconsistent.

@mbenadda
Copy link

mbenadda commented Jan 2, 2020

@vdboor I was also encountering the _parler_meta issue today.

I could work around it by using the Constructing the translations model manually method from the docs.

And in fact, this seems to me like a more idiomatic way to achieve this ("explicit is better than implicit"). It's also pretty simple and not a large amount of boilerplate either.

Do you think this method might be the default way to achieve this going forward?

@dkgitdev
Copy link

dkgitdev commented Feb 7, 2020

@vdboor, I've encountered the _parler_meta issue today.

Here's my traceback:

Traceback (most recent call last):
  File "/home/dk/PycharmProjects/platform24tv/venv/lib/python3.6/site-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/dk/PycharmProjects/platform24tv/venv/lib/python3.6/site-packages/django/core/management/base.py", line 335, in execute
    output = self.handle(*args, **options)
  File "/home/dk/PycharmProjects/platform24tv/venv/lib/python3.6/site-packages/django/core/management/commands/migrate.py", line 200, in handle
    fake_initial=fake_initial,
  File "/home/dk/PycharmProjects/platform24tv/venv/lib/python3.6/site-packages/django/db/migrations/executor.py", line 117, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/home/dk/PycharmProjects/platform24tv/venv/lib/python3.6/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/home/dk/PycharmProjects/platform24tv/venv/lib/python3.6/site-packages/django/db/migrations/executor.py", line 244, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/home/dk/PycharmProjects/platform24tv/venv/lib/python3.6/site-packages/django/db/migrations/migration.py", line 122, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/home/dk/PycharmProjects/platform24tv/venv/lib/python3.6/site-packages/django/db/migrations/operations/special.py", line 190, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "/home/dk/PycharmProjects/platform24tv/tv/apps/api/migrations/0012_migrate_translatable_fields.py", line 14, in forwards_func
    name=object.name
  File "/home/dk/PycharmProjects/platform24tv/venv/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/dk/PycharmProjects/platform24tv/venv/lib/python3.6/site-packages/parler/managers.py", line 36, in create
    return super(TranslatableQuerySet, self).create(**kwargs)
  File "/home/dk/PycharmProjects/platform24tv/venv/lib/python3.6/site-packages/django/db/models/query.py", line 415, in create
    obj = self.model(**kwargs)
  File "/home/dk/PycharmProjects/platform24tv/venv/lib/python3.6/site-packages/parler/models.py", line 245, in __init__
    for field in self._parler_meta.get_all_fields():
AttributeError: 'NoneType' object has no attribute 'get_all_fields'

I've attached both files:

0011_auto_20200207_1306.txt
0012_migrate_translatable_fields.txt

I want to translate existing models.
I've tried to add new field translated_name, because django forbids same names as original field name, so I've planned to add new translated model, copy date, remove original field and rename field on translated model.

@milkomeda
Copy link

See my comment here for solution and clarificaton. #263 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests