Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model factory should load model as late as possible #160

Open
maiksprenger opened this issue Aug 28, 2014 · 6 comments
Open

Model factory should load model as late as possible #160

maiksprenger opened this issue Aug 28, 2014 · 6 comments
Labels

Comments

@maiksprenger
Copy link

Django uses string notation in several places (e.g. ForeignKey) to be able to reference a model during the app startup phase. Factory boy recently added that syntax, but unfortunately due to the way it's implemented, the original purpose of lazy loading the model is defeated.

The model class is loaded in the factory's Meta class __new__ method. That means at import time, not when the factory is first instantiated. The model class is loaded via Django's get_model method. That means using the string syntax is actually worse than directly declaring the model, because at least on Django 1.7, get_model fails loudly if the entire app cache isn't populated yet, while passing the model lets Django deal with the population of the app cache.

This leads to all kinds of app startup problems when e.g. collecting tests for a test suite, because it suddenly requires the app cache to be fully initialised. But while directly referencing the model as a workaround is possible, it can still lead to tricky situations.

One way to fix it might be to only load the model class when the factory itself is initialised.

maiksprenger added a commit to django-oscar/django-oscar that referenced this issue Aug 28, 2014
This reverts commit f243314.

I mistakenly assumed that the string notation would delay the loading of
the model class as it does e.g. when used for ForeignKeys or in the
AUTH_USER_MODEL. But in fact it achieves the opposite, it not just loads
the model, but it requires the entire model registry to be present!

This has been reported in
FactoryBoy/factory_boy#160

Until then, we need to stick to the old syntax.
maiksprenger added a commit to django-oscar/django-oscar that referenced this issue Aug 28, 2014
This reverts commit f243314.

I mistakenly assumed that the string notation would delay the loading of
the model class as it does e.g. when used for ForeignKeys or in the
AUTH_USER_MODEL. But in fact it achieves the opposite, it not just loads
the model, but it requires the entire model registry to be present!

This has been reported in
FactoryBoy/factory_boy#160

Until then, we need to stick to the old syntax.
@rbarrois rbarrois added the Bug label Sep 3, 2014
@rbarrois
Copy link
Member

rbarrois commented Sep 3, 2014

Hi,

That's indeed a nasty bug; I'll look into it soon.

maiksprenger added a commit to django-oscar/django-oscar that referenced this issue Sep 26, 2014
This reverts commit f243314.

I mistakenly assumed that the string notation would delay the loading of
the model class as it does e.g. when used for ForeignKeys or in the
AUTH_USER_MODEL. But in fact it achieves the opposite, it not just loads
the model, but it requires the entire model registry to be present!

This has been reported in
FactoryBoy/factory_boy#160

Until then, we need to stick to the old syntax.
@maiksprenger
Copy link
Author

@rbarrois, did you get a chance to look at this? Do you think it's easy enough to fix by an outsider like me? Otherwise I can have a stab at a pull request.

@rbarrois
Copy link
Member

Well, it should be much easier now that I have removed the automatic sequence setup from __new__.

If you can prepare a PR (ideally with a test), I should have the time to merge it this week.

Thanks!

@gaffney
Copy link

gaffney commented Jan 9, 2015

Hello @maikhoepfel

Any update on this?

@maiksprenger
Copy link
Author

Gaffney, I'm afraid I'm pretty swamped at the moment; my OSS contributions are taking a bit of a hit at the moment.
I did investigate for a bit though. We had a workaround for this issue, and I could remove it without triggering any problems. That made me dig into the code, as @rbarrois mentioned having done some changes. But as far as I can see, the problem is still there. FactoryMetaClass.__new__ still calls contribute_to_class, which triggers a call to get_model. But I have to admit I don't understand what exactly @rbarrois changed. Is this still a problem for you, @gaffney?

Writing a test against model loading is rather fiddly, because by the time the test suite runs, Django will have already loaded all the models...

@rbarrois
Copy link
Member

Indeed, I can confirm that the bug is still here.
I see a few options here:

  • Make all factory loading lazy — which brings in the same issues that made Django switch to explicit loading
  • Find a way to hook factory.django.DjangoModelFactory into Django's app loading system
  • Ensure that test discovery systems call django.setup() before attempting to load models

asfaltboy pushed a commit to asfaltboy/factory_boy that referenced this issue Oct 30, 2015
This is the simplest way I found to fix FactoryBoy#160 in cases where
a test runner does not explicitly call django.setup priot to
test discovery.

This is also [the documented method](https://docs.djangoproject.com/en/1.8/ref/applications/#django.setup) for python scripts that hook into django apps.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants