New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

REUSE_DB causes it to use the non-test DBs? #76

Open
erikrose opened this Issue May 24, 2012 · 30 comments

Comments

Projects
None yet
@erikrose
Contributor

erikrose commented May 24, 2012

Upon doing REUSE_DB=1 NOSE=1 ./manage.py test members.tests.forms.form_network_filter_test --with-progressive, I had some very scary evidence (show processlist and some traceback messages) that suggested it was using the real DB, not the fake test one. Repro, fix, and cover with tests!

@erikrose

This comment has been minimized.

Contributor

erikrose commented May 24, 2012

This could have something to do with Votizen's funky DB router or DB mirror setup.

@orblivion

This comment has been minimized.

orblivion commented Jun 8, 2012

Does this happen with version 1.1 or just the dev tip?

@orblivion

This comment has been minimized.

orblivion commented Jun 8, 2012

I wonder if it would be worth it for you, or probably Django really, to do a final doublecheck of the database name to make sure it's the test database, before running any operations on it, and bail out if not.

@erikrose

This comment has been minimized.

Contributor

erikrose commented Jun 8, 2012

It's in 1.1, and I am suitably ashamed. I'll make sure it's covered by tests when I fix it.

As for "doing a final doublecheck", it might be sane to see if the DB name starts with "test_", but then that would preclude people from naming their actual DBs "test_whatever" (which would be weird, but hey).

@jMyles

This comment has been minimized.

jMyles commented Jun 11, 2012

Confirmed. I came here looking for a solution to an odd issue - when I was trying to run with REUSE_DB, I was getting DatabaseError - relation not found.

Upon reading this Issue, I took a look at my postgres databases. I found that, indeed, django-nose is creating test_dbname when REUSE Is off and creating all the appropriate tables.

Interestingly, if I change my DB configuration so that the database name is "test_dbname" (ie, the runner should make a new database called "test_test_dbname"), I do indeed get a run on what is ostensibly the "real" database.

Moreover, if I then rename "test_dbname" to "dbname" (in postgres) and change it back in the django settings, I get a run against "dbname" and not "test_dbname."

So I can confirm this issue. Obviously this is most critical.

Are there any other details I can provide that will be helpful?

@jMyles

This comment has been minimized.

jMyles commented Jun 12, 2012

I believe I have discovered the nature of the problem.

When Django-nose is discovering tests, it causes modules containing them to load (importer.py line 86).

If such a module adds a model to a registry (which many popular django packages do - in our case we're using django-helptext, the registry may attempt to introspect the table names using connection.introspection.table_names().

The table_names method causes cursor() to execute. Since we haven't run NoseTestSuiteRunner.setup_databases() yet, the settings_dict still contains the name of the non-test database.

@jMyles

This comment has been minimized.

jMyles commented Jun 12, 2012

I'm wrong. It's worse than that. I have cases which don't call table_names() and still use the regular database.

I'm not exactly sure how to write and run unit tests for django-nose - is there already a test suite that I'm not seeing?

@erikrose

This comment has been minimized.

Contributor

erikrose commented Jun 12, 2012

The tests are…a little weird right now. The shell script runtests.sh runs them. The tests themselves are in the testapp folder.

@jMyles

This comment has been minimized.

jMyles commented Jun 12, 2012

Hmm... Not really understanding how to add a new test. If I add a
TestCase, it doesn't seem to run - it claims that everything passes.
Since it runs from a .sh file, I'm also not run how to debug it.
What's your typical practice?

On Tue, Jun 12, 2012 at 5:07 PM, Erik Rose
reply@reply.github.com
wrote:

The tests are…a little weird right now. The shell script runtests.sh runs them. The tests themselves are in the testapp folder.


Reply to this email directly or view it on GitHub:
#76 (comment)

Justin Holmes
Intigator, slashRoot
Keynote Program Chair, DjangoCon 2012

slashRoot: Coffee House and Tech Dojo
60 Main Street
New Paltz, NY 12561
845.633.8330

@erikrose

This comment has been minimized.

Contributor

erikrose commented Jun 12, 2012

I'm afraid the current tests are in a really bad state—to the point where I haven't been writing any, which is how we got this bug in the first place. In the interrim, I recommend adding a TestCase to testapp/tests/test_database.py. As long as you name the file like that, it should run.

runtests.sh just runs django-nose over the testapp app several times, with various settings. Any errors should spit out just like under normal Django tests.

@monkeypants

This comment has been minimized.

monkeypants commented Jun 26, 2012

I'm new to django_nose (and nose) today, and immediately ran into a problem with my custom db routing when django_nose is running with REUSE_DB. I think it's probably related to this (issue #76).

In summary, try this router:

class Router(object):
    """dummy django.db.router demonstrating issue with django_nose """
    def db_for_read(self, model, **hints):
        print ''
        print 'router.db_for_read'
        print 'DEBUG: ', model
        print 'DEBUG: ', model._meta
        print 'DEBUG: ', model._meta.app_label
        print 'DEBUG: ', model.__module__

        return 'default' # or whatever makes sense in your test

    def db_for_write(self, model, **hints):
        print ''
        print 'router.db_for_write'
        print 'DEBUG: ', model
        print 'DEBUG: ', model._meta
        print 'DEBUG: ', model._meta.app_label
        print 'DEBUG: ', model.__module__

        return 'default' # or whatever..

In my environment, I get the debug messages as expected when REUSE_DB=0, but not when REUSE_DB=1.

I installed django_nose because I need REUSE_DB...

@jMyles

This comment has been minimized.

jMyles commented Jun 27, 2012

I have fallen off this issue. I'm just so busy with other stuff, as it appears Erik is. I'm still watching, though.

@erikrose

This comment has been minimized.

Contributor

erikrose commented Aug 6, 2012

Just sat down to attack this again, and now I can't repro it anymore. :-( The project I'm testing against has upgraded to 1.4 in the meantime. I wonder if that changed something. Btw, #87 and #88 are related to this.

@jMyles

This comment has been minimized.

jMyles commented Aug 6, 2012

Erik,

I love django-nose and don't want my only contribution to be a few
posts on a github issue.

However, I'm not sure how I can help. I'd like to write a test that
demonstrates the behavior, but I'm pretty removed from it now as well,
and I do not understand the test suite methodology.

Perhaps a good step will be to make the test suite thoroughly
explainable. Are you planning to attend DjangoCon?

On Mon, Aug 6, 2012 at 1:25 AM, Erik Rose
reply@reply.github.com
wrote:

Just sat down to attack this again, and now I can't repro it anymore. :-( The project I'm testing against has upgraded to 1.4 in the meantime. I wonder if that changed something. Btw, #87 and #88 are related to this.


Reply to this email directly or view it on GitHub:
#76 (comment)

Justin Holmes
Chief Chocobo Breeder, slashRoot
Keynote Program Chair, DjangoCon 2012

slashRoot: Coffee House and Tech Dojo
60 Main Street
New Paltz, NY 12561
845.633.8330

@erikrose

This comment has been minimized.

Contributor

erikrose commented Aug 6, 2012

Hi, Justin. First of all, thanks for your patience and continued interest. :-) Yes, I'll be at DjangoCon—speaking about django-nose, in fact. I don't really like the test suite methodology right now: the trips out to the shell bother me, both philosophically and because they won't work on Windows. I'd like to replace it with something that calls into the runner at a slightly lower level, perhaps invoking the "test" management command, for example. If you felt like porting the existing tests to such a framework, that'd be a welcome contribution.

My next step on this is to write a failing test for it; depending on my primary project to keep having the problem is obviously not working. Even if you or someone else could submit a toy project which exhibits the bug, that'd save me a couple hours and let me get right to the problem when I do have a moment.

@squidsoup

This comment has been minimized.

squidsoup commented Sep 19, 2012

Hi, I'm having issues with REUSE_DB too unfortunately. I'm currently using the dev version of django-nose (although the same behaviour occurs in 1.1.0 from pypi) with a Django 1.2 project and Postgresql.

Running:

REUSE_DB=1 ./manage test myapp
DatabaseError: relation "access_type_id_seq" does not exist

access_type is the first model/table in my app. The error is thrown very quickly, which makes me wonder if test dbs are not being built for some reason.

Tests run, and test dbs build correctly (and slowly...) when running without REUSE_DB.

The only difference in my configuration which is potentially relevant is that I've subclassed NoseTestSuiteRunner to allow unmanaged models to be built for the purpose of testing. This particular project is built on a legacy database which does not directly used the django orm.

Custom test runner

from django.db.models.loading import get_models

from django_nose import NoseTestSuiteRunner

class ManagedNoseTestRunner(NoseTestSuiteRunner):
    def setup_test_environment(self, *args, **kwargs):
        self.unmanaged_models = [m for m in get_models()
                                 if not m._meta.managed]
        for m in self.unmanaged_models:
            m._meta.managed = True
        super(ManagedNoseTestRunner, self).setup_test_environment(*args,
                                                                   **kwargs)
    def teardown_test_environment(self, *args, **kwargs):
        super(ManagedNoseTestRunner, self).teardown_test_environment(*args,
                                                                      **kwargs)
        # reset unmanaged models
        for m in self.unmanaged_models:
            m._meta.managed = False
@alexjg

This comment has been minimized.

Contributor

alexjg commented Oct 16, 2012

Hey guys, I'm being bitten by this issue at the moment. It occurs even if I remove all INSTALLED_APPS. How can I help?

@alexhayes

This comment has been minimized.

alexhayes commented Jun 17, 2014

I'm being bitten by this also and the patch submitted by @carymrobbins / @dullaran doesn't seem to rectify the issue for me.

@alexhayes

This comment has been minimized.

alexhayes commented Jun 18, 2014

I managed to trace down why this issue was affecting my project and can confirm that my comment above still stands, however I managed to get around the issue.

I traced the issue to a form that was attempting to set a property (FOO_CHOICES) equal to the [(f.pk, f.title) for f in Foo.objects.all()]. I assume this is similar to the behaviour that @jMyles is referring to in #76 (comment)

Simple solution for me was to fork the project and use a ModelMultipleChoiceField rather than a MultipleChoiceField with a queryset - however, this issue is VERY concerning!

cmyers734 pushed a commit to newfoundry/django-nose that referenced this issue Sep 8, 2014

Apply django-nose#101
for bug django-nose#76

When REUSE_DB=1, the test suite would end up using the non-test database
@andresfcamacho

This comment has been minimized.

andresfcamacho commented Jan 5, 2015

Our team has this issue as well. We are connecting to a PostGresql database, using nose 1.3.4, django-nose 1.3, Django 1.7. We've noticed that we can reproduce it 100% by executing a specific test (not the entire test suite).

For us we've been able to fix by adding the:

connection.connection = None

line from pull request #101 (line 286). If you look at that PR and compare with django=nose1.3 you'll notice that most of it was applied but line 286 was not. Note sure why. When we add the line in, the issue is fixed, executing a single test properly connects to the test database.

Should line 286 be added?

@trawick

This comment has been minimized.

trawick commented Feb 4, 2015

I'm a bit hesitant to chime in (surely I'm doing something wrong)

nose 1.3.1, django-nose 1.3, Django 1.7.3, psycopg2 2.5.1 talking to PostgreSQL 9.3, running selected tests, seeing indications that the wrong DB is being used with REUSE_DB=1 on a run where the test db doesn't have to be deleted

the indications: first testcase fails for duplicate key (seemingly with real db), subsequent testcases fail complaining that the test db doesn't exist

adding "connection.connection = None" as suggested by andrescamacho just above resolves it so far

@davidek

This comment has been minimized.

davidek commented Mar 12, 2015

It happened to me, too, and gave me a hard time to trace down and get to this issue. Apparently, when runnig selected test cases, the first one among them runs on the non-test database, while the subsequent ones run correctly on the test db (if it was previously set up), or fail (if the test db does not exist).

I found the most direct way for debugging this was to put in my setUp:

from django.db import connection
with open('/tmp/test', 'a') as f:
    f.write(connection.connection.dsn + '\n')

nose==1.3.4, django-nose==1.3, Django==1.7.1, psycopg2==2.6, Postgres 9.3,
using the postgis db engine 'ENGINE': 'django.contrib.gis.db.backends.postgis'

As suggested above, adding connection.connection = None (django_nose/runner.py line 306) fixed it.

@jwhitlock jwhitlock added the bug label Jul 7, 2015

@jwhitlock jwhitlock added this to the Fix REUSE_DB=1 milestone Jul 7, 2015

@evenicoulddoit

This comment has been minimized.

evenicoulddoit commented Sep 25, 2015

I just monkey-patched using 5568665 and it worked, came back here because I destroyed my env needing this again, any updates?

@robguttman

This comment has been minimized.

robguttman commented Sep 25, 2015

I also ran into this same issue today - local db got destroyed. The connection.connection = None fix also worked for me. Can someone speak to this patch or some other plan?

jwhitlock added a commit that referenced this issue Oct 7, 2015

@jwhitlock

This comment has been minimized.

Contributor

jwhitlock commented Oct 7, 2015

I don't use REUSE_DB=1 (and triaging the bugs has probably scared me away from using it), but two changes from @eroninjapan and @alexjg feel like reasonable safe additions that work:

  • Close all open connections
  • Set the connection to None

I'm adding these to v1.4.2. Current django-nose tests don't expose this issue, so I can't tell if these changes fix it either, so I'm leaving the issue open.

jwhitlock added a commit that referenced this issue Oct 7, 2015

@jwhitlock

This comment has been minimized.

Contributor

jwhitlock commented Dec 2, 2015

There are two new issues filed due to these changes. #247 blames a893caf (close all open connections). #235 looks like it might have been using this as a feature, not a bug.

I don't use REUSE_DB=1, so I've seen no change. I'm considering removing a893caf, and seeing if just setting the connection to None will work.

@electroniceagle

This comment has been minimized.

electroniceagle commented Apr 15, 2016

I think that we are experiencing this indirectly through django-aloe:
aloetesting/aloe_django#47

@owais

This comment has been minimized.

owais commented May 11, 2017

I'm facing this issue as well with django-nose==1.4.4 nose==1.3.7 and Django==1.10.7. Oddly, this happens on our CI environment if we run tests by mentioning the tests specifically on the command line. It does not surface if we just run the whole suite.

@owais

This comment has been minimized.

owais commented May 11, 2017

Also, I can consistently reproduce this with or without REUSE_DB=1. It happens even when a new DB is being created.

@owais

This comment has been minimized.

owais commented May 11, 2017

Turns out it was a function being called while setting a default argument for a test case that caused this. The function was trying to fetch data from DB but at the time of loading tests, the DB had not yet changed to the test DB. Fix was as simple as change

def test_example(self, value=get_value_from_db()):
      pass

to

def test_example(self, value=None):
      value = value or get_value_from_db()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment