Skip to content

Tests failing out of the box #116

Closed
greatestape opened this Issue Apr 19, 2011 · 13 comments

3 participants

@greatestape

I installed django-sentry using pip install django-sentry. I created a project with sentry, sentry.client and the dependencies in INSTALLED_APPS.

As far as I can tell I'm following the instructions exactly and my project literally has nothing in it but django sentry, but several of django sentry's tests are failing. Some have failing tests, others seem to be looking for tables that aren't there.

I've tried installing from different sources, using different databases, different logging configs (I'm using Django 1.3), but I can't get rid of the errors.

If you're able to run the sentry test suite without errors, please help me figure out what's wrong with my environment.

Thank you.

@dcramer
Sentry member
dcramer commented Apr 21, 2011

The problem is simply that we don't require certain dependencies as part of the core INSTALLED_APPS.

We'll need to document the requirements for getting tests to pass, as well as make certain tests only run if conditions are met (e.g. haystack [fork], and celery tests should check for their libraries)

@greatestape

Thanks. I agree, and I think there's test skipping support in Django 1.3 that makes this easy.

Unfortunately, I don't think this explains all the errors I'm seeing. For example, there are a few errors complaining about tables not being found in the db that are declared in tests/models.py and there's a missing template error for a template in tests/templates/sentry/. It looks like these tests at least are expecting that the tests dir itself will be added to INSTALLED_APPS.

I discovered the runtests.py file and I can see how it's tweaking the settings to let the test suite run correctly. The problem seems to be that these tests require a special environment in order to run properly, but they're being exposed as regular Django app unit tests so they're getting run in the regular environment every time somebody runs ./manage.py test.

I think you should consider extracting these internal tests into a test dir that's kept at the same level as the sentry package dir. I've seen that pattern elsewhere and I've followed it in my own reusable apps. That way the tests can still be run by django-sentry's developers, but they don't get run inappropriately by django's test runner. I think the only tests that should be kept inside sentry/tests should be tests that don't require any special configuration. The only tests I think you really need inside sentry/tests are the ones that test that your app has been configured properly by the deployer (proper settings set, dependencies met, etc.)

@dcramer
Sentry member
dcramer commented Apr 21, 2011

I think that's probably a good approach. Is there any good material around for the best way to do that? I think I've only ever written embedded tests using Django's test suite.

@greatestape

Well, you can check out my reusable app, transurlvania: https://github.com/trapeze/transurlvania. Basically, my test dir is a small django project implementing my app with enough stuff in it to exercise all my app's features.

It might not take much more than just relocating the bulk of your test code outside of the sentry package.

@greatestape

Oh, Django does something sort of similar for its tests, so you could check them out too.

@dcramer
Sentry member
dcramer commented Apr 25, 2011

First attempt at separating these out failed miserably. Not sure if I'll get this into 1.7 or not

@dcramer
Sentry member
dcramer commented Apr 27, 2011

I've added in some fake test skipping for now. Haystack and Celery tests will just return a pass (and warn) if they get skipped.

@jonrohan

We want to use sentry at simplegeo, but this is blocking us.

@dcramer
Sentry member
dcramer commented May 26, 2011

@JonRohan is this still a problem in the current version? I added some skip test functionality, albeit mostly untested, that should have worked

@jonrohan

@dcramer pulled an update today and when running ./manage.py test it runs the sentry tests and I get this https://gist.github.com/992570

How do I set the skip tests?

@dcramer
Sentry member
dcramer commented May 26, 2011

@JonRohan Hrm this could be a problem with the MySQL support. Does the test DB set encoding to utf-8?

@dcramer
Sentry member
dcramer commented May 26, 2011

Rather the first few look like that. Would it be acceptable if the tests just didn't run at all?

It's been on the roster for a while just havent managed to pull them out of the Django pattern

@jonrohan

If there was a way to just turn the tests off that'd be great. Sentry works, and we have it installed properly, our deployment system however doesn't allow the build to continue if any tests are failing.

@dcramer dcramer closed this in 87c763b May 26, 2011
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.