[bug 971014] Upgrade to django 1.6 #1877

rlr · 2014-03-19T17:17:59Z

Well, I think this is it!

I had to change some strings that get passed into _lazy to be unicode.
The password reset code form had to change to match the latest version in django.
The TransactionMiddleware is now deprecated and it's behavior is the default if you set ATOMIC_REQUESTS = True. So I did that.

r?

willkg · 2014-03-19T17:35:39Z

kitsune/settings.py

@@ -889,3 +881,6 @@ def read_only_mode(env):
 AXES_COOLOFF_TIME = 1  # hour
 AXES_BEHIND_REVERSE_PROXY = True
 AXES_REVERSE_PROXY_HEADER = 'HTTP_X_CLUSTER_CLIENT_IP'
+
+# Set this to True to wrap each HTTP request in a transaction on this database.
+ATOMIC_REQUESTS = True


For the record, I'm pretty sure this is the default.

But it's nice to have here as a reminder that we're doing ATOMIC_REQUESTS.

willkg · 2014-03-19T17:47:20Z

When I run the tests, I get this:

./manage.py test --noinput --nocapture --logging-clear-handlers --with-nicedots -s
nosetests --nocapture --logging-clear-handlers --with-nicedots -s --verbosity=1
Reusing old database "test_kitsune". Set env var FORCE_DB=1 if you need fresh DBs.
Traceback (most recent call last):
  File "./manage.py", line 34, in <module>
    execute_from_command_line(sys.argv)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/management/commands/test.py", line 50, in run_from_argv
    super(Command, self).run_from_argv(argv)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/management/commands/test.py", line 71, in execute
    super(Command, self).execute(*args, **options)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/home/willkg/mozilla/kitsune/vendor/packages/South/south/management/commands/test.py", line 8, in handle
    super(Command, self).handle(*args, **kwargs)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/management/commands/test.py", line 88, in handle
    failures = test_runner.run_tests(test_labels)
  File "/home/willkg/mozilla/kitsune/vendor/src/django-nose/django_nose/runner.py", line 200, in run_tests
    result = self.run_suite(nose_argv)
  File "/home/willkg/mozilla/kitsune/vendor/src/django-nose/django_nose/runner.py", line 147, in run_suite
    addplugins=plugins_to_add)
  File "/home/willkg/mozilla/kitsune/vendor/packages/nose/nose/core.py", line 118, in __init__
    **extra_args)
  File "/usr/lib64/python2.7/unittest/main.py", line 95, in __init__
    self.runTests()
  File "/home/willkg/mozilla/kitsune/vendor/packages/nose/nose/core.py", line 197, in runTests
    result = self.testRunner.run(self.test)
  File "/home/willkg/mozilla/kitsune/vendor/packages/nose/nose/core.py", line 50, in run
    wrapper = self.config.plugins.prepareTest(test)
  File "/home/willkg/mozilla/kitsune/vendor/packages/nose/nose/plugins/manager.py", line 94, in __call__
    return self.call(*arg, **kw)
  File "/home/willkg/mozilla/kitsune/vendor/packages/nose/nose/plugins/manager.py", line 162, in simple
    result = meth(*arg, **kw)
  File "/home/willkg/mozilla/kitsune/vendor/src/django-nose/django_nose/plugin.py", line 75, in prepareTest
    self.old_names = self.runner.setup_databases()
  File "/home/willkg/mozilla/kitsune/vendor/src/test-utils/test_utils/runner.py", line 140, in setup_databases
    connection.commit_unless_managed()  # which it is
AttributeError: 'DatabaseWrapper' object has no attribute 'commit_unless_managed'

If I run FORCE_DB=1 ./manage.py test, then I get tons of these:

Traceback (most recent call last):
  File "/home/willkg/mozilla/kitsune/kitsune/dashboards/tests/test_api.py", line 67, in test_product_filter
    urlparams(reverse('api.wikimetric_list'), format='json',
  File "/home/willkg/mozilla/kitsune/kitsune/sumo/urlresolvers.py", line 51, in reverse
    url = django_reverse(viewname, urlconf, args, kwargs, prefix)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/urlresolvers.py", line 509, in reverse
    return iri_to_uri(resolver._reverse_with_prefix(view, prefix, *args, **kwargs))
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/urlresolvers.py", line 387, in _reverse_with_prefix
    possibilities = self.reverse_dict.getlist(lookup_view)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/urlresolvers.py", line 296, in reverse_dict
    self._populate()
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/urlresolvers.py", line 262, in _populate
    for pattern in reversed(self.url_patterns):
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/urlresolvers.py", line 346, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/urlresolvers.py", line 341, in urlconf_module
    self._urlconf_module = import_module(self.urlconf_name)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/utils/importlib.py", line 40, in import_module
    __import__(name)
  File "/home/willkg/mozilla/kitsune/kitsune/urls.py", line 45, in <module>
    (r'^admin/', include(admin.site.urls)),
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/contrib/admin/sites.py", line 253, in urls
    return self.get_urls(), self.app_name, self.name
  File "/home/willkg/mozilla/kitsune/vendor/src/django-adminplus/adminplus/__init__.py", line 27, in get_urls
    from django.conf.urls.defaults import patterns, url
ImportError: No module named defaults

rlr · 2014-03-19T17:49:53Z

whoa! I only ran with FORCE_DB. I'm curious to see what travis says.

willkg · 2014-03-19T18:04:32Z

I can make the "no module named defaults" ones go away by updating django-adminplus to 0.2.1. That unearths lots of errors like this:

ERROR: kitsune/dashboards/tests/test_api.py:WikiMetricAPITests.test_product_filter
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/willkg/mozilla/kitsune/kitsune/dashboards/tests/test_api.py", line 67, in test_product_filter
    urlparams(reverse('api.wikimetric_list'), format='json',
  File "/home/willkg/mozilla/kitsune/kitsune/sumo/urlresolvers.py", line 51, in reverse
    url = django_reverse(viewname, urlconf, args, kwargs, prefix)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/urlresolvers.py", line 509, in reverse
    return iri_to_uri(resolver._reverse_with_prefix(view, prefix, *args, **kwargs))
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/urlresolvers.py", line 387, in _reverse_with_prefix
    possibilities = self.reverse_dict.getlist(lookup_view)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/urlresolvers.py", line 296, in reverse_dict
    self._populate()
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/urlresolvers.py", line 262, in _populate
    for pattern in reversed(self.url_patterns):
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/urlresolvers.py", line 346, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/urlresolvers.py", line 341, in urlconf_module
    self._urlconf_module = import_module(self.urlconf_name)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/utils/importlib.py", line 40, in import_module
    __import__(name)
  File "/home/willkg/mozilla/kitsune/kitsune/urls.py", line 11, in <module>
    from adminplus import AdminSitePlus
ImportError: cannot import name AdminSitePlus

I think that suggests django-adminplus changed. I didn't look into this further.

Running without FORCE_DB brings up the issue that the mysql backend no longer has that "commit_unless_managed" thing. There's one instance of commit_unless_managed, and it says its a no-op:

https://github.com/django/django/blob/master/django/db/transaction.py#L122

I think this means we need to fix test_utils. I nixed test_utils in Input. We might want to look at doing that for Kitsune. I vaguely remember trying to do that and having some problems, but I don't remember offhand what the problems were.

Hope that helps!

rlr · 2014-03-19T19:02:51Z

OK, tests are passing for me now after pyc cleanup and all ^^

willkg · 2014-03-19T19:32:11Z

I'm looking at this now....

willkg · 2014-03-19T19:33:22Z

Tests pass when I run with FORCE_DB=1, but they fail with this when I don't:

./manage.py test --noinput --nocapture --logging-clear-handlers --with-nicedots -s
nosetests --nocapture --logging-clear-handlers --with-nicedots -s --verbosity=1
Reusing old database "test_kitsune". Set env var FORCE_DB=1 if you need fresh DBs.
Traceback (most recent call last):
  File "./manage.py", line 34, in <module>
    execute_from_command_line(sys.argv)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/management/commands/test.py", line 50, in run_from_argv
    super(Command, self).run_from_argv(argv)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/management/commands/test.py", line 71, in execute
    super(Command, self).execute(*args, **options)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/home/willkg/mozilla/kitsune/vendor/packages/South/south/management/commands/test.py", line 8, in handle
    super(Command, self).handle(*args, **kwargs)
  File "/home/willkg/mozilla/kitsune/vendor/src/django/django/core/management/commands/test.py", line 88, in handle
    failures = test_runner.run_tests(test_labels)
  File "/home/willkg/mozilla/kitsune/vendor/src/django-nose/django_nose/runner.py", line 200, in run_tests
    result = self.run_suite(nose_argv)
  File "/home/willkg/mozilla/kitsune/vendor/src/django-nose/django_nose/runner.py", line 147, in run_suite
    addplugins=plugins_to_add)
  File "/home/willkg/mozilla/kitsune/vendor/packages/nose/nose/core.py", line 118, in __init__
    **extra_args)
  File "/usr/lib64/python2.7/unittest/main.py", line 95, in __init__
    self.runTests()
  File "/home/willkg/mozilla/kitsune/vendor/packages/nose/nose/core.py", line 197, in runTests
    result = self.testRunner.run(self.test)
  File "/home/willkg/mozilla/kitsune/vendor/packages/nose/nose/core.py", line 50, in run
    wrapper = self.config.plugins.prepareTest(test)
  File "/home/willkg/mozilla/kitsune/vendor/packages/nose/nose/plugins/manager.py", line 94, in __call__
    return self.call(*arg, **kw)
  File "/home/willkg/mozilla/kitsune/vendor/packages/nose/nose/plugins/manager.py", line 162, in simple
    result = meth(*arg, **kw)
  File "/home/willkg/mozilla/kitsune/vendor/src/django-nose/django_nose/plugin.py", line 75, in prepareTest
    self.old_names = self.runner.setup_databases()
  File "/home/willkg/mozilla/kitsune/vendor/src/test-utils/test_utils/runner.py", line 140, in setup_databases
    connection.commit_unless_managed()  # which it is
AttributeError: 'DatabaseWrapper' object has no attribute 'commit_unless_managed'

rlr · 2014-03-19T19:34:26Z

GAH! Forgot to look at that one.

It only pulls in settings_test.py

rlr · 2014-03-19T20:05:20Z

OK, this time I ran with and without FORCE_DB... Do I dare ask for a r?

willkg · 2014-03-20T11:43:55Z

kitsune/sumo/tests/__init__.py

+    def setup_test_environment(self, **kwargs):
+        # If we have a settings_test.py let's roll it into our settings.
+        try:
+            import settings_test


I would totally take the opportunity here to put settings_test.py in the right place. I.e. put it in kitsune/settings_test.py and import it with from kitsune import settings_test. Then we have one less file in the root dir and all our settings together!

excellent idea!

rlr · 2014-03-20T12:17:59Z

moved settings_test.py ^

willkg · 2014-03-20T13:22:07Z

Tests pass now. Everything looks good to me.

What's the plan for deploying this? Will we push it to stage, run the QA stuff and then go from there? Do we want this to sit on stage over the weekend?

rlr · 2014-03-20T13:28:07Z

afa2b06 [bug 971014] Upgrade to django 1.6
295285b Update django-adminplus to master tip.
a2b4d77 [bug 971014] Use hashlib instead of django.utils.hashcompat
aa5a535 Change to my fork+branch of django-qunit for now.
a8684ec [bug 971014] Use a less Radical test suite runner
67cb713 Move settings_test.py to kitsune/.

[bug 971014] Upgrade to django 1.6

f490e46

willkg reviewed Mar 19, 2014
View reviewed changes

rlr added 3 commits March 19, 2014 14:32

Update django-adminplus to master tip.

0fd05b1

[bug 971014] Use hashlib instead of django.utils.hashcompat

38e4e9e

Change to my fork+branch of django-qunit for now.

a635757

rlr added 2 commits March 19, 2014 15:38

[bug 971014] Use a less Radical test suite runner

268fad3

Use our own test runner.

f6b435a

It only pulls in settings_test.py

add a docstring.

a7ce53a

willkg reviewed Mar 20, 2014
View reviewed changes

Move settings_test.py to kitsune/.

1943bf4

rlr closed this Mar 20, 2014

rlr deleted the django-1-6 branch March 20, 2014 13:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[bug 971014] Upgrade to django 1.6 #1877

[bug 971014] Upgrade to django 1.6 #1877

rlr commented Mar 19, 2014

willkg Mar 19, 2014

willkg Mar 19, 2014

willkg commented Mar 19, 2014

rlr commented Mar 19, 2014

willkg commented Mar 19, 2014

rlr commented Mar 19, 2014

willkg commented Mar 19, 2014

willkg commented Mar 19, 2014

rlr commented Mar 19, 2014

rlr commented Mar 19, 2014

willkg Mar 20, 2014

rlr Mar 20, 2014

rlr commented Mar 20, 2014

willkg commented Mar 20, 2014

rlr commented Mar 20, 2014

[bug 971014] Upgrade to django 1.6 #1877

[bug 971014] Upgrade to django 1.6 #1877

Conversation

rlr commented Mar 19, 2014

willkg Mar 19, 2014

Choose a reason for hiding this comment

willkg Mar 19, 2014

Choose a reason for hiding this comment

willkg commented Mar 19, 2014

rlr commented Mar 19, 2014

willkg commented Mar 19, 2014

rlr commented Mar 19, 2014

willkg commented Mar 19, 2014

willkg commented Mar 19, 2014

rlr commented Mar 19, 2014

rlr commented Mar 19, 2014

willkg Mar 20, 2014

Choose a reason for hiding this comment

rlr Mar 20, 2014

Choose a reason for hiding this comment

rlr commented Mar 20, 2014

willkg commented Mar 20, 2014

rlr commented Mar 20, 2014