Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sqlite3 support #303

Closed
wants to merge 3 commits into from
Closed

Sqlite3 support #303

wants to merge 3 commits into from

Conversation

jpmckinney
Copy link
Contributor

Includes #296. Trying to fix #302.

Had to use db.execute instead of db.remove_index due to a bug in South.

Since I can't figure out how to update SQLite on Travis, here are the two failures and one error from my local machine. If these can't be fixed, I can back out the changes to _get_descendants and just get the other changes through in this PR.

======================================================================
ERROR: test_add_section_in_section (speeches.tests.section_tests.SectionSiteTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/james/Sites/shared/pulls/sayit/speeches/tests/section_tests.py", line 182, in test_add_section_in_section
    'title': 'A test subsection'
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/test/client.py", line 483, in post
    response = super(Client, self).post(path, data=data, content_type=content_type, **extra)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/test/client.py", line 302, in post
    return self.request(**r)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/test/client.py", line 444, in request
    six.reraise(*exc_info)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/views/generic/base.py", line 69, in view
    return self.dispatch(request, *args, **kwargs)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/utils/decorators.py", line 29, in _wrapper
    return bound_func(*args, **kwargs)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/instances/views.py", line 20, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/utils/decorators.py", line 25, in bound_func
    return func(self, *args2, **kwargs2)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/instances/views.py", line 31, in dispatch
    return super(InstanceFormMixin, self).dispatch(*args, **kwargs)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/views/generic/base.py", line 87, in dispatch
    return handler(request, *args, **kwargs)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/views/generic/edit.py", line 205, in post
    return super(BaseCreateView, self).post(request, *args, **kwargs)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/views/generic/edit.py", line 170, in post
    if form.is_valid():
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/forms/forms.py", line 129, in is_valid
    return self.is_bound and not bool(self.errors)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/forms/forms.py", line 121, in errors
    self.full_clean()
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/forms/forms.py", line 273, in full_clean
    self._clean_fields()
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/forms/forms.py", line 291, in _clean_fields
    value = getattr(self, 'clean_%s' % name)()
  File "/Users/james/Sites/shared/pulls/sayit/speeches/forms.py", line 364, in clean_parent
    descendant_ids = [ d.id for d in self.instance.get_descendants ]
  File "/Users/james/Sites/shared/pulls/sayit/speeches/models.py", line 55, in __get__
    result = self.method(inst)
  File "/Users/james/Sites/shared/pulls/sayit/speeches/models.py", line 316, in get_descendants
    return self._get_descendants_by_speech()
  File "/Users/james/Sites/shared/pulls/sayit/speeches/models.py", line 319, in _get_descendants_by_speech
    dqs = self._get_descendants(include_min=True, **kwargs)
  File "/Users/james/Sites/shared/pulls/sayit/speeches/models.py", line 205, in _get_descendants
    s = s[1:]
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/db/models/query.py", line 1456, in __getitem__
    return list(self)[k]
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/db/models/query.py", line 1402, in __iter__
    for pos, column in enumerate(self.columns):
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/db/models/query.py", line 1479, in columns
    self._columns = self.query.get_columns()
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/db/models/sql/query.py", line 68, in get_columns
    for column_meta in self.cursor.description]
TypeError: 'NoneType' object is not iterable
-------------------- >> begin captured logging << --------------------
django.request: ERROR: Internal Server Error: /sections/add
Traceback (most recent call last):
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/views/generic/base.py", line 69, in view
    return self.dispatch(request, *args, **kwargs)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/utils/decorators.py", line 29, in _wrapper
    return bound_func(*args, **kwargs)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/instances/views.py", line 20, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/utils/decorators.py", line 25, in bound_func
    return func(self, *args2, **kwargs2)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/instances/views.py", line 31, in dispatch
    return super(InstanceFormMixin, self).dispatch(*args, **kwargs)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/views/generic/base.py", line 87, in dispatch
    return handler(request, *args, **kwargs)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/views/generic/edit.py", line 205, in post
    return super(BaseCreateView, self).post(request, *args, **kwargs)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/views/generic/edit.py", line 170, in post
    if form.is_valid():
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/forms/forms.py", line 129, in is_valid
    return self.is_bound and not bool(self.errors)
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/forms/forms.py", line 121, in errors
    self.full_clean()
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/forms/forms.py", line 273, in full_clean
    self._clean_fields()
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/forms/forms.py", line 291, in _clean_fields
    value = getattr(self, 'clean_%s' % name)()
  File "/Users/james/Sites/shared/pulls/sayit/speeches/forms.py", line 364, in clean_parent
    descendant_ids = [ d.id for d in self.instance.get_descendants ]
  File "/Users/james/Sites/shared/pulls/sayit/speeches/models.py", line 55, in __get__
    result = self.method(inst)
  File "/Users/james/Sites/shared/pulls/sayit/speeches/models.py", line 316, in get_descendants
    return self._get_descendants_by_speech()
  File "/Users/james/Sites/shared/pulls/sayit/speeches/models.py", line 319, in _get_descendants_by_speech
    dqs = self._get_descendants(include_min=True, **kwargs)
  File "/Users/james/Sites/shared/pulls/sayit/speeches/models.py", line 205, in _get_descendants
    s = s[1:]
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/db/models/query.py", line 1456, in __getitem__
    return list(self)[k]
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/db/models/query.py", line 1402, in __iter__
    for pos, column in enumerate(self.columns):
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/db/models/query.py", line 1479, in columns
    self._columns = self.query.get_columns()
  File "/Users/james/.virtualenvs/sayit/lib/python2.7/site-packages/django/db/models/sql/query.py", line 68, in get_columns
    for column_meta in self.cursor.description]
TypeError: 'NoneType' object is not iterable
--------------------- >> end captured logging << ---------------------

======================================================================
FAIL: test_section_creation (speeches.tests.section_tests.SectionModelTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/james/Sites/shared/pulls/sayit/speeches/tests/section_tests.py", line 88, in test_section_creation
    self.assertEqual(children, [ 'Monday 25th March', 'Oral Answers to Questions - Silly Walks', 'Bill on Silly Walks', 'Friday 29th March', 'Fixed Easter Bill', 'New Clause 1', 'Clause 1', 'Z Clause' ])
AssertionError: Lists differ: [u'Monday 25th March', u'Oral ... != ['Monday 25th March', 'Oral An...

First differing element 5:
Clause 1
New Clause 1

- [u'Monday 25th March',
?  -

+ ['Monday 25th March',
-  u'Oral Answers to Questions - Silly Walks',
?  -

+  'Oral Answers to Questions - Silly Walks',
-  u'Bill on Silly Walks',
?  -

+  'Bill on Silly Walks',
-  u'Friday 29th March',
?  -

+  'Friday 29th March',
-  u'Fixed Easter Bill',
?  -

+  'Fixed Easter Bill',
-  u'Clause 1',
-  u'New Clause 1',
?  -

+  'New Clause 1',
+  'Clause 1',
-  u'Z Clause']
?  -

+  'Z Clause']

======================================================================
FAIL: test_add_speech_metadata_in_section (speeches.tests.speech_tests.SpeechTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/james/Sites/shared/pulls/sayit/speeches/tests/speech_tests.py", line 214, in test_add_speech_metadata_in_section
    _check_initial(self, resp, section, speakers[2], speech_data)
  File "/Users/james/Sites/shared/pulls/sayit/speeches/tests/speech_tests.py", line 152, in _check_initial_speech_data
    test.assertEqual(initial['speaker'], speaker)
AssertionError: <Speaker: Speaker 1> != <Speaker: Speaker 2>

----------------------------------------------------------------------
Ran 86 tests in 7.915s

FAILED (SKIP=3, errors=1, failures=2)
Destroying test database for alias 'default'...

@dracos
Copy link
Member

dracos commented Oct 15, 2014

Taking the core changes of this branch and running on the current codebase, it looks like this also needs a switch in _get_descendants to use MIN( start_date || ' ' || COALESCE(start_time, '00:00:00') ) (and a reparse of speech_min into datetimes after the query) as sqlite doesn't use + for string concatenation.
I don't understand the strange empty cursor.description, perhaps someone else will; the Speech error is to do with fetching the penultimate speech of the current section so is ordering related; and there's a couple of new issues in the thumbnail generation (I assume sqlite does things in a slightly different order). (was issue in my local install)

@jpmckinney
Copy link
Contributor Author

I re-applied the relevant commits to head, and added a commit based on your comments.

@jpmckinney
Copy link
Contributor Author

My searches re: cursor don't turn up anything relevant.

@jpmckinney
Copy link
Contributor Author

FYI, MySQL doesn't have WITH RECURSIVE. There are solutions using stored procedures or user-defined variables. The code would be very different, so probably don't want to support it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Docs: SayIt is PostgreSQL-specific
3 participants