TypeError: Element() keywords must be strings

[srusskih]

Traceback (most recent call last):
  File "/storage/pydev/feelfree-v4/feelfree/../lib/haystack/management/commands/update_index.py", line 210, in handle_
label
    self.update_backend(label, using)
  File "/storage/pydev/feelfree-v4/feelfree/../lib/haystack/management/commands/update_index.py", line 256, in update_
backend
    do_update(backend, index, qs, start, end, total, self.verbosity)
  File "/storage/pydev/feelfree-v4/feelfree/../lib/haystack/management/commands/update_index.py", line 78, in do_updat
e
    backend.update(index, current_qs)
  File "/storage/pydev/feelfree-v4/feelfree/../lib/haystack/backends/solr_backend.py", line 66, in update
    self.conn.add(docs, commit=commit, boost=index.get_field_weights())
  File "/storage/pydev/feelfree-v4/feelfree/../lib/pysolr.py", line 740, in add
    message.append(self._build_doc(doc, boost=boost))
  File "/storage/pydev/feelfree-v4/feelfree/../lib/pysolr.py", line 695, in _build_doc
    field = ET.Element('field', **attrs)
  File "lxml.etree.pyx", line 2560, in lxml.etree.Element (src/lxml/lxml.etree.c:52924)
TypeError: Element() keywords must be strings

attrs looks like {u'name': 'django_ct'}

env:

• Linux 3.5.0-22-generic Change for Python 2.4 compatibility #34-Ubuntu SMP Tue Jan 8 21:47:00 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
• python 2.7.3
• pysolr 3.0.5
• lxml 2.3.5

Possible fix (works for me)

@@ -687,10 +687,10 @@ class Solr(object):
                 if self._is_null_value(bit):
                     continue

-                attrs = {'name': key}
+                attrs = {str('name'): key}

                 if boost and key in boost:
-                    attrs['boost'] = force_unicode(boost[key])
+                    attrs[str('boost')] = force_unicode(boost[key])

                 field = ET.Element('field', **attrs)
                 field.text = self._from_python(bit)

[acdha]

This doesn't reproduce with newer versions of lxml - if possible, it'd be good to test with at least 2.3.6 but it appears that simply using a newer version of Cython during compilation might be sufficient based on this compatibility fix which they made about six months ago:

http://comments.gmane.org/gmane.comp.python.cython.user/7190
cython/cython@a8f70e2

Chris

[srusskih]

lxml was built without cython. and I have same problems with lxml-2.3.6
Works well with lxml-3.0

[acdha]

lxml 2.3 appears to ship with sources compiled using Cython 0.14. It looks like you will need to either switch to lxml 3.0 or install Cython 0.17 or later before installing lxml 2.3.x:

$ git tag --contains a8f70e2210b9a36c3b67ce5f426000c62473b62e
0.17
0.17.1
0.17.4
0.17b4
0.18

If lxml 2.x is still supported it might be worth creating an issue on lxml for an upstream release using a version of Cython which is fully compatible with Python 2.6+ and unicode_literals.

Chris

[srusskih]

Thanks. I have found the solution (change lxml version).

[stugots]

We just ran into this. Is lxml version xxxxxx specified as a dependency by pysolr? I.e., when pysolr installs, does it look for lxml>=3.x.y?

[acdha]

@stugots Not by default – it's optional and only used for parsing Tomcat error messages so we don't want to install it everywhere. I updated the extras_require stanza but you'll need to use the extras syntax to install – e.g. pip install pysolr[tomcat]==3.0.6. See http://pythonhosted.org/distribute/setuptools.html#declaring-dependencies for more information.

[xudonax]

This seems to be back in the latest GIT revision of django-haystack... The traceback seems to indicate that the error is in lxml, but there is practically no way to do this on Ubuntu 12.10. I tried the "patch" above and this seems to work.

Traceback (most recent call last):
  File "/home/patrick/spng/lib/python2.7/site-packages/django/core/handlers/base.py", line 115, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/home/patrick/spng/lib/python2.7/site-packages/django/db/transaction.py", line 223, in inner
    return func(*args, **kwargs)
  File "/home/patrick/spng/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 25, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/home/patrick/spng/src/internship/views.py", line 248, in register_step_4
    person.save()
  File "/home/patrick/spng/src/person/models.py", line 128, in save
    self.update_solr()
  File "/home/patrick/spng/src/person/models.py", line 135, in update_solr
    InternIndex().update_object(self.intern)
  File "/home/patrick/spng/src/haystack/indexes.py", line 266, in update_object
    self._get_backend(using).update(self, [instance])
  File "/home/patrick/spng/src/haystack/backends/solr_backend.py", line 66, in update
    self.conn.add(docs, commit=commit, boost=index.get_field_weights())
  File "/home/patrick/spng/lib/python2.7/site-packages/pysolr.py", line 740, in add
    message.append(self._build_doc(doc, boost=boost))
  File "/home/patrick/spng/lib/python2.7/site-packages/pysolr.py", line 695, in _build_doc
    field = ET.Element('field', **attrs)
  File "lxml.etree.pyx", line 2560, in lxml.etree.Element (src/lxml/lxml.etree.c:52924)
TypeError: Element() keywords must be strings

[acdha]

@xudonax Did you try upgrading to lxml 3? It's a clean install on Ubuntu.

[xudonax]

I couldn't update lxml in any way. apt-get update && apt-get upgrade didn't give me any upgrades. Then again, I'm still on 12.10, I guess I can wait a little while and check it out on Ubuntu 13.04 which should be released any day now.

[acdha]

@xudonax Ubuntu won't ship an updated lxml system package but you can install lxml 3 along with everything else from PyPI inside a virtualenv – I've run it on 12.04 servers that way for awhile.

[xudonax]

I got an gcc compilation error. I'll take a look at dependencies tomorrow. So far I installed only libxml-dev AFAIK, because it couldn't find python-dev...

[alex-dow]

I get the same issue. Using lxml 3.2.0 pysolr 3.0.6

[Brant]

I was running into this issue. The following setup worked for me:
Python 2.7.4
pysolr 3.0.6
lxml 3.2.1

[shibz]

I had this issue, but fixed it by changing to lxml 3.0.2. I'm using pysolr 3.0.6.

[t2y]

I had the same issue.
Python 2.7.3
pysolr 3.1.0 (02e7b65)
lxml 2.3.2

It works for me after upgrading lxml 2.3.2 to 3.2.4.

[acdha]

@toastdriven Think it's worth making our lxml check do something like issue a warning and fallback to stdlib if lxml is present but <3.x?

[tisto]

I just ran into that issue (from 2013?). Wouldn't it be better to make pysolr require a certain lxml version in its setup.py to avoid problems like this?

[acdha]

@tisto The problem was that people had other versions of lxml in their projects & it'd conflict for something which is only relevant to a subset of users. My preferred approach is the one from #133 (comment) where we simply drop lxml entirely since it's only used for error handling. I just wrapped that code into #149 for easier testing / commenting.