Skip to content
This repository has been archived by the owner on Feb 20, 2019. It is now read-only.

tests fail with elasticsearch 1.0 #241

Closed
willkg opened this issue May 29, 2014 · 0 comments · Fixed by #242
Closed

tests fail with elasticsearch 1.0 #241

willkg opened this issue May 29, 2014 · 0 comments · Fixed by #242
Milestone
0.10

Comments

@willkg
Copy link
Member

willkg commented May 29, 2014

Getting these test errors with elasticsearch 1.0 and elasticsearch-py 0.45

======================================================================
ERROR: elasticutils/contrib/django/tests/test_models.py:IndexableTest.test_bulk_index
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/willkg/mozilla/elasticutils/elasticutils/contrib/django/tests/test_models.py", line 74, in test_bulk_index
    FakeDjangoMappingType.bulk_index(documents, id_field='id')
  File "/home/willkg/mozilla/elasticutils/elasticutils/__init__.py", line 2215, in bulk_index
    raise_on_error=True
  File "/home/willkg/.virtualenvs/eu/lib/python2.7/site-packages/elasticsearch/helpers.py", line 148, in bulk
    for ok, item in streaming_bulk(client, actions, **kwargs):
  File "/home/willkg/.virtualenvs/eu/lib/python2.7/site-packages/elasticsearch/helpers.py", line 121, in streaming_bulk
    raise BulkIndexError('%i document(s) failed to index.' % len(errors), errors)
BulkIndexError: ('3 document(s) failed to index.', [{u'index': {u'status': 201, u'_type': u'fake', u'_id': u'1', u'_version': 1, u'_index': u'elasticutilstest_eutest'}}, {u'index': {u'status': 201, u'_type': u'fake', u'_id': u'2', u'_version': 1, u'_index': u'elasticutilstest_eutest'}}, {u'index': {u'status': 201, u'_type': u'fake', u'_id': u'3', u'_version': 1, u'_index': u'elasticutilstest_eutest'}}])

======================================================================
ERROR: elasticutils/contrib/django/tests/test_tasks.py:TestTasks.test_tasks
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/willkg/mozilla/elasticutils/elasticutils/contrib/django/tests/test_tasks.py", line 35, in test_tasks
    index_objects(FakeDjangoMappingType, [1, 2, 3])
  File "/home/willkg/.virtualenvs/eu/lib/python2.7/site-packages/celery/app/task.py", line 420, in __call__
    return self.run(*args, **kwargs)
  File "/home/willkg/mozilla/elasticutils/elasticutils/contrib/django/tasks.py", line 65, in index_objects
    mapping_type.bulk_index(documents, id_field='id', es=es, index=index)
  File "/home/willkg/mozilla/elasticutils/elasticutils/__init__.py", line 2215, in bulk_index
    raise_on_error=True
  File "/home/willkg/.virtualenvs/eu/lib/python2.7/site-packages/elasticsearch/helpers.py", line 148, in bulk
    for ok, item in streaming_bulk(client, actions, **kwargs):
  File "/home/willkg/.virtualenvs/eu/lib/python2.7/site-packages/elasticsearch/helpers.py", line 121, in streaming_bulk
    raise BulkIndexError('%i document(s) failed to index.' % len(errors), errors)
BulkIndexError: ('3 document(s) failed to index.', [{u'index': {u'status': 201, u'_type': u'fake', u'_id': u'1', u'_version': 1, u'_index': u'elasticutilstest_eutest'}}, {u'index': {u'status': 201, u'_type': u'fake', u'_id': u'2', u'_version': 1, u'_index': u'elasticutilstest_eutest'}}, {u'index': {u'status': 201, u'_type': u'fake', u'_id': u'3', u'_version': 1, u'_index': u'elasticutilstest_eutest'}}])

======================================================================
ERROR: elasticutils/tests/test_types.py:ModelTest.test_bulk_index
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/willkg/mozilla/elasticutils/elasticutils/tests/test_types.py", line 110, in test_bulk_index
    FakeMappingType.bulk_index(documents, id_field='id')
  File "/home/willkg/mozilla/elasticutils/elasticutils/__init__.py", line 2215, in bulk_index
    raise_on_error=True
  File "/home/willkg/.virtualenvs/eu/lib/python2.7/site-packages/elasticsearch/helpers.py", line 148, in bulk
    for ok, item in streaming_bulk(client, actions, **kwargs):
  File "/home/willkg/.virtualenvs/eu/lib/python2.7/site-packages/elasticsearch/helpers.py", line 121, in streaming_bulk
    raise BulkIndexError('%i document(s) failed to index.' % len(errors), errors)
BulkIndexError: ('2 document(s) failed to index.', [{u'index': {u'status': 201, u'_type': u'elasticutilsmappingtype', u'_id': u'1', u'_version': 1, u'_index': u'elasticutilstest'}}, {u'index': {u'status': 201, u'_type': u'elasticutilsmappingtype', u'_id': u'2', u'_version': 1, u'_index': u'elasticutilstest'}}])

Need to look into that.
@willkg willkg added this to the 0.10 milestone May 29, 2014
@willkg willkg mentioned this issue May 30, 2014
Merged
einvalentin pushed a commit to einvalentin/elasticutils that referenced this issue Oct 2, 2014
@willkg @einvalentin
Fix bulk indexing problems with Elasticsearch 1.0
7936a36 
In order for ElasticUtils to work for both Elasticsearch 0.90 and
Elasticsearch 1.0 using elasticsearch-py 0.4.5, we need to do some
monkey-patching of elasticsearch-py.

In this case, calling Elasticsearch.client.bulk() returns an 'ok' field
with ES 0.90 and a 'status' field with ES 1.0. This patch sets the 'ok'
field based on the 'status' field so that the bulk indexing
infrastructure in elasticsearch-py 0.4.5 is testing the right thing and
not raising BulkIndexingErrors.

Fixes mozilla#241
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
0.10
Development

Successfully merging a pull request may close this issue.

Fix bulk indexing problems with Elasticsearch 1.0
1 participant
@willkg