Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-field and suggest api error #3469

Closed
mathieu007 opened this Issue Aug 9, 2013 · 8 comments

Comments

Projects
None yet
4 participants
@mathieu007
Copy link

commented Aug 9, 2013

The following error occur while trying to do a simple phrase query on a multi-field using the suggest api:

"failures" : [ {
      "status" : 400,
      "reason" : "ElasticSearchIllegalArgumentException[generator field [title] doesn't exist]"
    }, {
      "status" : 400,
      "reason" : "RemoteTransportException[[Bella Donna][inet[/127.0.0.1:9301]][search/phase/query]]; nested: ElasticSearchIllegalArgumentException[generator field [title] doesn't exist]; "
    } ]

And here is the code:

curl -XDELETE "http://localhost:9200/test?pretty"
curl -XPOST "http://localhost:9200/test?pretty" -d '{
  "settings": {
    "index": {
      "number_of_replicas": 0,
      "analysis":{
        "analyzer":{
          "suggest":{
            "type": "custom",
            "tokenizer": "standard",
            "filter": [ "standard", "lowercase", "suggest_shingle" ]
          }
        },
        "filter":{
          "suggest_shingle":{
            "type": "shingle",
            "min_shingle_size": 2,
            "max_shingle_size": 5,
            "output_unigrams": true
          }
        }
      }
    }
  }
}'
curl -XPOST "http://localhost:9200/test/test/_mapping?pretty" -d '{
  "test": {
 "properties" : {
    "title": {
      "path": "just_name",
      "type": "multi_field",
      "fields": 
        {
          "title": {
            "type": "string",
            "index": "analyzed",
            "similarity": "BM25",
            "analyzer": "suggest"
          }
        }
      }
      
    }
  }
}'


curl -XPOST "http://localhost:9200/test/test?pretty" -d '{
  "title": "Just testing the suggestions api"
}'

curl -XPOST "http://localhost:9200/test/test?pretty" -d '{
  "title": "An other title"
}'

curl -XGET "http://localhost:9200/test/test/_search?pretty" -d '{
  "query": {
    "match_all": {}
  },
  "suggest": {
    "text": "tetsting sugestion",
    "phrase":{
      "phrase":{
        "field": "title",
        "max_errors": 5
      }
    }
  }
}'
@dadoonet

This comment has been minimized.

Copy link
Member

commented Aug 9, 2013

Does it work if you search on title.title?

@ghost ghost assigned dadoonet Aug 9, 2013

@dadoonet

This comment has been minimized.

Copy link
Member

commented Aug 9, 2013

That's true, it fails.

But when using another sub field in multi field, it works fine:

curl -XDELETE "http://localhost:9200/test?pretty"
curl -XPOST "http://localhost:9200/test?pretty" -d '{
  "settings": {
    "index": {
      "number_of_replicas": 0,
      "analysis":{
        "analyzer":{
          "suggest":{
            "type": "custom",
            "tokenizer": "standard",
            "filter": [ "standard", "lowercase", "suggest_shingle" ]
          }
        },
        "filter":{
          "suggest_shingle":{
            "type": "shingle",
            "min_shingle_size": 2,
            "max_shingle_size": 5,
            "output_unigrams": true
          }
        }
      }
    }
  }
}'
curl -XPOST "http://localhost:9200/test/test/_mapping?pretty" -d '{
  "test": {
 "properties" : {
    "title": {
      "path": "just_name",
      "type": "multi_field",
      "fields": 
        {
           "title": {
            "type": "string",
            "index": "analyzed",
            "similarity": "BM25",
            "analyzer": "suggest"
          },
           "title_suggest": {
            "type": "string",
            "index": "analyzed",
            "similarity": "BM25",
            "analyzer": "suggest"
          }
        }
      }
    }
  }
}'


curl -XPOST "http://localhost:9200/test/test?pretty" -d '{
  "title": "Just testing the suggestions api"
}'

curl -XPOST "http://localhost:9200/test/test?pretty&refresh" -d '{
  "title": "An other title"
}'

curl -XGET "http://localhost:9200/test/test/_search?pretty" -d '{
  "suggest": {
    "text": "tetsting sugestion",
    "phrase":{
      "phrase":{
        "field": "title_suggest",
        "max_errors": 5
      }
    }
  }
}'

I'm going to look at it but I don't think you can use suggest with multifield and only one field declared under (the default one).

@dadoonet

This comment has been minimized.

Copy link
Member

commented Aug 9, 2013

I found the issue which is not exactly an issue.

If you add a sleep in your script, it works fine:

curl -XDELETE "http://localhost:9200/test?pretty"
curl -XPOST "http://localhost:9200/test?pretty" -d '{
  "settings": {
    "index": {
      "number_of_replicas": 0,
      "analysis":{
        "analyzer":{
          "suggest":{
            "type": "custom",
            "tokenizer": "standard",
            "filter": [ "standard", "lowercase", "suggest_shingle" ]
          }
        },
        "filter":{
          "suggest_shingle":{
            "type": "shingle",
            "min_shingle_size": 2,
            "max_shingle_size": 5,
            "output_unigrams": true
          }
        }
      }
    }
  }
}'

curl -XPOST "http://localhost:9200/test/test/_mapping?pretty" -d '{
  "test": {
 "properties" : {
    "title": {
      "path": "just_name",
      "type": "multi_field",
      "fields": 
        {
           "title": {
            "type": "string",
            "index": "analyzed",
            "similarity": "BM25",
            "analyzer": "suggest"
          }
        }
      }
    }
  }
}'


curl -XPOST "http://localhost:9200/test/test?pretty" -d '{
  "title": "Just testing the suggestions api"
}'

curl -XPOST "http://localhost:9200/test/test?pretty&refresh" -d '{
  "title": "An other title"
}'

sleep 5

curl -XGET "http://localhost:9200/test/test/_search?pretty" -d '{
  "suggest": {
    "text": "tetsting sugestion",
    "phrase":{
      "phrase":{
        "field": "title",
        "max_errors": 5
      }
    }
  }
}'
@dadoonet

This comment has been minimized.

Copy link
Member

commented Aug 9, 2013

Sorry. I was wrong.

You have two issues here:

  • In your script, you need to add a refresh after the last index command
  • You have 5 shards but only 2 documents. Suggestion on other empty shards does not work which cause the error you are seeing.

The following script is working fine:

curl -XDELETE "http://localhost:9200/test?pretty"
curl -XPOST "http://localhost:9200/test?pretty" -d '{
  "settings": {
    "index": {
      "number_of_shards": 1,
      "number_of_replicas": 0,
      "analysis":{
        "analyzer":{
          "suggest":{
            "type": "custom",
            "tokenizer": "standard",
            "filter": [ "standard", "lowercase", "suggest_shingle" ]
          }
        },
        "filter":{
          "suggest_shingle":{
            "type": "shingle",
            "min_shingle_size": 2,
            "max_shingle_size": 5,
            "output_unigrams": true
          }
        }
      }
    }
  }
}'

curl -XPOST "http://localhost:9200/test/test/_mapping?pretty" -d '{
  "test": {
 "properties" : {
    "title": {
      "path": "just_name",
      "type": "multi_field",
      "fields": 
        {
           "title": {
            "type": "string",
            "index": "analyzed",
            "similarity": "BM25",
            "analyzer": "suggest"
          }
        }
      }
    }
  }
}'


curl -XPOST "http://localhost:9200/test/test?pretty" -d '{
  "title": "Just testing the suggestions api"
}'

curl -XPOST "http://localhost:9200/test/test?pretty&refresh" -d '{
  "title": "An other title"
}'

curl -XGET "http://localhost:9200/test/test/_search?pretty" -d '{
  "suggest": {
    "text": "tetsting sugestion",
    "phrase":{
      "phrase":{
        "field": "title",
        "max_errors": 5
      }
    }
  }
}'

I keep this issue opened to see if we can not fail in case of empty shards.

@mathieu007

This comment has been minimized.

Copy link
Author

commented Aug 9, 2013

Ah , yeah your right, i understand now why i received this error, it make sense...
Thanks for pointing me the errors in my script, will remember that for next time.

Math

@nik9000

This comment has been minimized.

Copy link
Contributor

commented Aug 9, 2013

I've been noticing that suggestions fail on empty shards problem for the past few days. If no one gets to it the in next few days I'll have a crack at fixing the errors on empty shards.

@dadoonet

This comment has been minimized.

Copy link
Member

commented Aug 9, 2013

Closing this one as there is now issue #3473 and PR #3475 relative to it.

@dadoonet dadoonet closed this Aug 9, 2013

dadoonet added a commit to dadoonet/elasticsearch that referenced this issue Aug 16, 2013

Suggest should ignore empty shards
From elastic#3469.
When running suggest on empty shards, it raises an error like:

```
"failures" : [ {
      "status" : 400,
      "reason" : "ElasticSearchIllegalArgumentException[generator field [title] doesn't exist]"
    } ]
```

We should ignore empty shards.

Closes elastic#3473.

dadoonet added a commit that referenced this issue Aug 16, 2013

Suggest should ignore empty shards
From #3469.
When running suggest on empty shards, it raises an error like:

```
"failures" : [ {
      "status" : 400,
      "reason" : "ElasticSearchIllegalArgumentException[generator field [title] doesn't exist]"
    } ]
```

We should ignore empty shards.

Closes #3473.

dadoonet added a commit that referenced this issue Aug 16, 2013

Suggest should ignore empty shards
From #3469.
When running suggest on empty shards, it raises an error like:

```
"failures" : [ {
      "status" : 400,
      "reason" : "ElasticSearchIllegalArgumentException[generator field [title] doesn't exist]"
    } ]
```

We should ignore empty shards.

Closes #3473.

s1monw added a commit to s1monw/elasticsearch that referenced this issue Aug 16, 2013

Prevent Phrase Suggester from failing on missing fields.
Unless the field is not mapped phrase suggester should return
empty results or skip candidate generation if a field in not in
the index rather than failing hard with an illegal argument exception.
Some shards might not have a value in a certain field.

Closes elastic#3469
@s1monw

This comment has been minimized.

Copy link
Contributor

commented Aug 16, 2013

reopening since this is really a different issue than having an entirely empty shard. here we can have no value in a field rather than having no docs.

@s1monw s1monw reopened this Aug 16, 2013

s1monw added a commit to s1monw/elasticsearch that referenced this issue Aug 16, 2013

Prevent Phrase Suggester from failing on missing fields.
Unless the field is not mapped phrase suggester should return
empty results or skip candidate generation if a field in not in
the index rather than failing hard with an illegal argument exception.
Some shards might not have a value in a certain field.

Closes elastic#3469

@s1monw s1monw closed this in 57c0d29 Aug 16, 2013

s1monw added a commit that referenced this issue Aug 16, 2013

Prevent Phrase Suggester from failing on missing fields.
Unless the field is not mapped phrase suggester should return
empty results or skip candidate generation if a field in not in
the index rather than failing hard with an illegal argument exception.
Some shards might not have a value in a certain field.

Closes #3469

@ghost ghost assigned s1monw Aug 16, 2013

mute pushed a commit to mute/elasticsearch that referenced this issue Jul 29, 2015

Suggest should ignore empty shards
From elastic#3469.
When running suggest on empty shards, it raises an error like:

```
"failures" : [ {
      "status" : 400,
      "reason" : "ElasticSearchIllegalArgumentException[generator field [title] doesn't exist]"
    } ]
```

We should ignore empty shards.

Closes elastic#3473.

mute pushed a commit to mute/elasticsearch that referenced this issue Jul 29, 2015

Prevent Phrase Suggester from failing on missing fields.
Unless the field is not mapped phrase suggester should return
empty results or skip candidate generation if a field in not in
the index rather than failing hard with an illegal argument exception.
Some shards might not have a value in a certain field.

Closes elastic#3469
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.