Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-field and suggest api error #3469

Closed
mathieu007 opened this issue Aug 9, 2013 · 8 comments
Closed

Multi-field and suggest api error #3469

mathieu007 opened this issue Aug 9, 2013 · 8 comments

Comments

@mathieu007
Copy link

The following error occur while trying to do a simple phrase query on a multi-field using the suggest api:

"failures" : [ {
      "status" : 400,
      "reason" : "ElasticSearchIllegalArgumentException[generator field [title] doesn't exist]"
    }, {
      "status" : 400,
      "reason" : "RemoteTransportException[[Bella Donna][inet[/127.0.0.1:9301]][search/phase/query]]; nested: ElasticSearchIllegalArgumentException[generator field [title] doesn't exist]; "
    } ]

And here is the code:

curl -XDELETE "http://localhost:9200/test?pretty"
curl -XPOST "http://localhost:9200/test?pretty" -d '{
  "settings": {
    "index": {
      "number_of_replicas": 0,
      "analysis":{
        "analyzer":{
          "suggest":{
            "type": "custom",
            "tokenizer": "standard",
            "filter": [ "standard", "lowercase", "suggest_shingle" ]
          }
        },
        "filter":{
          "suggest_shingle":{
            "type": "shingle",
            "min_shingle_size": 2,
            "max_shingle_size": 5,
            "output_unigrams": true
          }
        }
      }
    }
  }
}'
curl -XPOST "http://localhost:9200/test/test/_mapping?pretty" -d '{
  "test": {
 "properties" : {
    "title": {
      "path": "just_name",
      "type": "multi_field",
      "fields": 
        {
          "title": {
            "type": "string",
            "index": "analyzed",
            "similarity": "BM25",
            "analyzer": "suggest"
          }
        }
      }
      
    }
  }
}'


curl -XPOST "http://localhost:9200/test/test?pretty" -d '{
  "title": "Just testing the suggestions api"
}'

curl -XPOST "http://localhost:9200/test/test?pretty" -d '{
  "title": "An other title"
}'

curl -XGET "http://localhost:9200/test/test/_search?pretty" -d '{
  "query": {
    "match_all": {}
  },
  "suggest": {
    "text": "tetsting sugestion",
    "phrase":{
      "phrase":{
        "field": "title",
        "max_errors": 5
      }
    }
  }
}'
@dadoonet
Copy link
Member

dadoonet commented Aug 9, 2013

Does it work if you search on title.title?

@ghost ghost assigned dadoonet Aug 9, 2013
@dadoonet
Copy link
Member

dadoonet commented Aug 9, 2013

That's true, it fails.

But when using another sub field in multi field, it works fine:

curl -XDELETE "http://localhost:9200/test?pretty"
curl -XPOST "http://localhost:9200/test?pretty" -d '{
  "settings": {
    "index": {
      "number_of_replicas": 0,
      "analysis":{
        "analyzer":{
          "suggest":{
            "type": "custom",
            "tokenizer": "standard",
            "filter": [ "standard", "lowercase", "suggest_shingle" ]
          }
        },
        "filter":{
          "suggest_shingle":{
            "type": "shingle",
            "min_shingle_size": 2,
            "max_shingle_size": 5,
            "output_unigrams": true
          }
        }
      }
    }
  }
}'
curl -XPOST "http://localhost:9200/test/test/_mapping?pretty" -d '{
  "test": {
 "properties" : {
    "title": {
      "path": "just_name",
      "type": "multi_field",
      "fields": 
        {
           "title": {
            "type": "string",
            "index": "analyzed",
            "similarity": "BM25",
            "analyzer": "suggest"
          },
           "title_suggest": {
            "type": "string",
            "index": "analyzed",
            "similarity": "BM25",
            "analyzer": "suggest"
          }
        }
      }
    }
  }
}'


curl -XPOST "http://localhost:9200/test/test?pretty" -d '{
  "title": "Just testing the suggestions api"
}'

curl -XPOST "http://localhost:9200/test/test?pretty&refresh" -d '{
  "title": "An other title"
}'

curl -XGET "http://localhost:9200/test/test/_search?pretty" -d '{
  "suggest": {
    "text": "tetsting sugestion",
    "phrase":{
      "phrase":{
        "field": "title_suggest",
        "max_errors": 5
      }
    }
  }
}'

I'm going to look at it but I don't think you can use suggest with multifield and only one field declared under (the default one).

@dadoonet
Copy link
Member

dadoonet commented Aug 9, 2013

I found the issue which is not exactly an issue.

If you add a sleep in your script, it works fine:

curl -XDELETE "http://localhost:9200/test?pretty"
curl -XPOST "http://localhost:9200/test?pretty" -d '{
  "settings": {
    "index": {
      "number_of_replicas": 0,
      "analysis":{
        "analyzer":{
          "suggest":{
            "type": "custom",
            "tokenizer": "standard",
            "filter": [ "standard", "lowercase", "suggest_shingle" ]
          }
        },
        "filter":{
          "suggest_shingle":{
            "type": "shingle",
            "min_shingle_size": 2,
            "max_shingle_size": 5,
            "output_unigrams": true
          }
        }
      }
    }
  }
}'

curl -XPOST "http://localhost:9200/test/test/_mapping?pretty" -d '{
  "test": {
 "properties" : {
    "title": {
      "path": "just_name",
      "type": "multi_field",
      "fields": 
        {
           "title": {
            "type": "string",
            "index": "analyzed",
            "similarity": "BM25",
            "analyzer": "suggest"
          }
        }
      }
    }
  }
}'


curl -XPOST "http://localhost:9200/test/test?pretty" -d '{
  "title": "Just testing the suggestions api"
}'

curl -XPOST "http://localhost:9200/test/test?pretty&refresh" -d '{
  "title": "An other title"
}'

sleep 5

curl -XGET "http://localhost:9200/test/test/_search?pretty" -d '{
  "suggest": {
    "text": "tetsting sugestion",
    "phrase":{
      "phrase":{
        "field": "title",
        "max_errors": 5
      }
    }
  }
}'

@dadoonet
Copy link
Member

dadoonet commented Aug 9, 2013

Sorry. I was wrong.

You have two issues here:

  • In your script, you need to add a refresh after the last index command
  • You have 5 shards but only 2 documents. Suggestion on other empty shards does not work which cause the error you are seeing.

The following script is working fine:

curl -XDELETE "http://localhost:9200/test?pretty"
curl -XPOST "http://localhost:9200/test?pretty" -d '{
  "settings": {
    "index": {
      "number_of_shards": 1,
      "number_of_replicas": 0,
      "analysis":{
        "analyzer":{
          "suggest":{
            "type": "custom",
            "tokenizer": "standard",
            "filter": [ "standard", "lowercase", "suggest_shingle" ]
          }
        },
        "filter":{
          "suggest_shingle":{
            "type": "shingle",
            "min_shingle_size": 2,
            "max_shingle_size": 5,
            "output_unigrams": true
          }
        }
      }
    }
  }
}'

curl -XPOST "http://localhost:9200/test/test/_mapping?pretty" -d '{
  "test": {
 "properties" : {
    "title": {
      "path": "just_name",
      "type": "multi_field",
      "fields": 
        {
           "title": {
            "type": "string",
            "index": "analyzed",
            "similarity": "BM25",
            "analyzer": "suggest"
          }
        }
      }
    }
  }
}'


curl -XPOST "http://localhost:9200/test/test?pretty" -d '{
  "title": "Just testing the suggestions api"
}'

curl -XPOST "http://localhost:9200/test/test?pretty&refresh" -d '{
  "title": "An other title"
}'

curl -XGET "http://localhost:9200/test/test/_search?pretty" -d '{
  "suggest": {
    "text": "tetsting sugestion",
    "phrase":{
      "phrase":{
        "field": "title",
        "max_errors": 5
      }
    }
  }
}'

I keep this issue opened to see if we can not fail in case of empty shards.

@mathieu007
Copy link
Author

Ah , yeah your right, i understand now why i received this error, it make sense...
Thanks for pointing me the errors in my script, will remember that for next time.

Math

@nik9000
Copy link
Member

nik9000 commented Aug 9, 2013

I've been noticing that suggestions fail on empty shards problem for the past few days. If no one gets to it the in next few days I'll have a crack at fixing the errors on empty shards.

@dadoonet
Copy link
Member

dadoonet commented Aug 9, 2013

Closing this one as there is now issue #3473 and PR #3475 relative to it.

@dadoonet dadoonet closed this as completed Aug 9, 2013
dadoonet added a commit to dadoonet/elasticsearch that referenced this issue Aug 16, 2013
From elastic#3469.
When running suggest on empty shards, it raises an error like:

```
"failures" : [ {
      "status" : 400,
      "reason" : "ElasticSearchIllegalArgumentException[generator field [title] doesn't exist]"
    } ]
```

We should ignore empty shards.

Closes elastic#3473.
dadoonet added a commit that referenced this issue Aug 16, 2013
From #3469.
When running suggest on empty shards, it raises an error like:

```
"failures" : [ {
      "status" : 400,
      "reason" : "ElasticSearchIllegalArgumentException[generator field [title] doesn't exist]"
    } ]
```

We should ignore empty shards.

Closes #3473.
dadoonet added a commit that referenced this issue Aug 16, 2013
From #3469.
When running suggest on empty shards, it raises an error like:

```
"failures" : [ {
      "status" : 400,
      "reason" : "ElasticSearchIllegalArgumentException[generator field [title] doesn't exist]"
    } ]
```

We should ignore empty shards.

Closes #3473.
s1monw added a commit to s1monw/elasticsearch that referenced this issue Aug 16, 2013
Unless the field is not mapped phrase suggester should return
empty results or skip candidate generation if a field in not in
the index rather than failing hard with an illegal argument exception.
Some shards might not have a value in a certain field.

Closes elastic#3469
@s1monw
Copy link
Contributor

s1monw commented Aug 16, 2013

reopening since this is really a different issue than having an entirely empty shard. here we can have no value in a field rather than having no docs.

@s1monw s1monw reopened this Aug 16, 2013
s1monw added a commit to s1monw/elasticsearch that referenced this issue Aug 16, 2013
Unless the field is not mapped phrase suggester should return
empty results or skip candidate generation if a field in not in
the index rather than failing hard with an illegal argument exception.
Some shards might not have a value in a certain field.

Closes elastic#3469
@s1monw s1monw closed this as completed in 57c0d29 Aug 16, 2013
s1monw added a commit that referenced this issue Aug 16, 2013
Unless the field is not mapped phrase suggester should return
empty results or skip candidate generation if a field in not in
the index rather than failing hard with an illegal argument exception.
Some shards might not have a value in a certain field.

Closes #3469
@ghost ghost assigned s1monw Aug 16, 2013
mute pushed a commit to mute/elasticsearch that referenced this issue Jul 29, 2015
From elastic#3469.
When running suggest on empty shards, it raises an error like:

```
"failures" : [ {
      "status" : 400,
      "reason" : "ElasticSearchIllegalArgumentException[generator field [title] doesn't exist]"
    } ]
```

We should ignore empty shards.

Closes elastic#3473.
mute pushed a commit to mute/elasticsearch that referenced this issue Jul 29, 2015
Unless the field is not mapped phrase suggester should return
empty results or skip candidate generation if a field in not in
the index rather than failing hard with an illegal argument exception.
Some shards might not have a value in a certain field.

Closes elastic#3469
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants