New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Date Math Not Working for GET #13665

Closed
sirdavidhuang opened this Issue Sep 20, 2015 · 5 comments

Comments

Projects
None yet
6 participants
@sirdavidhuang

sirdavidhuang commented Sep 20, 2015

I try to get a document using date math, but Elasticsearch returns a error.

Version: Elasticsearch 2.0.0-beta2

Put Document

$ curl -XPUT 'localhost:9200/logstash-2015.09.19/animals/1' -d '{"name":"puppy"}'
{"_index":"logstash-2015.09.19","_type":"animals","_id":"1","_version":1,"_shards":{"total":2,"successful":1,"failed":0},"created":true}

Get Document

$ curl -XGET 'localhost:9200/logstash-2015.09.19/animals/1'
{"_index":"logstash-2015.09.19","_type":"animals","_id":"1","_version":1,"found":true,"_source":{"name":"puppy"}}

Get Document with Date Math

$ curl -XGET 'localhost:9200/<logstash-{now/d}>/animals/1'
No handler found for uri [/<logstash-now/d>/animals/1] and method [GET]

@dadoonet

This comment has been minimized.

Show comment
Hide comment
@dadoonet

dadoonet Sep 20, 2015

Member

It's not supposed to work AFAIK. Did you see that somewhere in docs?
You should do that on the client side.

May be using an alias would help you?

Member

dadoonet commented Sep 20, 2015

It's not supposed to work AFAIK. Did you see that somewhere in docs?
You should do that on the client side.

May be using an alias would help you?

@sirdavidhuang

This comment has been minimized.

Show comment
Hide comment
@sirdavidhuang

sirdavidhuang Sep 20, 2015

This page includes the date math information.
https://www.elastic.co/guide/en/elasticsearch/reference/master/date-math-index-names.html

Example from page.
curl -XGET 'localhost:9200/<logstash-{now/d-2d}>/_search' {
"query" : {
...
}
}

I try this curl command and get a error.
curl -XGET 'localhost:9200/<logstash-{now/d}>/_search' -d '{
"query" : {
"match" : {}
}
}'

{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"<logstash-now","index":"<logstash-now"}],"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"<logstash-now","index":"<logstash-now"},"status":404}

There is also this pull request.

Add date math support in index names #12209
#12209

sirdavidhuang commented Sep 20, 2015

This page includes the date math information.
https://www.elastic.co/guide/en/elasticsearch/reference/master/date-math-index-names.html

Example from page.
curl -XGET 'localhost:9200/<logstash-{now/d-2d}>/_search' {
"query" : {
...
}
}

I try this curl command and get a error.
curl -XGET 'localhost:9200/<logstash-{now/d}>/_search' -d '{
"query" : {
"match" : {}
}
}'

{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"<logstash-now","index":"<logstash-now"}],"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"<logstash-now","index":"<logstash-now"},"status":404}

There is also this pull request.

Add date math support in index names #12209
#12209

@dadoonet

This comment has been minimized.

Show comment
Hide comment
@dadoonet

dadoonet Sep 20, 2015

Member

Great. I missed that !

Member

dadoonet commented Sep 20, 2015

Great. I missed that !

@martijnvg

This comment has been minimized.

Show comment
Hide comment
@martijnvg

martijnvg Sep 21, 2015

Member

Thanks for reporting @sirdavidhuang, this is indeed a bug. The issue here is that the index name is cut of at the date rounding (/) and only <logstash-{now ends up being received as index name.

Member

martijnvg commented Sep 21, 2015

Thanks for reporting @sirdavidhuang, this is indeed a bug. The issue here is that the index name is cut of at the date rounding (/) and only <logstash-{now ends up being received as index name.

@javanna

This comment has been minimized.

Show comment
Hide comment
@javanna

javanna Oct 20, 2015

Member

Hi @sirdavidhuang we had initially treated this as a bug and fixed it, but afterwards we found out that the fix introduced a regression (#14177), which made us take a step back. Any slash that should not be used as a path separator in a uri should be properly escaped, and it is wrong to try and make distinctions between the different slashes on the server side depending on what surrounds them, that heuristic is not going to fly (in fact it didn't :) )

We are going to revert the initial fix then, the solution to this problem is to escape the '/' in the url like this: curl -XGET 'localhost:9200/<logstash-{now%2Fd}>/animals/1'. That is going to work and didn't require any fix in the first place.

Member

javanna commented Oct 20, 2015

Hi @sirdavidhuang we had initially treated this as a bug and fixed it, but afterwards we found out that the fix introduced a regression (#14177), which made us take a step back. Any slash that should not be used as a path separator in a uri should be properly escaped, and it is wrong to try and make distinctions between the different slashes on the server side depending on what surrounds them, that heuristic is not going to fly (in fact it didn't :) )

We are going to revert the initial fix then, the solution to this problem is to escape the '/' in the url like this: curl -XGET 'localhost:9200/<logstash-{now%2Fd}>/animals/1'. That is going to work and didn't require any fix in the first place.

javanna added a commit to javanna/elasticsearch that referenced this issue Oct 21, 2015

Restore support for escaped '/' as part of document id
With #13691 we introduced some custom logic to make sure that date math expressions like <logstash-{now/D}> don't get broken up into two where the slash appears in the expression. That said the only correct way to provide such a date math expression as part of the uri would be to properly escape the '/' instead. This fix also introduced a regression, as it would make sure that unescaped '/' are handled only in the case of date math expressions, but it removed support for properly escaped slashes anywhere else. The solution is to keep supporting escaped slashes only and require client libraries to properly escape them.

This commit reverts 93ad696 and makes sure that our REST tests runner supports escaping of path parts, which was more involving than expected as each single part of the path needs to be properly escaped. I am not too happy with the current solution but it's the best I could do for now, maybe not that concerning anyway given that it's just test code. I do find uri encoding quite frustrating in java.

Relates to #13691
Relates to #13665

Closes #14177
Closes #14216

javanna added a commit to javanna/elasticsearch that referenced this issue Oct 21, 2015

Restore support for escaped '/' as part of document id
With #13691 we introduced some custom logic to make sure that date math expressions like <logstash-{now/D}> don't get broken up into two where the slash appears in the expression. That said the only correct way to provide such a date math expression as part of the uri would be to properly escape the '/' instead. This fix also introduced a regression, as it would make sure that unescaped '/' are handled only in the case of date math expressions, but it removed support for properly escaped slashes anywhere else. The solution is to keep supporting escaped slashes only and require client libraries to properly escape them.

This commit reverts 93ad696 and makes sure that our REST tests runner supports escaping of path parts, which was more involving than expected as each single part of the path needs to be properly escaped. I am not too happy with the current solution but it's the best I could do for now, maybe not that concerning anyway given that it's just test code. I do find uri encoding quite frustrating in java.

Relates to #13691
Relates to #13665

Closes #14177
Closes #14216

javanna added a commit to javanna/elasticsearch that referenced this issue Oct 21, 2015

Restore support for escaped '/' as part of document id
With #13691 we introduced some custom logic to make sure that date math expressions like <logstash-{now/D}> don't get broken up into two where the slash appears in the expression. That said the only correct way to provide such a date math expression as part of the uri would be to properly escape the '/' instead. This fix also introduced a regression, as it would make sure that unescaped '/' are handled only in the case of date math expressions, but it removed support for properly escaped slashes anywhere else. The solution is to keep supporting escaped slashes only and require client libraries to properly escape them.

This commit reverts 93ad696 and makes sure that our REST tests runner supports escaping of path parts, which was more involving than expected as each single part of the path needs to be properly escaped. I am not too happy with the current solution but it's the best I could do for now, maybe not that concerning anyway given that it's just test code. I do find uri encoding quite frustrating in java.

Relates to #13691
Relates to #13665

Closes #14177
Closes #14216

javanna added a commit to javanna/elasticsearch that referenced this issue Oct 21, 2015

Restore support for escaped '/' as part of document id
With #13691 we introduced some custom logic to make sure that date math expressions like <logstash-{now/D}> don't get broken up into two where the slash appears in the expression. That said the only correct way to provide such a date math expression as part of the uri would be to properly escape the '/' instead. This fix also introduced a regression, as it would make sure that unescaped '/' are handled only in the case of date math expressions, but it removed support for properly escaped slashes anywhere else. The solution is to keep supporting escaped slashes only and require client libraries to properly escape them.

This commit reverts 93ad696 and makes sure that our REST tests runner supports escaping of path parts, which was more involving than expected as each single part of the path needs to be properly escaped. I am not too happy with the current solution but it's the best I could do for now, maybe not that concerning anyway given that it's just test code. I do find uri encoding quite frustrating in java.

Relates to #13691
Relates to #13665

Closes #14177
Closes #14216
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment