[AWS Billing] Add fingerprint dimension #6749
Conversation
Signed-off-by: constanca-m <constanca.manteigas@elastic.co>
Signed-off-by: constanca-m <constanca.manteigas@elastic.co>
| @@ -85,6 +85,9 @@ | |||
| object_type: keyword | |||
| object_type_mapping_type: "*" | |||
| description: Cost explorer group by key values. | |||
| - name: group_by.fingerprint | |||
| type: keyword | |||
There was a problem hiding this comment.
do we actually need to index the fingerprint field?
There was a problem hiding this comment.
Yes, because we need to set the dimension as true.
There was a problem hiding this comment.
i was meaning to keep the mapping, but add index: false and doc_values: false to save disk space.
micro optimization here for sure, but we want to ensure we are applying best practices (and micro optimizations do add up).
There was a problem hiding this comment.
That is interesting, I was not aware of those options:
The index option controls whether field values are indexed. It accepts true or false and defaults to true.
If the fingerprint is not indexed, can it still be used to create the _tsid? What if there is overlapping in documents? If we don't have the fingerprint value, then we will have to search a document based on all fields used for the fingerprint. Is that a good idea?
doc_values: false
The same problem, what if we want to search a document based on the fingerprint?
There was a problem hiding this comment.
My main concern is that for TSDB testing we need all dimensions to be searchable.
There was a problem hiding this comment.
What if there is overlapping in documents?
How do you mean?
If the fingerprint is not indexed, can it still be used to create the _tsid?
I would guess so, but let's find out.
what if we want to search a document based on the fingerprint?
Why would we ever want to do that? we would have to know the content of the document beforehand in order to generate the hash to search.
There was a problem hiding this comment.
Why would we ever want to do that?
The fingerprint would be a dimension, which means that if documents overlap we will lose data. In that case we can find those documents based on the set of dimensions, including the fingerprint. That is why I think it is useful to be able to search for it.
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
🤖 GitHub commentsExpand to view the GitHub comments
To re-run your PR in the CI, just comment with:
|
|
/test |
🌐 Coverage report
|
|
Package aws - 1.45.9 containing this change is available at https://epr.elastic.co/search?package=aws |
What does this PR do?
Add dimension fingerprint for
group_byfield.Currently, the dimensions defined for AWS Billing are not enough as the documents keep being overwritten. The fingerprint field solves this problem.
Checklist
changelog.ymlfile.Related issues
Relates to #6293.