Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

search filter issue #41108

Closed
wlsaud619 opened this Issue Apr 11, 2019 · 5 comments

Comments

Projects
None yet
4 participants
@wlsaud619
Copy link

commented Apr 11, 2019

Describe the feature:

Elasticsearch version (bin/elasticsearch --version):
Version: 7.0.0-rc2, Build: default/tar/f076a79/2019-04-02T17:20:40.380908Z, JVM: 1.8.0_201

Plugins installed: []

JVM version (java -version):
java version "1.8.0_201"
Java(TM) SE Runtime Environment (build 1.8.0_201-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode

OS version (uname -a if on a Unix-like system):
Linux ip-10-20-30-101 4.15.0-1035-aws #37-Ubuntu SMP Mon Mar 18 16:15:14 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Description of the problem including expected versus actual behavior:

  1. "stats" : { "stats" : { "field" : "val_commissionProfitPerBlock_atom" } }
  2. "stats" : { "stats" : { "field" : "val_votingPower(%)" } }
  3. "stats" : { "stats" : { "field" : "val_commissionProfitPerBlock(ateam)_atom" } }

In cases 1 and 2, the value is retrieved.
However, in case of 3, the value exists but is not retrieved.

Steps to reproduce:

Please include a minimal but complete recreation of the problem, including
(e.g.) index creation, mappings, settings, query etc. The easier you make for
us to reproduce it, the more likely that somebody will take the time to look at it.

curl -s -XGET 'localhost:9200/cosmos_mainnet_validator_*/_search?&pretty' -H 'Content-Type: application/json; charset=utf-8' -d '
{
  "size": 1,
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "@timestamp": {
              "from": "2019-04-10 18:00:01",
              "to": "2019-04-11 19:00:00",
              "time_zone": "Asia/Seoul",
              "format": "yyyy-MM-dd HH:mm:ss"
            }
          }
        }
      ]
    }
  },
  "aggs" : {
    "stats" : { "stats" : { "field" : "**val_commissionProfitPerBlock(ateam)_atom**" } }
  }
}'

Provide logs (if relevant):

{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 398,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "cosmos_mainnet_validator_2019-04-11",
        "_type" : "validator",
        "_id" : "1554970508128230798",
        "_score" : 1.0,
        "_source" : {
          "@timestamp" : 1554970505000,
          "block" : 357547,
          "val_minSelfDelegation" : 5000,
          "val_commissionRate" : 9.9,
          "val_currentCommissionProfit_atom" : 440.05425493687,
          "val_commissionProfitPerBlock_atom" : 0.0036971945048662747,
          "val_commissionProfitDaily_atom" : 0.011091580265201628,
          "val_votingPower" : 884328,
          "val_votingPower(%)" : 0.73,
          "val_delegationAmount" : 884328.165456,
          "del_count" : 16,
          "val_jailStatus" : 0,
          "val_proposingStatus" : 0,
          "val_proposerPriority" : 32,
          "val_currentReward_atom" : 46.973781051226304,
          **"val_commissionProfitPerBlock(ateam)_atom" : 0.0015661428787581144,**
          "val_precommitStatus" : 1
        }
      }
    ]
  },
  "aggregations" : {
    "stats" : {
      "count" : 398,
      "min" : 0.0,
      "max" : 0.0,
      "avg" : 0.0,
      "sum" : 0.0
    }
  }
}
@elasticmachine

This comment has been minimized.

Copy link

commented Apr 11, 2019

@polyfractal

This comment has been minimized.

Copy link
Member

commented Apr 11, 2019

Can you provide the mapping of the index?

It might be that val_commissionProfitPerBlock(ateam)_atom has been mapped as a long or some other non-float type, and so values like 0.0015661428787581144 are being rounded to zero. Which would make stats aggs on them look like zero min/max/avg, etc.

@wlsaud619

This comment has been minimized.

Copy link
Author

commented Apr 16, 2019

Can you provide the mapping of the index?

It might be that val_commissionProfitPerBlock(ateam)_atom has been mapped as a long or some other non-float type, and so values like 0.0015661428787581144 are being rounded to zero. Which would make stats aggs on them look like zero min/max/avg, etc.

"val_commissionProfitPerBlock(ateam)_atom" : {"type" : "float"},

@polyfractal

This comment has been minimized.

Copy link
Member

commented Apr 18, 2019

I see that your search request is querying multiple indices (cosmos_mainnet_validator_*). Can you confirm that all the indices have the same mappings? It might be a mismatch between mapping types across indices, causing the field to be "downgraded" to the lowest compatible type.

I tried to replicate your issue but the aggregations work just fine in this example:

PUT /test/
{
  "mappings": {
    "properties": {
      "@timestamp": {
        "type": "date"
      },
      "val_commissionProfitPerBlock(ateam)_atom": {
        "type": "float"
      }
    }
  }
}

PUT test/_doc/1
{
  "@timestamp": 1554970505000,
  "block": 357547,
  "val_minSelfDelegation": 5000,
  "val_commissionRate": 9.9,
  "val_currentCommissionProfit_atom": 440.05425493687,
  "val_commissionProfitPerBlock_atom": 0.0036971945048662747,
  "val_commissionProfitDaily_atom": 0.011091580265201628,
  "val_votingPower": 884328,
  "val_votingPower(%)": 0.73,
  "val_delegationAmount": 884328.165456,
  "del_count": 16,
  "val_jailStatus": 0,
  "val_proposingStatus": 0,
  "val_proposerPriority": 32,
  "val_currentReward_atom": 46.973781051226304,
  "val_commissionProfitPerBlock(ateam)_atom": 0.0015661428787581144,
  "val_precommitStatus": 1
}

GET /test/_search
{
  "size": 1,
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "@timestamp": {
              "from": "2019-04-10 18:00:01",
              "to": "2019-04-11 19:00:00",
              "time_zone": "Asia/Seoul",
              "format": "yyyy-MM-dd HH:mm:ss"
            }
          }
        }
      ]
    }
  },
  "aggs": {
    "stats": {
      "stats": {
        "field": "val_commissionProfitPerBlock(ateam)_atom"
      }
    }
  }
}

Yielding the response:

{
  "took" : 17,
  "timed_out" : false,
  "_shards" : {...},
  "hits" : {
    "total" : {...},
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "test",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "@timestamp" : 1554970505000,
          "block" : 357547,
          "val_minSelfDelegation" : 5000,
          "val_commissionRate" : 9.9,
          "val_currentCommissionProfit_atom" : 440.05425493687,
          "val_commissionProfitPerBlock_atom" : 0.0036971945048662747,
          "val_commissionProfitDaily_atom" : 0.011091580265201628,
          "val_votingPower" : 884328,
          "val_votingPower(%)" : 0.73,
          "val_delegationAmount" : 884328.165456,
          "del_count" : 16,
          "val_jailStatus" : 0,
          "val_proposingStatus" : 0,
          "val_proposerPriority" : 32,
          "val_currentReward_atom" : 46.973781051226304,
          "val_commissionProfitPerBlock(ateam)_atom" : 0.0015661428787581144,
          "val_precommitStatus" : 1
        }
      }
    ]
  },
  "aggregations" : {
    "stats" : {
      "count" : 1,
      "min" : 0.0015661428915336728,
      "max" : 0.0015661428915336728,
      "avg" : 0.0015661428915336728,
      "sum" : 0.0015661428915336728
    }
  }
@wlsaud619

This comment has been minimized.

Copy link
Author

commented Apr 22, 2019

The index name was shortened so it was fixed.
If issues occur again later, raise issues.
Thanks. :)

@wlsaud619 wlsaud619 closed this Apr 22, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.