Solr 6 to Solr 7 missed measurements #4713

Aminolroaya · 2018-09-18T09:32:52Z

Hi. We are migrating from Solr 6 to Solr 7; However, In Solr 7, we only can access to four measurements of solr_admin, solr_core, solr_cache and other measures like solr_queryhandler and solr_updatehandler are gone. Is the limitation from the telegraph side?

danielnelson · 2018-09-18T21:16:44Z

It sounds likely this is a limitation in the Telegraf plugin, can you adjust the URL and grab the output of this command:

curl 'http://localhost:8983/solr/main/admin/mbeans?stats=true&wt=json&cat=CORE&cat=QUERYHANDLER&cat=UPDATEHANDLER&cat=CACHE'

Aminolroaya · 2018-09-22T05:51:55Z

Thanks Daniel. As you see the query and update handlers' outputs are null as the output in follows for a collection;

[{
  "responseHeader":{
    "status":0,
    "QTime":5},
  "solr-mbeans":[
    "CORE",{
      "searcher":{
        "class":"org.apache.solr.search.SolrIndexSearcher",
        "description":"index searcher",
        "stats":{
          "SEARCHER.searcher.maxDoc":4950860,
          "SEARCHER.searcher.caching":true,
          "SEARCHER.searcher.registeredAt":"2018-09-22T05:36:46.913Z",
          "SEARCHER.searcher.deletedDocs":721459,
          "SEARCHER.searcher.warmupTime":0,
          "SEARCHER.searcher.searcherName":"Searcher@5b32b9fd[twitter_shard1_replica_n5] main",
          "SEARCHER.searcher.indexVersion":7315750,
          "SEARCHER.searcher.openedAt":"2018-09-22T05:36:46.912Z",
          "SEARCHER.searcher.readerDir":"NRTCachingDirectory(MMapDirectory@/home/solr/data/twitter_shard1_replica_n5/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b4c887f; maxCacheMB=48.0 maxMergeSizeMB=4.0)",
          "SEARCHER.searcher.numDocs":4229401,
          "SEARCHER.searcher.reader":"ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_kbdu(7.4.0):C909240/138664:delGen=30039) Uninverting(_myz6(7.4.0):C959377/135819:delGen=29888) Uninverting(_qeat(7.4.0):C624397/107470:delGen=17260) Uninverting(_slfk(7.4.0):C1078185/131792:delGen=17628) Uninverting(_udzj(7.4.0):C421094/34810:delGen=6215) Uninverting(_t9b0(7.4.0):c181203/54749:delGen=8518) Uninverting(_tkjt(7.4.0):c108062/28798:delGen=5675) Uninverting(_u8fj(7.4.0):c244276/70359:delGen=10307) Uninverting(_uqlm(7.4.0):c66974/3403:delGen=1482) Uninverting(_ulid(7.4.0):c56568/4521:delGen=1698) Uninverting(_vhgt(7.4.0):c174569/4961:delGen=1596) Uninverting(_vhgj(7.4.0):c7085/938:delGen=501) Uninverting(_vjlu(7.4.0):c28884/1943:delGen=711) Uninverting(_vkso(7.4.0):c38832/1929:delGen=492) Uninverting(_vluk(7.4.0):c39142/904:delGen=131) Uninverting(_vm2c(7.4.0):c1130/29:delGen=13) Uninverting(_vlw4(7.4.0):c2078/109:delGen=50) Uninverting(_vm3g(7.4.0):c2714/33:delGen=7) Uninverting(_vm16(7.4.0):c1312/39:delGen=22) Uninverting(_vm2w(7.4.0):c1506/31:delGen=13) Uninverting(_vm36(7.4.0):c1172/21:delGen=8) Uninverting(_vm21(7.4.0):c1253/27:delGen=13) Uninverting(_vm3h(7.4.0):C261/17:delGen=3) Uninverting(_vm3i(7.4.0):C242/17:delGen=2) Uninverting(_vm3j(7.4.0):C268/13:delGen=3) Uninverting(_vm3k(7.4.0):C240/12:delGen=4) Uninverting(_vm3l(7.4.0):C267/19:delGen=3) Uninverting(_vm3m(7.4.0):C245/7:delGen=2) Uninverting(_vm3n(7.4.0):C284/25:delGen=1)))"}},
      "core":{
        "class":"twitter_shard1_replica_n5",
        "description":"SolrCore",
        "stats":{
          "CORE.fs.totalSpace":799542874112,
          "CORE.fs.path":"/home/solr/data/twitter_shard1_replica_n5/data",
          "CORE.coreName":"twitter_shard1_replica_n5",
          "CORE.indexDir":"/home/solr/data/twitter_shard1_replica_n5/data/index/",
          "INDEX.sizeInBytes":2504885718,
          "CORE.shard":"shard1",
          "SEARCHER.new.errors":537,
          "CORE.refCount":2,
          "CORE.collection":"twitter",
          "INDEX.size":"2.33 GB",
          "CORE.startTime":"2018-09-12T07:34:44.591Z",
          "SEARCHER.new":1529918,
          "CORE.fs.usableSpace":767805669376,
          "CORE.aliases":["twitter_shard1_replica_n5"],
          "SEARCHER.new.maxReached":1,
          "CORE.instanceDir":"/home/solr/data/twitter_shard1_replica_n5",
          "SEARCHER.new.time.meanRate":0.93942060425354,
          "SEARCHER.new.warmup.meanRate":0.8479559856587974,
          "CORE.fs.spins":false}},
      "Searcher@5b32b9fd[twitter_shard1_replica_n5] main":{
        "class":"org.apache.solr.search.SolrIndexSearcher",
        "description":"index searcher",
        "stats":{
          "SEARCHER.searcher.maxDoc":4950860,
          "SEARCHER.searcher.caching":true,
          "SEARCHER.searcher.registeredAt":"2018-09-22T05:36:46.913Z",
          "SEARCHER.searcher.deletedDocs":721459,
          "SEARCHER.searcher.warmupTime":0,
          "SEARCHER.searcher.searcherName":"Searcher@5b32b9fd[twitter_shard1_replica_n5] main",
          "SEARCHER.searcher.indexVersion":7315750,
          "SEARCHER.searcher.openedAt":"2018-09-22T05:36:46.912Z",
          "SEARCHER.searcher.readerDir":"NRTCachingDirectory(MMapDirectory@/home/solr/data/twitter_shard1_replica_n5/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b4c887f; maxCacheMB=48.0 maxMergeSizeMB=4.0)",
          "SEARCHER.searcher.numDocs":4229401,
          "SEARCHER.searcher.reader":"ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_kbdu(7.4.0):C909240/138664:delGen=30039) Uninverting(_myz6(7.4.0):C959377/135819:delGen=29888) Uninverting(_qeat(7.4.0):C624397/107470:delGen=17260) Uninverting(_slfk(7.4.0):C1078185/131792:delGen=17628) Uninverting(_udzj(7.4.0):C421094/34810:delGen=6215) Uninverting(_t9b0(7.4.0):c181203/54749:delGen=8518) Uninverting(_tkjt(7.4.0):c108062/28798:delGen=5675) Uninverting(_u8fj(7.4.0):c244276/70359:delGen=10307) Uninverting(_uqlm(7.4.0):c66974/3403:delGen=1482) Uninverting(_ulid(7.4.0):c56568/4521:delGen=1698) Uninverting(_vhgt(7.4.0):c174569/4961:delGen=1596) Uninverting(_vhgj(7.4.0):c7085/938:delGen=501) Uninverting(_vjlu(7.4.0):c28884/1943:delGen=711) Uninverting(_vkso(7.4.0):c38832/1929:delGen=492) Uninverting(_vluk(7.4.0):c39142/904:delGen=131) Uninverting(_vm2c(7.4.0):c1130/29:delGen=13) Uninverting(_vlw4(7.4.0):c2078/109:delGen=50) Uninverting(_vm3g(7.4.0):c2714/33:delGen=7) Uninverting(_vm16(7.4.0):c1312/39:delGen=22) Uninverting(_vm2w(7.4.0):c1506/31:delGen=13) Uninverting(_vm36(7.4.0):c1172/21:delGen=8) Uninverting(_vm21(7.4.0):c1253/27:delGen=13) Uninverting(_vm3h(7.4.0):C261/17:delGen=3) Uninverting(_vm3i(7.4.0):C242/17:delGen=2) Uninverting(_vm3j(7.4.0):C268/13:delGen=3) Uninverting(_vm3k(7.4.0):C240/12:delGen=4) Uninverting(_vm3l(7.4.0):C267/19:delGen=3) Uninverting(_vm3m(7.4.0):C245/7:delGen=2) Uninverting(_vm3n(7.4.0):C284/25:delGen=1)))"}}},
    "QUERYHANDLER",{},
    "UPDATEHANDLER",{},
    "CACHE",{
      "perSegFilter":{
        "class":"org.apache.solr.search.LRUCache",
        "description":"LRU Cache(maxSize=10, initialSize=0, autowarmCount=10, regenerator=org.apache.solr.search.NoOpRegenerator@561b2eac)",
        "stats":{
          "CACHE.searcher.perSegFilter.lookups":0,
          "CACHE.searcher.perSegFilter.hits":0,
          "CACHE.searcher.perSegFilter.hitratio":0.0,
          "CACHE.searcher.perSegFilter.warmupTime":0,
          "CACHE.searcher.perSegFilter.cumulative_lookups":0,
          "CACHE.searcher.perSegFilter.cumulative_inserts":0,
          "CACHE.searcher.perSegFilter.cumulative_hitratio":0.0,
          "CACHE.searcher.perSegFilter.size":0,
          "CACHE.searcher.perSegFilter.evictions":0,
          "CACHE.searcher.perSegFilter.cumulative_hits":0,
          "CACHE.searcher.perSegFilter.cumulative_evictions":0,
          "CACHE.searcher.perSegFilter.inserts":0}},
      "queryResultCache":{
        "class":"org.apache.solr.search.LRUCache",
        "description":"LRU Cache(maxSize=512, initialSize=512)",
        "stats":{
          "CACHE.searcher.queryResultCache.cumulative_inserts":6,
          "CACHE.searcher.queryResultCache.evictions":0,
          "CACHE.searcher.queryResultCache.lookups":0,
          "CACHE.searcher.queryResultCache.cumulative_lookups":7,
          "CACHE.searcher.queryResultCache.cumulative_evictions":0,
          "CACHE.searcher.queryResultCache.warmupTime":0,
          "CACHE.searcher.queryResultCache.inserts":0,
          "CACHE.searcher.queryResultCache.hitratio":0.0,
          "CACHE.searcher.queryResultCache.size":0,
          "CACHE.searcher.queryResultCache.cumulative_hitratio":0.0,
          "CACHE.searcher.queryResultCache.cumulative_hits":0,
          "CACHE.searcher.queryResultCache.hits":0}},
      "fieldValueCache":{
        "class":"org.apache.solr.search.FastLRUCache",
        "description":"Concurrent LRU Cache(maxSize=10000, initialSize=10, minSize=9000, acceptableSize=9500, cleanupThread=false)",
        "stats":{
          "CACHE.searcher.fieldValueCache.warmupTime":0,
          "CACHE.searcher.fieldValueCache.inserts":0,
          "CACHE.searcher.fieldValueCache.hits":0,
          "CACHE.searcher.fieldValueCache.lookups":0,
          "CACHE.searcher.fieldValueCache.cumulative_hitratio":0.0,
          "CACHE.searcher.fieldValueCache.hitratio":0.0,
          "CACHE.searcher.fieldValueCache.cumulative_hits":0,
          "CACHE.searcher.fieldValueCache.size":0,
          "CACHE.searcher.fieldValueCache.evictions":0,
          "CACHE.searcher.fieldValueCache.cumulative_lookups":0,
          "CACHE.searcher.fieldValueCache.cumulative_inserts":0,
          "CACHE.searcher.fieldValueCache.cumulative_evictions":0}},
      "fieldCache":{
        "class":"org.apache.solr.search.SolrFieldCacheBean",
        "description":"Provides introspection of the Solr FieldCache ",
        "stats":{
          "CACHE.core.fieldCache.entries_count":0,
          "CACHE.core.fieldCache.total_size":"0 bytes"}},
      "filterCache":{
        "class":"org.apache.solr.search.FastLRUCache",
        "description":"Concurrent LRU Cache(maxSize=512, initialSize=512, minSize=460, acceptableSize=486, cleanupThread=false)",
        "stats":{
          "CACHE.searcher.filterCache.hits":0,
          "CACHE.searcher.filterCache.cumulative_evictions":0,
          "CACHE.searcher.filterCache.cumulative_inserts":0,
          "CACHE.searcher.filterCache.size":0,
          "CACHE.searcher.filterCache.hitratio":0.0,
          "CACHE.searcher.filterCache.warmupTime":0,
          "CACHE.searcher.filterCache.inserts":0,
          "CACHE.searcher.filterCache.evictions":0,
          "CACHE.searcher.filterCache.cumulative_hitratio":0.0,
          "CACHE.searcher.filterCache.lookups":0,
          "CACHE.searcher.filterCache.cumulative_lookups":0,
          "CACHE.searcher.filterCache.cumulative_hits":0}},
      "documentCache":{
        "class":"org.apache.solr.search.LRUCache",
        "description":"LRU Cache(maxSize=512, initialSize=512)",
        "stats":{
          "CACHE.searcher.documentCache.evictions":0,
          "CACHE.searcher.documentCache.cumulative_lookups":10120,
          "CACHE.searcher.documentCache.hitratio":0.0,
          "CACHE.searcher.documentCache.size":0,
          "CACHE.searcher.documentCache.cumulative_hitratio":0.01,
          "CACHE.searcher.documentCache.lookups":0,
          "CACHE.searcher.documentCache.warmupTime":0,
          "CACHE.searcher.documentCache.inserts":0,
          "CACHE.searcher.documentCache.hits":0,
          "CACHE.searcher.documentCache.cumulative_hits":92,
          "CACHE.searcher.documentCache.cumulative_inserts":10028,
          "CACHE.searcher.documentCache.cumulative_evictions":9456}}}]}]

I think the api should change for solr 7. Am I right?

maxiko · 2020-07-12T09:55:13Z

I stuck with same problem. I have Solr 7.1
metrics like QUERYHANDLER and UPDATEHANDLER
http://localhost:8983/solr/admin/metrics?group=core&prefix=UPDATE./update.requestTimes
http://localhost:8983/solr/admin/metrics?group=core&prefix=QUERY./select.requestTimes

anthosz · 2021-09-13T09:35:06Z

Hi,

Same issue with Solr 7.7

Metrics like QUERYHANDLER and UPDATEHANDLER cannot be found via telegraf but works via direct request.

srebhan · 2023-07-12T15:19:55Z

@anthosz can you please provide some data sample from a Solr 7.7 instance?

anthosz · 2023-07-12T15:38:12Z

@srebhan
Example on Solr 8:

"metrics":{
  "solr.core.CORE":{
    "QUERY./select.requestTimes":{
      "count":1714321,
      "meanRate":0.9549196918356667,
      "1minRate":2.9766355274754592,
      "5minRate":2.895530516865424,
      "15minRate":2.7859878768994792,
      "min_ms":0.095462,
      "max_ms":494.960079,
      "mean_ms":4.825921795836523,
      "median_ms":0.311355,
      "stddev_ms":28.638890787218546,
      "p75_ms":0.434746,
      "p95_ms":15.25432,
      "p99_ms":243.489748
    },
    "UPDATE./update.requestTimes":{
      "count":1167237,
      "meanRate":0.6501998597537556,
      "1minRate":0.37396404902535996,
      "5minRate":0.518798270163239,
      "15minRate":0.5802406362958978,
      "min_ms":0.153042,
      "max_ms":357.006608,
      "mean_ms":33.465509097435145,
      "median_ms":5.081386,
      "stddev_ms":57.80858101085427,
      "p75_ms":7.27542,
      "p95_ms":158.273004,
      "p99_ms":165.341841,
      "p999_ms":252.302073
    }
  }
}

EDIT: I took the previous query from @maxiko
I will add all metrics available tomorrow (I guess you can focus now on Solr 8+, solr 7 is deprecated)

srebhan · 2023-07-12T20:17:30Z

If you can also provide the queries necessary to get the information that would be awesome!

anthosz · 2023-07-13T08:00:02Z

If you can also provide the queries necessary to get the information that would be awesome!

So:

Query for updatehandler:
curl 'http://localhost:8983/solr/admin/metrics?group=core&prefix=UPDATE.updateHandler'
Query for queryhandler (not sure for this one, it will be global metrics for QUERY but cannot found anymore stuff related to queryHandler):
curl 'http://localhost:8983/solr/admin/metrics?group=core&prefix=QUERY'

srebhan · 2023-07-26T18:47:12Z

Thanks @anthosz!

srebhan · 2023-07-27T14:10:54Z

@anthosz sorry for bothering you again. I would need the output of

curl 'http://localhost:8983/solr/admin/metrics?wt=json'

and

curl 'http://localhost:8983/solr/admin/info/system?wt=json'

As I think the former will give us all metrics reliable since v6.6 and the latter will allow us to determine the version automatically... Please make sure you redact any sensitive information.

Furthermore, I will use your data in unit-tests. Hope that's OK!?

srebhan · 2023-07-31T17:02:24Z

@Aminolroaya and @anthosz please try the binary in PR #13699 which should add support for version 7+. Let me know if this fixes the issue!

anthosz · 2023-08-01T08:36:37Z

@Aminolroaya and @anthosz please try the binary in PR #13699 which should add support for version 7+. Let me know if this fixes the issue!

Thank you @srebhan !

I confirm that solr_queryhandler are now present in the output with the binary mentioned in the PR!

Do you always want the json output of solr?

srebhan · 2023-08-01T09:48:28Z

@anthosz thanks for testing!

Regarding JSON output, it's fine, I don't need the JSON as I now have an integration test against docker images for the different versions. Just a heads-up, I pushed another version of the PR that now also reports all available update-handlers (not just "updateHandler"). If you could test again that would be much appreciated!

anthosz · 2023-08-01T12:16:21Z

@anthosz thanks for testing!

Regarding JSON output, it's fine, I don't need the JSON as I now have an integration test against docker images for the different versions. Just a heads-up, I pushed another version of the PR that now also reports all available update-handlers (not just "updateHandler"). If you could test again that would be much appreciated!

Confirmed (not sure it's relevant but tested with solr 8)!

solr_admin
solr_cache
solr_core
solr_queryhandler
solr_updatehandler

danielnelson added the bug unexpected problem or unintended behavior label Sep 18, 2018

anthosz mentioned this issue Sep 13, 2021

input solr 8 support #7264

Closed

srebhan added the waiting for response waiting for response from contributor label Jul 12, 2023

srebhan self-assigned this Jul 12, 2023

telegraf-tiger bot removed the waiting for response waiting for response from contributor label Jul 12, 2023

srebhan added the plugin/input 1. Request for new input plugins 2. Issues/PRs that are related to input plugins label Jul 26, 2023

This was referenced Jul 28, 2023

test(inputs.solr): Rework testing #13692

Merged

feat(inputs.solr): Support version 7.x to 9.3 #13699

Merged

srebhan closed this as completed in #13699 Aug 2, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Solr 6 to Solr 7 missed measurements #4713

Solr 6 to Solr 7 missed measurements #4713

Aminolroaya commented Sep 18, 2018

danielnelson commented Sep 18, 2018

Aminolroaya commented Sep 22, 2018

maxiko commented Jul 12, 2020 •

edited

Loading

anthosz commented Sep 13, 2021

srebhan commented Jul 12, 2023

anthosz commented Jul 12, 2023 •

edited

Loading

srebhan commented Jul 12, 2023

anthosz commented Jul 13, 2023 •

edited

Loading

srebhan commented Jul 26, 2023

srebhan commented Jul 27, 2023

srebhan commented Jul 31, 2023

anthosz commented Aug 1, 2023

srebhan commented Aug 1, 2023

anthosz commented Aug 1, 2023 •

edited

Loading

Solr 6 to Solr 7 missed measurements #4713

Solr 6 to Solr 7 missed measurements #4713

Comments

Aminolroaya commented Sep 18, 2018

danielnelson commented Sep 18, 2018

Aminolroaya commented Sep 22, 2018

maxiko commented Jul 12, 2020 • edited Loading

anthosz commented Sep 13, 2021

srebhan commented Jul 12, 2023

anthosz commented Jul 12, 2023 • edited Loading

srebhan commented Jul 12, 2023

anthosz commented Jul 13, 2023 • edited Loading

srebhan commented Jul 26, 2023

srebhan commented Jul 27, 2023

srebhan commented Jul 31, 2023

anthosz commented Aug 1, 2023

srebhan commented Aug 1, 2023

anthosz commented Aug 1, 2023 • edited Loading

maxiko commented Jul 12, 2020 •

edited

Loading

anthosz commented Jul 12, 2023 •

edited

Loading

anthosz commented Jul 13, 2023 •

edited

Loading

anthosz commented Aug 1, 2023 •

edited

Loading