Enable point based sort optimization for all custom comparators

[gashutos]

As a part of #6321 and #6424 , we enabled Lucene's numeric sort optimization which are based on BKD point values skipping logic.
We can enable this for our custom comparator as well. Which were disabled while upgrading to Lucene 9.1.0 in this PR 2487.

There is no harm adding this back, queries like below in nyc_taxis are showing 15x improvement on scaled_float numeric types. total_amount is scaled_float.

GET nyc_taxis/_search?size=1000
{
  "sort": [
    {
      "total_amount": {
        "order": "asc"
      }
    }
  ]
}

Edit : Above query takes 1350~ ms without optimization, 75~ ms with optimization.

[anasalkouz]

Hi @gashutos, could you share the performance testing results?

[reta]

#8168

[gashutos]

@anasalkouz In OSB, we dont have such numeric types queries where it use custom comparators.
But if you trigger asc/desc order sort queries on nyc_taxis like mentioned below, without this optimization it takes 1300/1400 ms, while with this optimization, it came down to 80 ms.

GET nyc_taxis/_search?size=1000
{
  "sort": [
    {
      "total_amount": {
        "order": "asc"
      }
    }
  ]
}

[reta]

@anasalkouz In OSB, we dont have such numeric types queries where it use custom comparators.

@gashutos @anasalkouz we could actually come up with the relevant benchmarks since we have unsigned_long now, that implements custom comparator, added opensearch-project/opensearch-benchmark-workloads#85

[reta]

Fixed by #8168