Reject range index on ingestion-aggregated no-dictionary columns

[raghavyadav01]

Problem

Configuring a range index on an ingestion-aggregated metric column causes segments to fail to commit.

Ingestion-time metrics aggregation (aggregationConfigs, or the legacy aggregateMetrics flag) forces its aggregated metric columns to be no-dictionary, and their min/max values are never tracked while the consuming segment is being built. The BitSliced range index (version 2) creator reads min/max for a single-value no-dictionary column:

// RangeIndexType.createIndexCreator
return new BitSlicedRangeIndexCreator(context.getIndexDir(), fieldSpec,
    context.getMinValue(), context.getMaxValue());   // min/max are null here

For INT/LONG columns this throws a NullPointerException at segment build time (((Number) minValue).longValue()); for FLOAT/DOUBLE it silently builds a degenerate index over a wrong value domain. Either way, the affected real-time segments can never commit.

Fix

1. RangeIndexType.validate() — reject a version-2 range index on a single-value, no-dictionary, ingestion-aggregated column upfront, with an actionable message. Scoped to REALTIME tables, since ingestion aggregation only materializes on consuming segments (the offline path computes min/max normally and is unaffected). Version-1 range index does not read min/max and is left allowed.

2. BitSlicedRangeIndexCreator — defense-in-depth: the raw-column constructor now fails with a clear IllegalStateException when min/max are absent, instead of a bare NullPointerException. This also covers direct/programmatic creation and segment reload paths that config validation does not gate.

Testing

• IndexCombinationValidationTest — both aggregation paths are rejected for real-time tables; version-1 range index passes; a non-aggregated no-dictionary numeric column still supports the range index; the same config on an offline table is allowed.
• BitSlicedIndexCreatorTest — added a test that the raw-column constructor fails loudly on absent min/max.

Release Notes

Configuring a version-2 (BitSliced) range index on an ingestion-aggregated no-dictionary column is now rejected at table-config validation time. Use range index version 1, or remove the range index on such columns.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 72.22222% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.57%. Comparing base (81b82b2) to head (2ae1fdf).
⚠️ Report is 28 commits behind head on master.

Files with missing lines	Patch %	Lines
...ment/local/segment/index/range/RangeIndexType.java	71.42%	0 Missing and 4 partials ⚠️
...t/creator/impl/inv/BitSlicedRangeIndexCreator.java	75.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #19111      +/-   ##
============================================
+ Coverage     65.50%   65.57%   +0.06%     
- Complexity     1421     1423       +2     
============================================
  Files          3430     3432       +2     
  Lines        218000   218392     +392     
  Branches      34642    34749     +107     
============================================
+ Hits         142799   143208     +409     
+ Misses        63641    63614      -27     
- Partials      11560    11570      +10     

Flag	Coverage Δ
custom-integration1	100.00% <ø> (ø)
integration	100.00% <ø> (ø)
integration1	100.00% <ø> (ø)
integration2	0.00% <ø> (ø)
java-25	65.57% <72.22%> (+0.06%)	⬆️
temurin	65.57% <72.22%> (+0.06%)	⬆️
unittests	65.57% <72.22%> (+0.06%)	⬆️
unittests1	56.94% <22.22%> (+0.14%)	⬆️
unittests2	37.90% <72.22%> (+0.03%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Jackie-Jiang]

This is not the correct fix. We should allow building range index, and scan the column to figure out min/max value when they are unavailable.

[raghavyadav01]

Thanks @Jackie-Jiang .
Will the following approach be ok ?

I am thinking to compute min/max on demand in MutableNoDictColumnStatistics — when the mutable segment reports null (aggregated columns skip min/max tracking since the value mutates), scan the sealed forward index once (same pass isSorted() already does), before the BitSliced creator is constructed.

This adds one extra O(numDocs) scan per such column at seal time. Will this be OK?

[Jackie-Jiang]

I am thinking to compute min/max on demand in MutableNoDictColumnStatistics — when the mutable segment reports null (aggregated columns skip min/max tracking since the value mutates), scan the sealed forward index once (same pass isSorted() already does), before the BitSliced creator is constructed.

This adds one extra O(numDocs) scan per such column at seal time. Will this be OK?

Yes. That is the overhead we need to pay in order to add range index. We should also check how RangeIndexHandler handles null min/max value.