Table Index Config settings not preserved when migrating to fieldConfigList

[clarahong]

When setting table index configs, Pinot documentation recommends deprecating usage of single-column index configurations in favor of the Field Config List.

However, when attempting to migrate an existing table from the deprecated format, these settings are not respected. For example, using an old table config, dictionary indexes are disabled:

{
  ...
    "tableIndexConfig": {
      "noDictionaryColumns": [
        "col1",
        “range1”,
      ],
      ...
      "optimizeDictionaryForMetrics": true,
      "optimizeDictionary": false,
      "noDictionarySizeRatioThreshold": 0.85
    },
  }

Updating to a new definition:

{
...
  "fieldConfigList": [
    {
      "name": "range1",
      "indexes": {
        "dictionary": {
          "disabled": true
        },
        "range": {}
      }
    },
    {
      "name": "col1",
      "indexes": {
        "dictionary": {
          "disabled": true
        },
        "json": {
          "maxLevels": 1,
          "excludeArray": true,
          "disableCrossArrayUnnest": true
        }
      }
    }
  ]
}

Making the table config change and triggering a table reload creates new dictionary indexes even though they are specified as disabled. Adding the property "encodingType": "RAW" also does not resolve this issue.

[Jackie-Jiang]

Thanks for reporting the issue. I believe this is a bug and should be fixed with #14258. Can you try with the latest master?

[clarahong]

Thanks for addressing. We'll test out the changes on the next release with 1.3.0 and report back with any issues.