Feature Proposal: declare index aliases in the schema definition API

[markyang-toast]

Summary

ElasticGraph generates an aliases key into datastore_config.yaml for both indices and index
templates, but always leaves it empty, and there's no schema definition API for it. Proposing one:

 schema.object_type "Campaign" do |t|
   t.field "id", "ID!"
   t.field "createdAt", "DateTime"
   t.field "status", "String"

   t.index "campaigns" do |i|
     i.rollover :monthly, "createdAt"
 
     i.alias "campaigns_read"
     i.alias "campaigns_active", filter: {term: {status: "ACTIVE"}}
   end
 end

Use cases are the usual ones: a stable name across a reindex cutover, a single queryable name in front
of rollover indices for consumers that read the index directly rather than through GraphQL, and
filtered aliases for coarse read scoping.

The problem

Indexing::Index hardcodes "aliases" => {} in both emit paths:

def to_index_config              # indexing/index.rb:245 — concrete index
  {"aliases" => {}, "mappings" => mappings, "settings" => settings}.compact
end

def to_index_template_config     # indexing/index.rb:256 — index template
  {"index_patterns" => [...], "template" => {"aliases" => {}, ...}}
end

Because it's emitted rather than omitted, an alias created outside ElasticGraph doesn't survive:

• Template. ForIndexTemplate#put_index_template PUTs the whole template, so a template alias is
  removed on the next mapping or settings change.
• New concrete index. ForIndex#create_new_index passes aliases: {}, so newly created indices
  never get aliases — including rollover indices, since ForIndexTemplate configures those through
  ForIndex using the template config.
• Existing concrete index. configure! reconciles settings and mappings only. Aliases are never
  inspected, so a missing one is never repaired.

So alias state can't be expressed in the schema, can't be recreated on a clean cluster, and silently
regresses after a routine clusters:configure.

Proposal

1. i.alias <name>, optionally with filter:, populating the aliases key in both
to_index_config and to_index_template_config. Because ForIndexTemplate passes the template config
down to ForIndex for related rollover indices, one declaration covers the template, indices created
by admin, and indices auto-created by the datastore from the template.

2. Reconcile aliases on indices that already exist, via _aliases in ForIndex#configure! —
currently the only surface with no path to pick up a newly declared alias.

3. Absent means "leave alone", not "remove all." elasticgraph-admin adds and updates declared
aliases and never removes one it didn't declare. This also fixes the clobbering above regardless of
whether anyone adopts the new API, so it may be worth landing on its own first.

Low blast radius: ElasticGraph never writes through an alias — the indexer computes concrete rollover
index names, and related_rollover_indices uses cat indices, which doesn't return aliases. Read path
only, so is_write_index never comes up.

[markyang-toast]

An alternative is to have this alias set in environment-specific YAML. But I think this interface is simpler, and I don't see a need for it to be env-specific yet.

[myronmarston]

Thanks for the thorough writeup, @markyang-toast.

One nuance to flag: aliases on existing concrete indices are never clobbered (configure! only touches settings/mappings on them), so the silent-regression risk is specific to index templates and the rollover indices created from them.

Taking a step back

ElasticGraph itself does nothing with aliases — it never writes through them (the indexer computes concrete rollover index names) and never queries through them. A dedicated i.alias API would suggest EG uses aliases in some manner, when really it would just be emitting config into datastore_config.yaml for the datastore's benefit.

Your use case — clients querying OpenSearch directly — isn't really about aliases specifically; it's "configure the datastore in ways EG doesn't model." Aliases are one instance; field aliases, ISM policies, and future ES/OS features are others. I'd rather solve that class of need once, with a general-purpose customization hook.

This also keeps the door open for a first-class alias API later, if and when EG actually does something with aliases (e.g. querying or writing through them). If we shipped i.alias today with "EG ignores this, apart from dumping it in datastore_config.yaml" semantics, giving the API real behavior later would be a breaking change for anyone relying on the ignore semantics. A generic hook carries no such commitment.

Proposal: i.customize_config

schema.object_type "Campaign" do |t|
  # ...
  t.index "campaigns" do |i|
    i.rollover :monthly, "createdAt"

    i.customize_config do |config|
      config["aliases"] = {
        "campaigns_read" => {},
        "campaigns_active" => {"filter" => {"term" => {"status" => "ACTIVE"}}}
      }
    end
  end
end

Semantics:

• The block mutates the yielded config hash; the return value is ignored. EG yields a defensive deep copy, so mutations can't corrupt EG's internal structures.
• The yielded hash is "the config every concrete index for this index definition gets" — the template body for rollover index defs, the index config otherwise. You don't have to care about the indices vs index_templates artifact nesting.
• Mutation (rather than returning an updated copy) is deliberate: nested customizations like a field alias stay trivial —

  config["mappings"]["properties"]["created"] = {"type" => "alias", "path" => "createdAt"}

• The result is baked into datastore_config.yaml, so it's diffable in PRs and elasticgraph-admin treats it identically to built-in config.

How this covers your alias use cases

A template's aliases isn't an alias "on the template" — the template is a factory, and every index created from it gets stamped with those aliases. So campaigns_read ends up pointing at every rollover index, and GET campaigns_read/_search fans out across all of them — the "single queryable name in front of rollover indices" case. Filtered aliases work the same way, with the filter applied per index at query time. Admin's two index-creation paths already agree here: datastore auto-creation applies the template's aliases, and admin-created rollover indices are configured from the same template body.

Low blast radius: ElasticGraph never writes through an alias — the indexer computes concrete rollover
index names, and related_rollover_indices uses cat indices, which doesn't return aliases. Read path
only, so is_write_index never comes up.

Regarding is_write_index: it doesn't really make sense to use is_write_index with this (is_write_index needs to be set on one specific index, not an entire family of rollover indices), and as you said, EG won't write via an alias so there's not much point to trying that. We could validate/reject is_write_index but I'd rather not: the point of the customization hook is to apply whatever arbitrary customizations a user wants. EG just passes it through and makes no claim to the safety or correctness of what occurs--that's up to the user. (We'd probably want to document this: e.g. "use with care!").

Why index-level rather than a global hook

ES/OS has cluster-level features (cluster settings, ISM policies, ingest pipelines, component templates), but elasticgraph-admin only manages indices, index templates, and EG's own scripts — a global hook could emit artifact content that nothing applies, which is worse than no feature (the YAML would look authoritative). An index-level hook covers everything admin actually manages; a global hook can come later if a concrete need plus admin support for it shows up.

Changes needed for admin to fully respect customizations

Customized config already flows through admin untouched on create_index and put_index_template (the normalizer and merges only rewrite settings/mappings). Three fixes make it fully respected:

1. Your item 3, generalized: absent means "leave alone." On template updates, merge desired aliases over the template's current ones instead of replacing — the same policy admin already applies to no-longer-desired mapping fields. This fixes the out-of-band clobbering you found regardless of whether anyone uses the new API, so it's worth landing first as a standalone change.
2. Template update trigger. ForIndexTemplate#configure! only PUTs when mappings or settings changed — a customization-only change would never reach the datastore. Broaden the trigger to diff the full desired vs. current template config.
3. Your item 2, generalized: complete reconciliation of existing concrete indices. Without it, a newly declared alias only lands on indices created afterward — with a year of monthly rollover indices, campaigns_read would silently miss all history. An index config body has exactly three top-level keys — settings, mappings, aliases — and the datastore forces a per-surface update API for each (_settings, _mapping, _aliases); there's no generic "PUT full index config" on a live index. ForIndex#configure! already reconciles the first two, so adding an _aliases step (add/update desired aliases, never remove undeclared ones) completes coverage of every surface the datastore allows updating in place. Nested customizations (like field aliases) ride the existing mappings/settings reconciliation for free. Note this isn't alias-feature support leaking into EG: admin's contract is "converge the cluster to datastore_config.yaml," and per-surface code lives there only because the datastore's API shape demands it.

Would this work for you? If there's a reason you'd still want a purpose-built i.alias (validation, discoverability), I'm open to hearing it — but I lean towards a general-purpose hook as I think it fits ElasticGraph better.

[markyang-toast]

I think this makes more sense than doing a purposely built feature.
I will create a PR with this proposed API shape.

[myronmarston]

Implemented in #1366.