Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Limited settings for ILM policy created via file-based settings #94465

Closed
thbkrkr opened this issue Mar 10, 2023 · 2 comments · Fixed by #94477
Closed

Limited settings for ILM policy created via file-based settings #94465

thbkrkr opened this issue Mar 10, 2023 · 2 comments · Fixed by #94477
Assignees
@williamrandolph
Labels
>bug :Core/Infra/Settings Settings infrastructure and APIs Team:Core/Infra Meta label for core/infra team

Comments

@thbkrkr
Copy link
Contributor

thbkrkr commented Mar 10, 2023 •
edited
Loading

Elasticsearch Version

8.6.2

Installed Plugins

No response

Java Version

bundled

OS Version

Linux 5.10.133+ #1 SMP Thu Aug 18 14:49:28 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Problem Description

Cannot define an ILM policy with frozen.actions.searchable_snapshot or hot.actions.set_priority using the file-based settings.

It works using the API:

PUT _ilm/policy/my_policy
{
  "policy": {
    "phases": {
      "delete": {
        "actions": {
          "delete": {}
        },
        "min_age": "14d"
      },
      "frozen": {
        "actions": {
          "searchable_snapshot": {
            "force_merge_index": true,
            "snapshot_repository": "my_fs_backup"
          }
        },
        "min_age": "8d"
      },
      "hot": {
        "actions": {
          "rollover": {
            "max_age": "1d",
            "max_primary_shard_size": "50gb"
          },
          "set_priority": {
            "priority": 100
          }
        },
        "min_age": "0ms"
      },
      "warm": {
        "actions": {},
        "min_age": "1d"
      }
    }
  }
}

But it fails using this file-based settings:

# /usr/share/elasticsearch/config/operator/settings.json
{
  "metadata": {
    "version": "1678372780726094000",
    "compatibility": "8.6.1"
  },
  "state": {
    "cluster_settings": {},
    "snapshot_repositories": {},
    "slm": {},
    "role_mappings": {},
    "ilm": {
      "my_policy": {
        "phases": {
          "delete": {
            "actions": {
              "delete": {}
            },
            "min_age": "14d"
          },
          "frozen": {
            "actions": {
              "searchable_snapshot": {
                "force_merge_index": true,
                "snapshot_repository": "my_fs_backup"
              }
            },
            "min_age": "8d"
          },
          "hot": {
            "actions": {
              "rollover": {
                "max_age": "1d",
                "max_primary_shard_size": "50gb"
              },
              "set_priority": {
                "priority": 100
              }
            },
            "min_age": "0ms"
          },
          "warm": {
            "actions": {},
            "min_age": "1d"
          }
        }
      }
    },
    "ingest_pipelines": {},
    "index_templates": {
      "component_templates": {},
      "composable_index_templates": {}
    }
  }
}

Message:  java.lang.IllegalStateException: Error processing state change request for file_settings, errors: org.elasticsearch.xcontent.XContentParseException: [1:553] [reserved_state_chunk] failed to parse field [state]
Caused by: org.elasticsearch.xcontent.XContentParseException: [1:553] [state] failed to parse field [ilm]
Caused by: org.elasticsearch.xcontent.XContentParseException: [1:46] [lifecycle_policy] failed to parse field [phases]
Caused by: org.elasticsearch.xcontent.XContentParseException: [1:46] [phases] failed to parse field [warm]
Caused by: org.elasticsearch.xcontent.XContentParseException: [1:46] [phase] failed to parse field [actions]
Caused by: org.elasticsearch.xcontent.XContentParseException: [1:46] [actions] failed to parse field [set_priority]
Caused by: org.elasticsearch.xcontent.NamedObjectNotFoundException: [1:46] unknown field [set_priority]

Steps to Reproduce

Start Elasticsearch with a file config/operators/settings.json containing:

{
  "metadata": {
    "version": "1",
    "compatibility": "8.6.2"
  },
  "state": {
    "ilm": {
      "my_timeseries_lifecycle": {
        "phases": {
          "warm": {
            "actions": {
              "readonly" : { }
            }
          }
        }
      }
    }
  }
}

Logs (if relevant)

{
    "@timestamp": "2023-03-10T14:43:52.924Z",
    "log.level": "ERROR",
    "message": "Error processing operator settings json file",
    "ecs.version": "1.2.0",
    "service.name": "ES_ECS",
    "event.dataset": "elasticsearch.server",
    "process.thread.name": "elasticsearch[file-settings-watcher]",
    "log.logger": "org.elasticsearch.reservedstate.service.FileSettingsService",
    "elasticsearch.cluster.uuid": "zCjM0LVOSoq-PuPHag1H8Q",
    "elasticsearch.node.id": "QEmQS0GeRgalpl8ehz1FLg",
    "elasticsearch.node.name": "wxyz-es-default-0",
    "elasticsearch.cluster.name": "wxyz",
    "error.type": "java.lang.IllegalStateException",
    "error.message": "Error processing state change request for file_settings, errors: java.lang.IllegalStateException: Error processing state change request for file_settings, errors: org.elasticsearch.xcontent.XContentParseException: [1:218] [reserved_state_chunk] failed to parse field [state]
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.throwFailedToParse(ObjectParser.java:616)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.parseValue(ObjectParser.java:611)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.parseSub(ObjectParser.java:629)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.parse(ObjectParser.java:315)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ConstructingObjectParser.parse(ConstructingObjectParser.java:166)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ConstructingObjectParser.apply(ConstructingObjectParser.java:158)
    	at org.elasticsearch.server@8.6.2/org.elasticsearch.reservedstate.service.ReservedClusterStateService.parse(ReservedClusterStateService.java:102)
    	at org.elasticsearch.server@8.6.2/org.elasticsearch.reservedstate.service.ReservedClusterStateService.process(ReservedClusterStateService.java:124)
    	at org.elasticsearch.server@8.6.2/org.elasticsearch.reservedstate.service.FileSettingsService.processFileSettings(FileSettingsService.java:411)
    	at org.elasticsearch.server@8.6.2/org.elasticsearch.reservedstate.service.FileSettingsService.processSettingsAndNotifyListeners(FileSettingsService.java:331)
    	at org.elasticsearch.server@8.6.2/org.elasticsearch.reservedstate.service.FileSettingsService.watcherThread(FileSettingsService.java:311)
    	at java.base/java.lang.Thread.run(Thread.java:1589)
    Caused by: org.elasticsearch.xcontent.XContentParseException: [1:218] [state] failed to parse field [ilm]
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.rethrowFieldParseFailure(ObjectParser.java:554)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.lambda$declareNamedObjects$14(ObjectParser.java:494)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.lambda$declareNamedObjects$15(ObjectParser.java:505)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.parseValue(ObjectParser.java:609)
    	... 10 more
    Caused by: org.elasticsearch.xcontent.XContentParseException: [1:31] [lifecycle_policy] failed to parse field [phases]
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.throwFailedToParse(ObjectParser.java:616)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.parseValue(ObjectParser.java:611)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.parseSub(ObjectParser.java:629)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.parse(ObjectParser.java:315)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ConstructingObjectParser.parse(ConstructingObjectParser.java:166)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ConstructingObjectParser.apply(ConstructingObjectParser.java:158)
    	at org.elasticsearch.xcore@8.6.2/org.elasticsearch.xpack.core.ilm.LifecyclePolicy.parse(LifecyclePolicy.java:134)
    	at org.elasticsearch.ilm@8.6.2/org.elasticsearch.xpack.ilm.action.ReservedLifecycleAction.fromXContent(ReservedLifecycleAction.java:116)
    	at org.elasticsearch.ilm@8.6.2/org.elasticsearch.xpack.ilm.action.ReservedLifecycleAction.fromXContent(ReservedLifecycleAction.java:40)
    	at org.elasticsearch.server@8.6.2/org.elasticsearch.reservedstate.service.ReservedClusterStateService.lambda$new$1(ReservedClusterStateService.java:95)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.lambda$declareNamedObjects$14(ObjectParser.java:492)
    	... 12 more
    Caused by: org.elasticsearch.xcontent.XContentParseException: [1:31] [phases] failed to parse field [warm]
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.rethrowFieldParseFailure(ObjectParser.java:554)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.lambda$declareNamedObjects$14(ObjectParser.java:494)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.lambda$declareNamedObjects$15(ObjectParser.java:505)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.parseValue(ObjectParser.java:609)
    	... 21 more
    Caused by: org.elasticsearch.xcontent.XContentParseException: [1:31] [phase] failed to parse field [actions]
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.throwFailedToParse(ObjectParser.java:616)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.parseValue(ObjectParser.java:611)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.parseSub(ObjectParser.java:629)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.parse(ObjectParser.java:315)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ConstructingObjectParser.parse(ConstructingObjectParser.java:166)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ConstructingObjectParser.apply(ConstructingObjectParser.java:158)
    	at org.elasticsearch.xcore@8.6.2/org.elasticsearch.xpack.core.ilm.Phase.parse(Phase.java:76)
    	at org.elasticsearch.xcore@8.6.2/org.elasticsearch.xpack.core.ilm.LifecyclePolicy.lambda$static$1(LifecyclePolicy.java:63)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.lambda$declareNamedObjects$14(ObjectParser.java:492)
    	... 23 more
    Caused by: org.elasticsearch.xcontent.XContentParseException: [1:31] [actions] failed to parse field [readonly]
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.rethrowFieldParseFailure(ObjectParser.java:554)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.lambda$declareNamedObjects$14(ObjectParser.java:494)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.lambda$declareNamedObjects$15(ObjectParser.java:505)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.parseValue(ObjectParser.java:609)
    	... 30 more
    Caused by: org.elasticsearch.xcontent.NamedObjectNotFoundException: [1:31] unknown field [readonly]
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.NamedXContentRegistry.lookupParser(NamedXContentRegistry.java:162)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.NamedXContentRegistry.parseNamedObject(NamedXContentRegistry.java:146)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.support.AbstractXContentParser.namedObject(AbstractXContentParser.java:414)
    	at org.elasticsearch.xcore@8.6.2/org.elasticsearch.xpack.core.ilm.Phase.lambda$static$2(Phase.java:69)
    	at org.elasticsearch.xcontent@8.6.2/org.elasticsearch.xcontent.ObjectParser.lambda$declareNamedObjects$14(ObjectParser.java:492)
    	... 32 more

    "
}
@thbkrkr thbkrkr added >bug :Core/Infra/Settings Settings infrastructure and APIs needs:triage Requires assignment of a team area label labels Mar 10, 2023
@elasticsearchmachine elasticsearchmachine added Team:Core/Infra Meta label for core/infra team and removed needs:triage Requires assignment of a team area label labels Mar 10, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@williamrandolph
Copy link
Contributor

I believe the problem here is that the file-based settings code uses a json parser with a named x-content registry that only provides parsers for four actions:

In ReservedLifecycleAction:

        var config = XContentParserConfiguration.EMPTY.withRegistry(LifecyclePolicyConfig.DEFAULT_X_CONTENT_REGISTRY);

The definition of this constant in LifecyclePolicyConfig:

    public static final NamedXContentRegistry DEFAULT_X_CONTENT_REGISTRY = new NamedXContentRegistry(
        List.of(
            new NamedXContentRegistry.Entry(
                LifecycleType.class,
                new ParseField(TimeseriesLifecycleType.TYPE),
                (p) -> TimeseriesLifecycleType.INSTANCE
            ),
            new NamedXContentRegistry.Entry(LifecycleAction.class, new ParseField(RolloverAction.NAME), RolloverAction::parse),
            new NamedXContentRegistry.Entry(LifecycleAction.class, new ParseField(ForceMergeAction.NAME), ForceMergeAction::parse),
            new NamedXContentRegistry.Entry(LifecycleAction.class, new ParseField(ShrinkAction.NAME), ShrinkAction::parse),
            new NamedXContentRegistry.Entry(LifecycleAction.class, new ParseField(DeleteAction.NAME), DeleteAction::parse)
        )
    );

It looks as if the purpose of LifecyclePolicyConfig is to provide enough ILM actions for certain built-in templates, so I think there are two possibilities that would allow the policies shown in the issue:

  1. Add the rest of the available ILM actions to LifecyclePolicyConfig
  2. Have ReservedLifecycleAction use the named content registry from the ILM plugin class.

If there is a reason we only want to provide operators with a subset of ILM functionality, it doesn't seem to be documented. The third option could be to leave things as-is, improve the error message, and add documentation.

@williamrandolph williamrandolph mentioned this issue Mar 10, 2023
Merged
@thbkrkr thbkrkr mentioned this issue Apr 13, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@williamrandolph williamrandolph

Labels
>bug :Core/Infra/Settings Settings infrastructure and APIs Team:Core/Infra Meta label for core/infra team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix parse failures for ILM operator settings williamrandolph/elasticsearch
3 participants
@thbkrkr @williamrandolph @elasticsearchmachine