Skip to content

Conversation

@kkrik-es
Copy link
Contributor

Follow-up on #115397, where it was suggested to merge the two providers, to avoid inter-dependencies for provided settings and simplify the logic.

@kkrik-es kkrik-es self-assigned this Dec 10, 2024
@kkrik-es kkrik-es added auto-backport Automatically create backport pull requests when merged v8.18.0 labels Dec 10, 2024
@kkrik-es kkrik-es marked this pull request as ready for review December 10, 2024 16:00
@kkrik-es kkrik-es requested a review from martijnvg December 10, 2024 16:00
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

Copy link
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

But I think there is an issue if the new index settings provider returns index.mode setting and user template uses index.mode setting as well. In that case we would always overrule index.mode=logsdb, while the user may have opted out of logsdb?

return Settings.builder().put("index.mode", IndexMode.LOGSDB.getName()).build();
settingsBuilder = Settings.builder().put(IndexSettings.MODE.getKey(), IndexMode.LOGSDB.getName());
if (supportSyntheticSourceDemotion()) {
indexTemplateAndCreateRequestSettings = Settings.builder()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also remove index.mode from settingsBuilder if indexTemplateAndCreateRequestSettings contains index.mode setting? I think that is an simpler approach than compared to injecting index.mode=logsdb into indexTemplateAndCreateRequestSettings

Otherwise we would enforce index.mode=logsdb even if user template define index.mode?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is copied from:

https://github.com/elastic/elasticsearch/blob/main/x-pack/plugin/logsdb/src/main/java/org/elasticsearch/xpack/logsdb/SyntheticSourceIndexSettingsProvider.java#L75

I suggest we keep this as pure refactoring, we can improve the logic separately.

Btw, don't we check for index.mode in settings in line 84 above?

createdIndexVersion.set(indexVersion);
}

private boolean supportSyntheticSourceDemotion() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe rename to supportFallbackToStoredSource()? Which is similar to how it has been named before.


@Override
public boolean overrulesTemplateAndRequestSettings() {
// Indicates that the provider value takes precedence over any user setting.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This means that both index.mode and index.mapping.source.mode will be overruling if returned by this settings provider.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, this is a change. I believe index mode is injected only if it's not present in incoming settings, so there is no delta?

@kkrik-es
Copy link
Contributor Author

But I think there is an issue if the new index settings provider returns index.mode setting and user template uses index.mode setting as well. In that case we would always overrule index.mode=logsdb, while the user may have opted out of logsdb?

I think the two settings are injected independently, there should be no change in the logic wrt when each gets applied. More so, index.mode is only injected if it's not present in the incoming settings, so the fact that this is an overruling provider shouldn't change much.

@martijnvg
Copy link
Member

More so, index.mode is only injected if it's not present in the incoming settings, so the fact that this is an overruling provider shouldn't change much.

Ah right, I missed that this: && resolveIndexMode(settings.get(IndexSettings.MODE.getKey())) == null is checked before index.mode setting is injected.

Copy link
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kkrik-es kkrik-es enabled auto-merge (squash) December 11, 2024 14:38
@kkrik-es kkrik-es merged commit c792595 into elastic:main Dec 11, 2024
16 checks passed
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
8.x Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 118342

@kkrik-es
Copy link
Contributor Author

💚 All backports created successfully

Status Branch Result
8.x

Questions ?

Please refer to the Backport tool documentation

kkrik-es added a commit to kkrik-es/elasticsearch that referenced this pull request Dec 11, 2024
* Unify logsdb index settings providers

* restore diff

* rename method

(cherry picked from commit c792595)

# Conflicts:
#	x-pack/plugin/logsdb/src/main/java/org/elasticsearch/xpack/logsdb/LogsDBPlugin.java
elasticsearchmachine pushed a commit that referenced this pull request Dec 11, 2024
* Unify logsdb index settings providers (#118342)

* Unify logsdb index settings providers

* restore diff

* rename method

(cherry picked from commit c792595)

# Conflicts:
#	x-pack/plugin/logsdb/src/main/java/org/elasticsearch/xpack/logsdb/LogsDBPlugin.java

* Update LogsDBPlugin.java
@kkrik-es kkrik-es deleted the fix/logsdb-settings-provider branch January 7, 2025 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Automatically create backport pull requests when merged backport pending >non-issue :StorageEngine/Logs You know, for Logs Team:StorageEngine v8.18.0 v9.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants