Skip to content

Emit metric dimension for minor compaction and other refactors#19151

Merged
gianm merged 7 commits intoapache:masterfrom
kfaraz:minor_compact_metric
Mar 16, 2026
Merged

Emit metric dimension for minor compaction and other refactors#19151
gianm merged 7 commits intoapache:masterfrom
kfaraz:minor_compact_metric

Conversation

@kfaraz
Copy link
Contributor

@kfaraz kfaraz commented Mar 13, 2026 •
edited
Loading

Description

#19059 introduced MSQ-based minor compaction using compaction supervisors.
Once minor compaction becomes widely adopted, it would be useful to be able to easily distinguish between minor and major compaction jobs.

Changes

  • Add compactionMode and compactionPolicyResult to compaction task context.
  • Use prefix coordinator-issued-minor for minor compaction tasks
  • Emit compaction mode for compact/task/count metric
  • Add ClientCompactionInputSpec to mirror CompactionInputSpec
  • Add new impl ClientMinorCompactionInputSpec to mirror MinorCompactionInputSpec
  • Simplify ClientCompactionIntervalSpec
  • Move Eligibility into a separate class

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

@kfaraz kfaraz requested a review from cecemei March 13, 2026 11:24
gianm
gianm reviewed Mar 13, 2026
View reviewed changes
server/src/main/java/org/apache/druid/client/indexing/ClientCompactionIntervalSpec.java Outdated
}

@JsonProperty
public String getType()
Copy link
Contributor

@gianm gianm Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is now going to conflict with the type field added by @JsonTypeInfo. I think you can remove it.

Copy link
Contributor Author

@kfaraz kfaraz Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

server/src/main/java/org/apache/druid/server/compaction/Eligibility.java Outdated

/**
* @return {@code Eligibility} denoting that the candidate interval is
* eligibile for a {@link CompactionMode#UNCOMPACTED_SEGMENTS_ONLY minor}
Copy link
Contributor

@gianm gianm Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eligible (spelling)

Copy link
Contributor Author

@kfaraz kfaraz Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@kfaraz kfaraz requested a review from gianm March 16, 2026 04:23
@kfaraz kfaraz mentioned this pull request Mar 16, 2026
Open
10 tasks
@gianm gianm merged commit 226f277 into apache:master Mar 16, 2026
37 checks passed
@github-actions github-actions bot added this to the 37.0.0 milestone Mar 16, 2026
@kfaraz kfaraz deleted the minor_compact_metric branch March 16, 2026 17:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gianm gianm gianm approved these changes

@cecemei cecemei Awaiting requested review from cecemei

Assignees

No one assigned

Labels

Area - Ingestion

Projects

None yet

Milestone

37.0.0

Development

Successfully merging this pull request may close these issues.

2 participants

@kfaraz @gianm