feat: enable Calcite engine for Prometheus datasources with filter pushdown

[robertpaschedag]

Description

Enables all Calcite-only PPL commands (join, lookup, flatten, expand, eventstats, etc.) to work with Prometheus datasources by making PrometheusMetricTable implement Calcite's TranslatableTable interface with a custom logical/physical scan operator and filter pushdown.

Key changes:

• PrometheusMetricTable now extends AbstractTable and implements both TranslatableTable (Calcite) and Table (V2), preserving backward compatibility
• CalciteLogicalPrometheusScan — custom logical scan node (Convention.NONE) with filter pushdown support for label matchers and time ranges
• CalciteEnumerablePrometheusScan — physical scan (EnumerableConvention) that executes PromQL via PrometheusClient.queryRange()
• PrometheusFilterPushDownRule — Calcite optimizer rule that pushes label equality and time range filters into the scan
• OpenSearchSchema — dynamic DataSourceSubSchema resolution for multi-part table references (e.g., source = prometheus.metric_name)
• CalciteRelNodeVisitor.visitRelation() — relaxed datasource guard to allow non-default datasources when their table implements Calcite's Table interface

Filter pushdown behavior:

Filter	Pushed?	Target
service = 'frontend'	Yes	PromQL label selector {service="frontend"}
@timestamp >= X AND @timestamp <= Y	Yes	Query range start/end parameters
@value > 0.5	No	In-memory EnumerableCalc
service = 'frontend' AND @value > 0.5	Partial	Label pushed, @value remains in-memory

Explain output examples:

Full pushdown:
CalciteEnumerablePrometheusScan(table=[OpenSearch, prometheus, request], PushDownContext=[LABELS->{service=frontend}])

Partial pushdown:
EnumerableCalc(condition=>($t4, 0.5))
CalciteEnumerablePrometheusScan(table=[OpenSearch, prometheus, request], PushDownContext=[LABELS->{service=frontend}])

Related Issues

Resolves #5705

Check List

• New functionality includes testing.
• New functionality has been documented.
  • New functionality has javadoc added.
  • New functionality has a user manual doc added.
• New PPL command checklist all confirmed.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff or -s.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Yes..... the code got generated with assistence of AI. I was able to do some example PPL queries in local dev environment, using otel-demo sendings logs to opensearch, while sending metrics to prometheus (victoriametrics).

Screenshots

Example 1:

Example 2:

Example 3:

Example 4:

Example 5:

I generally hope, that this might be useful.

[github-actions]

PR Code Analyzer ❗

AI-powered 'Code-Diff-Analyzer' found issues on commit 0f0796a.

Path	Line	Severity	Description
prometheus/build.gradle	21	high	New dependency added: 'org.immutables:value-annotations:2.8.8'. Per mandatory supply chain review policy, all dependency additions must be flagged for maintainer verification regardless of apparent legitimacy. Namespace hijacking and typosquatting are common attack techniques.
prometheus/src/main/java/org/opensearch/sql/prometheus/planner/logical/rules/PrometheusPushDownContext.java	115	medium	In buildPromQL(), the metricName parameter is interpolated directly into the PromQL string without any escaping (only label values are escaped via escapePromQLLabelValue). A metric name containing PromQL special characters such as '{', '}', or '#' could result in malformed or injected PromQL queries if the metric name originates from user-controlled input.
core/src/main/java/org/opensearch/sql/calcite/OpenSearchSchema.java	103	low	DataSourceSubSchema.resolveTable() hardcodes 'default' as the schema name when calling getTable(new DataSourceSchemaName(dataSourceName, "default"), tableName). For datasources with multiple schemas, this bypasses schema-level resolution and could cause tables to be resolved in the wrong schema context, potentially sidestepping schema-based authorization checks depending on downstream storage engine implementations.

The table above displays the top 10 most important findings.

Total: 3 | Critical: 0 | High: 1 | Medium: 1 | Low: 1

---

Pull Requests Author(s): Please update your Pull Request according to the report above.

Repository Maintainer(s): You can bypass diff analyzer by adding label skip-diff-analyzer after reviewing the changes carefully, then re-run failed actions. To re-enable the analyzer, remove the label, then re-run all actions.

---

⚠️ Note: The Code-Diff-Analyzer helps protect against potentially harmful code patterns. Please ensure you have thoroughly reviewed the changes beforehand.

Thanks.

[robertpaschedag]

I will try to tackle the code analyzer comments....