refactor: clarify architecture naming and unify planner workloads - #714
Conversation
|
已按审查完成命名迁移,并删除 LegacyMetricWorkload:注册与重规划统一使用 ASAPPlanner 的 QueryWorkload 和 DataWorkload。以下附英文流程图与完整中文报告,包含字段归属、重复判断、源码链接及兼容输入的迁移要求。 English architecture flowflowchart TB
subgraph CP[Control plane]
DTO["JSON / YAML compatibility input"]
NORMALIZE["Analyzer<br/>Validate and normalize once"]
WORKLOAD["ASAPPlanner QueryWorkload<br/>Query + requirements + recurrence + time selection"]
DATA["ASAPPlanner DataWorkload<br/>Arrival + rate + cardinality + evidence freshness"]
OPTIONS["DeploymentOptions<br/>Sketch constraint + retained labels + wire size + memory cap"]
REGISTRY["WorkloadStore<br/>Canonical workload + deployment options per metric and role"]
METRIC["Metric deployment planning and replanning<br/>Derive query metadata and fresh cost inputs"]
DTO --> NORMALIZE --> WORKLOAD
DATA --> WORKLOAD
WORKLOAD --> REGISTRY
OPTIONS --> REGISTRY
REGISTRY --> METRIC
INPUT["BackendLocalPlanningInput<br/>Workload demand + physical inputs + deployment context"]
LOGICAL["ASAPPlanner + selection adapter<br/>Legal semantic DAG selection"]
REQUEST["PhysicalCompilationRequest<br/>QueryCompilationInput + enabled materialization keys"]
WINDOWS["Generate window candidates<br/>Cadence + evaluation phase + WindowCostModel"]
COMPILE["PhysicalPlanCompiler<br/>Compile concrete candidate plans"]
MANIFEST["WorkloadCostManifest<br/>Component implementations and pricing basis"]
QUOTE["WorkloadQuote<br/>Provider feasibility and component prices"]
EVALUATE["CandidatePlanEvaluation<br/>Select the lowest-cost feasible enumerated candidate"]
SQL["ClickHouse SQL selection and binding<br/>Separate compilation path"]
PUBLICATION["PhysicalPlanPublication<br/>PhysicalPlanInstallRequest"]
WORKLOAD --> INPUT
INPUT --> LOGICAL --> REQUEST --> WINDOWS --> COMPILE --> MANIFEST
MANIFEST --> QUOTE --> EVALUATE --> PUBLICATION
SQL --> PUBLICATION
end
CONTRACT["Shared asap_types contracts<br/>SummaryCatalog + CollectorPlan + PrecomputePlan<br/>TransmissionPlan + QueryPlan + StorageRouting"]
subgraph DP[Data plane]
STAGE["Validate and stage RuntimePhysicalPlan"]
ACTIVE["Activate via ActivePhysicalPlanHandle<br/>One immutable generation for all readers"]
MAINTAIN["Ingest and precompute<br/>StreamingConfigHandle materialization view"]
QUERY["Execute installed query DAG<br/>Residual operators, bound reads, exact subqueries"]
STORE["Summary storage<br/>SummarySeriesMetadata + physical series state"]
STAGE --> ACTIVE
ACTIVE --> MAINTAIN --> STORE
ACTIVE --> QUERY --> STORE
end
PUBLICATION --> CONTRACT --> STAGE
PUBLICATION -->|OpAMP| COLLECTOR[ASAPCollector]
METRIC -->|Stage configuration via OpAMP| COLLECTOR
COLLECTOR -->|OTLP frames| MAINTAIN
RAW[Raw time-series samples] -->|Remote Write| MAINTAIN
QUERY -->|Exact execution or fallback| EXACT[Prometheus / VictoriaMetrics / ClickHouse]
完整中文架构审查与命名映射(含最终落地清单)ASAPQuery-backend 架构与命名审查审查基线:GitHub 结论:#709 应从“术语替换表”改为“组件边界和数据语义对齐”。优先澄清逻辑候选、物理候选、报价对象、部署状态和存储身份,再决定具体命名。不能因为几个对象都包含计划字段,就把它们合成一个类型;也不能因为都叫 本文件保留实施前的架构审查与决策依据。当前实现及迁移规则见 Planning terminology and architecture。其中明确列出此次命名迁移与保留为后续工作的结构调整。 追加实施:删除扁平 workload,统一 canonical 模型 注册模型、输入转换 和 注册存储 已改为直接使用 ASAPPlanner 的
迁移包含有意的入口行为变化:字段式 quantile 生成显式 0.99 分位查询,frequency 生成时间范围内的 item count;不再给已有查询硬塞 单 metric 注册接口拒绝不能无损投影的多 selector、非等值过滤、offset/@、子查询、 测试覆盖 canonical 查询与精度/周期/数据证据的存储往返、aggregate sample rate、过期/未知证据、cadence 范围检查及不支持的投影。英文流程架构图见 Planning terminology and architecture。下文中提出临时命名 本 PR 的最终落地与迁移方式 下表记录最终采用的名称;后文保留审查时的备选与理由。实施基线为
主分支集成补充:共享 pane 保留 cadence、phase 与 evaluation alignment;发布的 storage routing 从选中物理计划生成; 迁移规则:保留字段的 Rust 名称更新,输出 wire 名称保持原样;反序列化接受新名称作为 alias。旧公共类型导入及主要入口提供 deprecated 转发,但 Rust struct literal 的旧字段拼写无法通过类型别名兼容,源码消费者需要按表迁移。
一、组件职责与当前真实路径
英文流程架构图见 Planning terminology and architecture。下文表格的“当前名称”指审查基线上的名称,右栏是迁移建议。 二、不要把系统画成一条所有入口都相同的流水线 PromQL / MetricsQL 的主要计划路径是:
这里有三个不同范围的选择:语义 DAG 的选择、单个窗口物理实现的选择、整个工作负载物理候选的成本比较。它们并非重复实现。
ClickHouse 的自动路径直接从 SQL 选择与绑定构建 旧 flat workload → 三、#709 中应该修正或收紧的映射
依据:编译输入与选择 C1、候选与定价 C2、ERP C5、候选 key C6、pane 重定价 C7、运行时生命周期 C8。 以下映射方向正确,可以作为同一轮小范围迁移:
四、全仓中 #709 漏掉的歧义与重复
依据:旧 workload C4、共享 materialization C9、SDS C10、catalog C11、runtime config C12、存储元数据 C13、residual operators C14、shared publication C15。 五、必须保留的身份、时间和生命周期区别
生命周期至少有三条轴:计划
六、迁移与验收建议
额外维护问题: |
Why
Closes #709. Planning, pricing and runtime names overlap across component boundaries, and the metric registration path stores a second workload model alongside ASAPPlanner's canonical model.
What / How
Clarify compilation inputs, candidate diagnostics, residual operators, runtime generations and summary-series metadata. Delete
LegacyMetricWorkloadand the oldtypes::QueryWorkloadalias. Registration now stores ASAPPlannerQueryWorkloadwith its embeddedDataWorkload, plus deployment options; parsed metric fields and cost characteristics are derived when needed.Before this PR: JSON/YAML analysis creates a mutable flat workload and separately stores data characteristics. Query requirements and their scalar reporting view can diverge, while query strings and explicit overrides can describe different work.
After this PR: both field-only and string requests produce one canonical expression with requirements, recurrence and time selection. Registration and replanning preserve that model and its data evidence. For example, 10 series at 5 Hz becomes an aggregate ingestion rate of 50 samples/s. Sketch pins are passed to canonical binding as physical constraints, including for field-only requests.
The English architecture diagram and complete Chinese review and naming map document the component boundaries, field ownership and migration. Both are also posted in the PR and issue replies.
Compatibility
Existing serialized field spellings remain for the naming changes. Rust consumers must migrate the removed workload type and its field accesses. The metric registration adapter explicitly rejects conflicting query/field overrides and unsupported projections such as multiple selectors, non-equality filters, historical modifiers and unsupported recurrence. Full query compilation retains its broader query support. Dollar constraints are rejected rather than silently ignored.
Metric stage emission remains a deployment adapter; this does not remove its existing MVP policies or merge SQL and time-series pricing. Unknown or stale canonical data evidence does not become a fabricated default estimate.
Verification
clippy --workspace --all-targets -- -D warningspass.e25d53b1(Unify window candidate planning and preserve evaluation alignment #712, Publish storage routing from the physical plan #713, Remove test-only query planning and unused backend APIs and dependencies #717, Remove the unreferenced legacy planning endpoints #715),cargo test --workspacepasses: 2,092 tests passed, zero failed, three existing ignored tests/doc-tests. Existing environment gates remain in place. The six process E2E failures recorded before that integration now pass.