Requirements SPEC.en

SecFusionAgent Requirements Baseline

English | 中文

Interactive view: Requirements view (module layers and data flow) — 中文 · English

Read this document before changing product behavior, technical design or evaluation criteria. This document defines the decomposition rationale, module division, data semantics and acceptance criteria of SecFusionAgent. The Requirements Review completed on 2026-09-25, and this file serves as the frozen requirement baseline; design changes that alter M1–M8, C1–C3, the acceptance criteria or the four risk boundaries re-enter Requirements Review. The scoring table, the deliverable list and the competition schedule are held by Topic 9 original text; technical architecture, component selection, models, databases and interfaces belong to the Technical Design Review.

Current implementation acceptance is tracked separately in Current acceptance; this requirements baseline does not assert that every criterion has passed.

Decomposition Rationale

The competition scoring criteria give three capabilities, whose module boundaries follow the stages of the data flow: monitoring, enrichment and question answering read and write the same security intelligence, and each answers the question of one stage — where the material comes from, whether different materials describe the same object, what is still missing around one question, and what conclusion the existing evidence supports. One function page can expose several modules in parallel, and one module can be consumed by several pages.

Performance metrics correspond one-to-one with functional metrics: monitoring timeliness belongs to monitoring, enrichment precision and recall belong to enrichment, and question answering accuracy, reasoning depth and response speed belong to question answering. Each module holds its own metrics, benchmark and regression, and cross-module comparison uses layered metrics, so that an average does not hide the degradation of a single module.

The competition scoring criteria give metrics but no unified test set and gold standard (human-adjudicated ground truth), so evaluation develops as a product module (M7) in parallel with the functional modules: a benchmark fixes the question text and the evidence visible at that time, human adjudication records its basis, and this is the only comparison baseline until the official definition is settled.

The four risk categories of the competition scoring criteria mix two dimensions: AI infrastructure, AI applications and Agent, and supply chain describe the object or the lifecycle position where the risk occurs, while prompt injection, system prompt leakage and data poisoning describe the attack method. An intelligence item records the object, the attack method and the impact in three separate fields, and one event can fall into several object categories; the classification serves assessment and coverage statistics. All four categories are in scope, and depth is allocated by sample availability and evidence availability.

The system processes public intelligence and the evidence relations within it. Public PoC, 0day reports and real attack events enter the intelligence chain as analysis material, for analyzing impact, versions, fixes and evidence relations; exploitation actions against real internet assets stay in the customer environment, alert aggregation and enterprise asset operations are carried by the existing SOC/SIEM, and multi-tenant operation belongs to the scope of the commercialization stage.

Module Structure

Layer	Module	Responsibility	Main deliverables
Intelligence acquisition	M1 Intelligence Monitoring	Discover newly added intelligence and record its source and time	Collection records, source health
Intelligence acquisition	M2 Evidence Retention and Normalization	Preserve the original text and unify the event identity	raw evidence, deduplicated intelligence items
Intelligence processing	M3 Intelligence Enrichment and Correlation	Fill in the fields and cross-source relations that support assessment	Enriched fields and their source evidence
Intelligence processing	M4 Investigation Context	Maintain the confirmed facts, conflicts and unknowns of one event or vulnerability	Readable, writable and traceable context
Investigation delivery	M5 Agent Investigation	Execute multi-step investigation and tool calls for complex questions	Investigation trajectory, newly added evidence
Investigation delivery	M6 Assessment and Question Answering	Form conclusions and answer, marking unconfirmed items	Conclusions, citations, answers
Support	M7 Evaluation	Fix benchmark, regression and security testing	Metric results, adversarial sample set
Support	M8 Engineering and Operations	Deployment, observability, failure recovery	Run chain, alerts and recovery records

Three cross-cutting constraints act on all modules, and their concrete implementation and verification land in the modules in parentheses: C1 evidence traceability (every conclusion can be located to its source, M2/M3/M6), C2 external content is untrusted (system rules, tool privileges and permitted actions are decided by configuration and policy; external material changes the evidence visible to the system, M2/M5/M7), C3 bounded execution (a task has a definite terminal state and a retry limit, M5/M8).

Data flow: M1 produces collection records, M2 produces intelligence items and raw evidence, M3 enriches and correlates on top of them, and the results of the three together form the M4 context; M5 reads and writes M4 to complete an investigation, M6 produces conclusions and answers from M4, M7 evaluates M1 to M6 module by module, and M8 carries the operation of all modules.

Risk Classification

Risk classification is a content dimension, orthogonal to the modules: every module handles the four risk categories, and benchmark samples for each category are drawn in proportion.

AI Infrastructure / Inference Frameworks

The targets are selected by public deployment volume, historical vulnerability density, public PoC, patch, version information and community activity, and these quantities decide whether a repo can support enrichment and a fixed benchmark. Real vulnerabilities are recorded separately by cause: authentication and exposure surface, input handling and code execution, model or plugin loading, resource and tenant isolation, configuration, dependencies and runtime environment, each with different evidence sources, affected objects and fix forms. The target repos, the L0–L5 function layers and the repo graph are held by 01-Data-Sources-and-Processing.

Model and Data Layer

The model and data layer is organized into five groups: Training / Fine-tuning Integrity, Model Artifact Integrity, Inference / Alignment Integrity, Confidentiality / Privacy, and Retrieval / Embedding Integrity; prompt injection acts as a cross-layer technique linking model behavior, retrieval content and Agent/tool execution. The taxonomy, the literature evidence chain, frontier intelligence monitoring and residual risk expression are held by 04-AI-Model-Data-and-Agent-Application-Security.

AI Applications and Agent

Agent/Application organizes risk by six trust boundaries: goal/context, tool/action, identity/privilege, execution boundary, memory/state, and inter-agent/workflow. Model decisions and software execution record their cause and their control location separately; the security semantics of sandbox, network, credential, memory, MCP/tool and side effect are held by 04-AI-Model-Data-and-Agent-Application-Security.

Supply Chain

Supply chain covers upstream development introduction, midstream build and delivery, and downstream runtime distribution, with the main objects including dependency, package, artifact, container image, model/data artifact, plugin/skill, build and distribution channel. The impact propagation semantics of depends-on, contains, built-from, published-as, distributed-as, deployed-with and so on are held by 02-CTI-Baseline-Reuse-and-Ownership.

Model theft, privacy leakage, memory poisoning, model backdoors, compute abuse and multi-tenant isolation all fall into the object categories above, and are further subdivided by attack method, impact and control boundary. No evidence currently supports adding a fifth parallel risk domain; when a new problem appears that the four object categories and the existing relations cannot express, it re-enters Requirements Review.

M1 Intelligence Monitoring

Responsibility: discover newly published or updated intelligence, record its source, publication time and collection time, and expose the health status of each source. The input is public sources, and the output is collection records and event candidates. Value judgement belongs to M6 and the assessment step, field enrichment belongs to M3, and event merging belongs to M2.

Internal stages: source discovery, fetching or subscription, publication time parsing, event candidate generation, health reporting. The publication mechanism and update time of a source decide the collection method, and CVE/NVD, vendor advisories, GitHub, papers and blogs differ from one another; the timing start point definition is in Acceptance Criteria. When a single source is unavailable the remaining sources keep collecting, and after recovery collection continues with idempotent writes.

Acceptance: source category coverage ≥7; the proportion of events in selected sources that actually enter the system is counted as coverage; timeliness is computed only for samples with a reliable publication time; source failure and recovery are observable in the status records.

M2 Evidence Retention and Normalization

Responsibility: preserve the original text and its metadata, and merge the representations of the same event in different sources into one intelligence item. The input is the M1 collection records, and the output is raw evidence and intelligence items with a unified identity. This layer is the evidence source of the whole system, and any later conclusion can be traced back here.

Internal stages: original text retention, field extraction, time and source annotation, identity determination, deduplication. When sources conflict, the material of both sides and their respective times are kept, and the conflict itself is recorded as one item of the context; the results of an incremental update and of a full re-collection are recorded separately. Field values come from the material, and a field the material does not provide is recorded as unknown.

Acceptance: raw evidence and processed results are stored separately; duplicated source content is merged and multi-source relations are retained; the material of both sides of conflicting sources is complete; duplicate and parser correctness are counted separately.

M3 Intelligence Enrichment and Correlation

Responsibility: fill in, around one intelligence item, the information needed to support a security judgement, and establish cross-source relations. The input is intelligence items and raw evidence, and the output is enriched fields with source evidence. The output of enrichment is fields and relations, and M6 forms conclusions from these fields.

Internal stages: dimension extraction, cross-source correlation, entity alignment, evidence attachment. The dimensions include affected object and version, vulnerability type, exploitation conditions, public exploitation evidence, patch or workaround, related research, and vendor statements; paper association, affected assets, PoC and CVSS are the dimension examples given by the competition topic. Every field records its source evidence, and a case where the material is correct but attached to the wrong version, product or vulnerability is counted as an entity resolution (entity alignment) error.

Acceptance: precision and recall are computed per field; entity resolution and evidence correctness are counted separately; a correct field value whose cited material does not support it counts as an evidence error; when two sources discuss the same product but their versions or components do not correspond, the correlation is not judged successful.

M4 Investigation Context

Responsibility: maintain the current state of a security incident or vulnerability — confirmed facts, source conflicts, unknowns, and the evidence supporting each of them. The input is the output of M1 to M3 and the evidence newly added by an M5 investigation, and the output is the readable and writable context shared by M5 and M6. The context is the central object of the system: question answering and investigation continue from its current state, and completed retrievals and steps are reused in it.

Internal stages: item creation, fact writing and correction, conflict registration, unknown registration, evidence indexing. raw evidence is a read-only layer, processed facts and assessment results are written to their own layers, and a result written later can correct an earlier judgement. The context keeps the source and time of every write, so that conclusions can be reviewed.

Acceptance: given one event, the confirmed facts, conflicts and unknowns can be listed from the context; every fact can be located to the M2 original text; asking the same question again reuses existing retrieval results; the context stays consistent under concurrent writes.

M5 Agent Investigation

Responsibility: execute multi-step investigation for questions that the existing evidence cannot answer directly — decompose the goal, select tools, read external material, and write newly added evidence back to M4. The input is an investigation task and the M4 context, and the output is an investigation trajectory and newly added evidence. The Agent handles the uncertainty and branches in the investigation process; collection, field validation, deduplication and metric computation stay deterministic.

Internal stages: task decomposition, tool selection and invocation, result evaluation, failure handling, termination decision. Task states are started, running, succeeded, failed, timed out and cancelled; the execution record covers task decomposition, tool calls and arguments, tool results, failures and retries, and the termination reason. Tool failure, timeout, partial results and source unavailability change subsequent actions; retry assumes idempotent writes, and the same fact stays a single record under repeated execution. Tool privileges, secret and sandbox keep a boundary against external content.

Acceptance: task success, tool selection, tool arguments, trajectory, recovery, timeout, latency and cost are counted separately; after a change to the model, prompt, tool policy, retrieval or parsing logic, regression runs on the same sample batch; every task reaches a terminal state within a bounded number of steps and time limit.

M6 Assessment and Question Answering

Responsibility: organize the verified evidence in the context into conclusions and answers, and mark unconfirmed items. The input is the M4 context, and the output is conclusions, citations and answers. A simple question is answered directly from the existing evidence; a complex question (whether a vulnerability affects a given deployment version, whether a public PoC applies to that version, whether a vendor fix covers an exploitation path) triggers an M5 investigation, and the answer completes after the newly added evidence returns to M4.

Internal stages: intent parsing, evidence selection, cross-source combination, multi-hop association, conclusion annotation. The answer keeps the three layers of fact, processed fact and assessment separate; when the user asks follow-up questions around the same vulnerability, the object, version and confirmed conditions stay consistent. When the evidence is insufficient, the sources conflict, or a key fact cannot be confirmed, the answer marks the unconfirmed items and the related evidence already found.

Acceptance: accuracy looks at the final answer, groundedness (the conclusion rests on evidence) looks at the evidence coverage of a conclusion, citation correctness (citations are correct) looks at whether a citation supports the corresponding statement, and multi-hop questions additionally check the intermediate relations; latency is counted separately for ordinary interactive QA and for long tasks that need continued investigation over the network.

M7 Evaluation

Responsibility: provide a repeatable quality conclusion for each of M1 to M6, and maintain security testing for AI and Agent. The input is the output and the benchmark of each module, and the output is metric results and an adversarial sample set. Evaluation develops together with the functional modules, and metric results are produced together with the module delivery.

Internal stages: benchmark maintenance, gold adjudication, metric computation, regression execution, fault injection, adversarial testing. A fixed benchmark pins the version and the time cross-section, and the question text is stored together with the evidence visible at that time; content that structured fields or rules can decide goes through deterministic judgement, and content that needs security assessment goes through human review or model-assisted review. The adversarial set（对抗样本集） covers direct and indirect prompt injection, poisoned source, malicious tool output, hallucinated evidence, tool misuse, privilege violation, and the traditional attack surface of the Web, backend and storage layers.

Acceptance: every module has corresponding metrics and a frozen sample set; after a change to the model, prompt, tool policy, retrieval or parsing logic, regression is comparable; fault injection verifies failure isolation, retry and bounded termination; the demonstration cases used in the competition are selected from the benchmark and include complete evidence, investigation trajectory and the failure/recovery process.

M8 Engineering and Operations

Responsibility: carry the deployment, observability and recovery of M1 to M7. The input is the runtime output of each module, and the output is the run chain, logs, metrics, alerts and recovery records. Credentials, tokens and sensitive configuration are kept in the secret management of the deployment environment, and the Agent uses them by reference when reading external content.

Internal stages: build and deployment, log and metric collection, alerting, failure recovery, version annotation. Every change automatically runs deterministic tests and key benchmarks and produces an identifiable version; when a deployment shows a source failure, wrong correlation, wrong answer or failed security test, it is located from the logs and task records, and a real failure becomes a new regression case. Sources are isolated from one another, and when a single source fails the remaining sources keep collecting; tool timeouts and retries have explicit limits, and a persistent failure raises an alert.

Acceptance: containerized deployment runs repeatably; CI/CD, logs, monitoring and alerting are verified in operation; a failure can be located to the task and the source; the design tradeoffs, evaluation method and failure handling of each module are explained clearly by team members and reproduced with experiments, and code is reviewed section by section after AI assistance.

The development order is SPEC → Requirements Review → DESIGN → Technical Review → PROBE → module implementation → integration testing / system evaluation → Release → Deploy → Operate → Incident / Feedback → next iteration. Probe answers only high-risk technical questions: whether a source can be collected reliably, whether the vulnerability samples of a certain kind of repo support a benchmark, whether an evaluation method can distinguish good from bad. The probe conclusion returns to the formal design.

M1–M3 Review Addendum — 2026-09-26

This Requirements Review Addendum does not change M1–M8, C1–C3, or the competition excellence targets. It fixes the previously open coverage denominator, evidence-quality semantics and enrichment P/R unit for M1–M3. The initial-round rubric specifies enrichment accuracy only, while the finals rubric specifies both precision and recall; the project continues to develop against the finals excellence target of precision ≥95% and recall ≥95%, and the initial-round report can reuse precision from the same benchmark.

M1 coverage and latency

The product source taxonomy is fixed to eight categories: vulnerability, development, academic, vendor, independent, normative, assets, and incidents. All eight count toward source-category coverage; the competition requires coverage of ≥7 categories, so the current product target exceeds that threshold. assets uses an M3 on-demand enrichment side path, but its processing lifecycle does not remove it from the source taxonomy. A category is supported only when the source inventory contains a coverage=owned fixed/grouped executable owner and the runtime adapter can be constructed; partial, dynamic_resolution, and point-in-time live reachability do not increase the category count.

category_coverage and source_delivery_coverage are separate metrics. The former is the supported-category count above. The latter freezes SourceDeliveryKey(source_id, external_object_id, external_revision | content_hash) gold keys for a source/window and measures the proportion of unique revisions that enter the corresponding accepted M1/M2 lifecycle, while also reporting missed and unexpected keys. Unexpected items do not increase the coverage numerator.

A monitoring-latency sample is defined as reliable published_at → M2 available_at/committed_at. Samples without reliable publication time are excluded from the latency-value denominator, while latency_evaluable_coverage is reported alongside the metric. The fixed evaluation report preserves per-sample results and internal regression also reports p50 / p95 / max / ≤6h pass-rate. Live-source probes remain point-in-time reachability reports rather than coverage or latency-correctness acceptance gates.

M2 identity and evidence quality

Cross-source identity prefers strong identifiers: CVE/GHSA/OSV/CNVD/CNNVD, DOI/arXiv, GitHub repository full name, package PURL or ecosystem+name, and commit SHA. Different namespaces are automatically merged only when an explicit source cross-reference or deterministic mapping proves identity; name similarity alone does not trigger an object merge.

An accepted claim/relation must resolve to an Observation / Artifact and locator. A structured-mapping locator must support the field or relation, and semantic extraction additionally requires quoted text to exist in the fixed source revision. M7 records parser-field accuracy, entity-resolution precision/recall, evidence correctness, replay suppression and conflict preservation separately; these diagnostic metrics are not mixed into competition M3 enrichment P/R.

M3 closed set and M7 enrichment metric

M3 vocabulary has three scopes: canonical / source_specific / exploratory. A canonical term enters formal enrichment P/R only when its registered subject/target shape also matches and the term is marked benchmarked; provider-native fields remain source-specific state. If semantic extraction uses a canonical name with a non-canonical shape, the result remains evidence-backed exploratory state; deterministic/source-asserted shape violations are rejected. Unregistered semantic terms likewise remain exploratory until vocabulary review creates a new revision.

enrichment-v1 fixes twelve dimensions: identity, severity, weakness, product/package, version applicability, fix/remediation, exploit state, exploit likelihood, advisory/reference, asset exposure, research/paper, and incident context. The competition examples CVSS, PoC, affected internet assets and paper association are all inside this closed set. PoC gold uses the canonical Vulnerability --has-poc→ ExploitArtifact relation; poc_available is only a derived convenience projection. Paper association v1 benchmarks ResearchWork/Document --discusses-vulnerability→ Vulnerability; deeper research relations remain non-benchmarked until their object model and gold protocol are frozen. Product/version applicability uses affected / not_affected / fixed / under_investigation / unknown and is represented by the scoped Vulnerability --applicability-status→ Product|Package|SoftwareVersion relation whose qualifier contains at least state and source_semantics. A query miss or incomparable version yields unknown, while not_affected requires a structured status, rule, or VEX justification. Open-text workaround / mitigation / attack_condition may remain canonical evidence-backed knowledge in v1, but they are benchmarked=False until a normalized schema is frozen and therefore stay out of closed-set P/R.

The minimum gold unit is EnrichmentFactKey(stable root object key/type, vocabulary revision, dimension, claim|relation kind, predicate/relation, normalized value/target, target object type, qualifier keys + normalized qualifier, temporal scope). A TP requires the tuple, entity/version scope, and EvidenceRef to be correct together. Wrong values, wrong entity/version binding, wrong relations, converting unknown to not_affected, and accepted facts without evidence are FP; missing gold is FN. The gold item itself must pass the same vocabulary subject/target-shape and required-qualifier checks. The primary report uses micro precision/recall for the competition thresholds and also reports dimension-macro P/R so high-frequency simple fields cannot hide weak PoC, asset, or paper-relation recall.

The storage/read M3→M4 handoff has passed its executable gate. This addendum fixes the input semantics of the semantic enrichment handoff: M3 can expose resolved/conflict/unknown/missing state over canonical dimensions. How M4 turns those gaps into Task / Tool / Policy behavior remains a later Technical Design 2 review and does not redefine the M3 vocabulary or gold unit.

Acceptance Criteria

The excellent-tier numbers are the goal of this stage: data source coverage ≥7 categories; monitoring latency ≤6h; enrichment precision and recall both ≥95%; QA accuracy ≥95%, including multi-hop reasoning and explanation of key evidence relations; ordinary interactive QA ≤5s. Every number is submitted together with the fixed benchmark, gold, computation method and raw results.

Timeliness is measured from the first public disclosure of a source to the point where M2 forms a usable record, and the computation range is limited to samples with a reliable publication time. The score structure of the competition scoring criteria determines the order of investment: in the preliminary round the functional metrics and optimization metrics total 70 points and the performance metrics total 15 points; in the final round the functional metrics and optimization metrics total 55 points, the performance metrics are 20 points while question answering quality alone is 10 points. In the preliminary round enrichment accuracy reaches the excellent tier at 95%; in the finals both enrichment precision and recall must reach 95%. Beyond the scoring threshold, further improvement changes intelligence quality; the Agent architecture is worth 6 points from "basic use of an Agent framework" to "multi-role Agent collaboration and autonomous orchestration", and system automation is worth 2 points from "with exception handling" to "with self-healing". The score details and the deliverable list are in Topic 9 original text.

Competition Scoring Criteria Coverage

The table below maps the scoring items of the competition scoring criteria to the modules of this document; the Requirements Review checks coverage against this table, and the score details are in Topic 9 original text.

Scoring item	Owning module
情报监测功能完整性 (intelligence monitoring completeness)	M1 Intelligence Monitoring
情报富化功能完整性 (intelligence enrichment completeness)	M3 Intelligence Enrichment and Correlation
情报问答功能完整性 (intelligence question answering completeness)	M6 Assessment and Question Answering
监测时效性与稳定性 (monitoring timeliness and stability)	M1 Intelligence Monitoring; M8 Engineering and Operations
富化准确率与召回率 (enrichment precision and recall)	M3 Intelligence Enrichment and Correlation
问答深度与准确率 (question answering depth and accuracy)	M6 Assessment and Question Answering; M7 Evaluation
智能体架构设计 / 智能体运用水平 (Agent architecture design / Agent utilization level)	M5 Agent Investigation
系统自动化程度 / 系统自动化与工程化 (degree of system automation / system automation and engineering)	M8 Engineering and Operations
文档质量 (document quality)	the report content items of Topic 9 original text
演示效果 (demonstration effect)	the demonstration cases of M7 Evaluation
创新性、系统性、实用性、团队协作 (innovation, systematicity, practicality and teamwork)	Decomposition Rationale; Module Structure; M8 Engineering and Operations

Requirements Review Conclusions and Design Handover

The Requirements Review is complete. The 2026-09-26 M1–M3 addendum additionally freezes the monitoring/enrichment evaluation semantics without changing module responsibilities. The former T0–T4 all no longer block Technical Design:

Item	Frozen conclusion	Subsequent ownership
T0 benchmark source	Contestants build the dataset themselves, and judges use it in the evaluation stage	M7 designs the fixed benchmark, gold and regression versions
T1 affected internet assets	The asset object, evidence semantics and provider types such as Censys/Shodan/FOFA/ZoomEye are fixed	Concrete queries, fingerprint mapping, geographic filtering and verification methods enter M3 Design / Probe
T2 AI Infra / supply chain	The AI Infra target repos, function layers and repo graph are fixed; supply chain objects, artifacts and propagation relations have entered the baseline	Concrete connectors, SBOM/provenance handling and graph execution enter M1–M3 Design
T3 model and data taxonomy	The five risk families and the cross-layer prompt injection semantics are fixed	The benchmark sample proportion is designed by M7 according to evidence availability and competition coverage goals, and does not change the requirement classification
T4 other AI risks	All current candidates fall into the four object categories and the existing attack/impact dimensions	When a new risk cannot be expressed, the Requirements Review reopens

实时监测 (real-time monitoring) has no unified polling requirement. M1 designs delta, API, Git, RSS, page diff and on-demand schedule separately according to the update semantics of the source registry, with ≤6h as the competition acceptance target. The routing threshold between ordinary QA and long investigation belongs to M5/M6 Technical Design; that threshold does not change the responsibility boundary of the two modules.

After freezing, the implementation order stays M1/M2 → M3 → M4 → M5 → M6, with M7 and M8 advancing cross-cutting. Technical Design starts from the information intake / knowledge plane and turns the already fixed source, evidence, object, relation and retrieval contracts into executable data flows and component interfaces.