[Feature Request] Reorganize agent skills around user workflows and data contracts

[SchrodingersCattt]

Summary

The current skill taxonomy under skills/ mostly mirrors implementation or API surfaces:

• dpdata-cli
• dpdata-driver
• dpdata-minimizer
• dpdata-plugin

This is useful for developers who already know dpdata internals, but it is difficult to discover and apply when a user or agent starts from a scientific workflow such as “convert these labeled outputs and verify that the resulting dataset is scientifically valid.” Could the skill organization be supplemented or reorganized around user intent and data contracts, while keeping extension guidance clearly developer-facing?

Detailed Description

What seems to be missing

A conversion skill should teach more than the command or format name. It should make the agent check the semantic invariants that determine whether a converted dataset is usable:

• labeled versus unlabeled data;
• one system versus multiple systems;
• frame alignment among coordinates, cells, energies, forces, and virials/stress;
• missing-label policy instead of silently accepting partial labels;
• units and stress/virial sign conventions;
• periodic-cell and PBC assumptions;
• atom ordering and type_map consistency;
• shapes, frame counts, and finite values;
• preservation of raw-data provenance;
• the output contract of the producer code and version being parsed.

At present, dpdata-cli is largely a command/format reference, while driver, minimizer, and plugin are extension mechanisms. The classification therefore mixes user workflows, command reference, and developer APIs at the same level, without an obvious place for reusable scientific validation knowledge.

Proposed information architecture

One possible structure is:

skills/
  dpdata-convert/
    SKILL.md
    references/
      semantic-validation.md
      producers/
        cp2k.md
        vasp.md
        abacus.md
        qe.md
  dpdata-inspect/
    SKILL.md
    scripts/
      validate_labeled_system.py
  dpdata-extend/
    references/
      format-plugin.md
      driver.md
      minimizer.md

The exact names are not important. The intended separation is:

1. Discovery/user intent: route requests such as convert, inspect, merge, filter, or validate.
2. Workflow skills: concise procedures such as dpdata-convert and dpdata-inspect.
3. Conditional producer references: load CP2K/VASP/ABACUS/QE contracts only when that producer is involved.
4. Developer extensions: plugin, driver, and minimizer guidance, clearly presented as implementation-facing material.
5. Deterministic validation: small scripts where checks are mechanical and should not depend on model judgment.

A single large skill would likely become another reference manual. A small router plus progressively loaded references would keep discovery simple without loading every producer-specific detail for every task.

Boundary of the knowledge layer

It may help to distinguish four kinds of knowledge:

• Reusable data invariants belong in dpdata skills: array alignment, units, atom/type ordering, label completeness, cell/PBC semantics, stress/virial conventions, and provenance checks.
• Parser/version contracts may belong in producer-specific references and should point to the responsible parser repository when implementation lives elsewhere.
• Project scientific choices should stay in project skills or records. For example, requiring CP2K PRINT_LEVEL MEDIUM may be necessary for a particular reference-data workflow, but is not a universal dpdata rule.
• Site execution details such as scheduler queues, module paths, or shared potential files should remain outside dpdata.

CP2K 2025 output compatibility is one motivating example, not the requested scope of this issue. Concrete parser defects are already better tracked in the parser repository (for example, cp2kdata issues #93 and #98). The dpdata skill should instead teach the reusable process: identify the producer/version contract, parse, validate labels and conventions, fail explicitly when required data are absent, and record provenance.

Further Information, Files, and Links

Questions for maintainers:

1. Would you prefer skills to be organized primarily by user workflow, while API-extension material is grouped separately?
2. Would a small routing skill plus several workflow skills be preferable to one broad dpdata-cli skill?
3. Where should producer-specific knowledge live when the parser implementation is maintained outside dpdata: here as a contract/reference, in the parser repository, or in both with cross-links?
4. Would a dpdata-inspect proof of concept, including a deterministic labeled-system validation script, be a useful first PR?

I am happy to prepare a focused follow-up PR after the preferred taxonomy and repository boundary are agreed.