Repository navigation
[BOARD FEEDBACK] SCAP-NG Assessment vocabulary options #213
Replies: 4 comments 8 replies
A — Current SCAP-NG 0.3 / OVAL-alignedPrefer current 0.3 tests, object, select, Test-local states, explicit existence/match, evaluate, and reported_elements. React 👍 to this comment to vote A. Please choose only one. |
|
Variables are definitely still required in all options here in order to maintain lossless data (it's the blessing and curse of OVAL) However, I have tried my best to decrease their use, by translating some to If/Else and foreach where possible. Also static variables are gone, as they just lead to confusion. Let me find some examples and I'll attach one. I'll be digging through the generated sample of six STIG's to find one for you. Ideally I would love to see variables all go away, but I think the best I can do with a lossless translation is to provide alternatives, and hope content authors use Variables only when needed. Below is an example from the conversion, showing how it would work. I could also see where the test itself would just contain the data that IS the variable, vs referencing the variable. |
|
@robertgendler If after reviewing the attached scenario, you could mock up a more intuitive method that still is able to perform the required check, I'm open to any/all ideas. Do I want Variables to be gone, definitely, or at least dramatically reduced and only when needed. Have I figured out how to decrease them? (yes, and by a sizable number), but quite a few still still exist. Oh and BTW, if an Object in OVAL exists, solely for it to be used by a Variable, that object is IN the variable. I'm not sure of the implications of taking the entire construct of the Variable, and making it part of the object itself, and losing variables all together. Brainstorming here. |
|
I have no idea what the answer is exactly. Just a thought I had when looking at it. I'm not 100% sure I love the new format. But I like it more than the old XML format. |
|
@robertgendler if you want to see how a sample of the SCAP 1.4 STIG content translates into this format, just download the six stig authoring zip from https://github.com/vanderpol/scap-ng/releases/tag/v0.3.0-six-stig-typed-operators-5796d016b12f You can see all of the variables in their painful glory. |
|
@robertgendler I'm with you on not completely loving it, but I have tried my best to git rid of most of the things that I hated about OVAL, while retaining the concepts that allows us to convert and use the mountains of existing OVAL automation. My goal is to get it good enough that I can write an editor or VS Code plugin that will generate an assessment template from the STIG manual, and likely make a good guess on how it should be assessed, and then I'll just QA and commit it, plus have the editor autocomplete all of the fields and syntax check it all. |
B — Ansible-inspiredPrefer check, collect, where, expect, require, and report, with shared_collections for named reusable acquisitions (hypothetical, not Ansible playbook syntax). React 👍 to this comment to vote B. Please choose only one. |
C — Inspection-orientedPrefer check, inspect, select, expect, items/field_values, and evidence, with shared_inspections for named reusable acquisitions (hypothetical syntax). React 👍 to this comment to vote C. Please choose only one. |
D — Different or hybrid designNone of A–C fits; prefer a different combination or vocabulary. Please explain if possible. React 👍 to this comment to vote D. Please choose only one. |
|
Note that if you are voting for something completely different than what has been proposed in A-C above, please provide your recommendation, and likely be prepared to help create this new method. |
|
It's been a long time, but from what I can remember of what we decided to do with slang, it was more like this kind of object, where we had a schema that specified the allowed field names for a given type of object, and we tried to combine the searching, existence checking and value testing more... intuitively. Conversion wasn't necessarily lossless in theory. But in practice, if we couldn't convert something that we found in the real world, we added to the vocabulary of the object. Checks could be combined arbitrarily using AND, OR, XOR, and nested, e.g.: That was our general idea, anyway. It felt less cumbersome than the other proposals (A, B, C) seem to me. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
SCAP-NG Assessment authoring: A / B / C / D
OVAL Board feedback — informal, nonbinding preference vote.
October 14, 2026 owner-approved 0.3 readability improvement: Ordinary scalar comparison datatypes can be inferred from YAML, and exceptional OVAL comparisons use descriptive typed operators. These six redundant
datatype: stringentries have been removed from A, B and C equally. This does not indicate Board ratification or prove the experimental B/C grammars are fully convertible.The same real RHEL 9 STIG rule SV-257851 (
/homemust be mounted withnosuid) is shown below in three authoring styles. Which is easiest to read and maintain—including when acquisition is reused across checks?How to vote: After reviewing the comparison, the three complete check excerpts, and shared-acquisition example, add 👍 to exactly one of the four voting comments directly beneath this opening post:
D is a substantive alternative (not abstention). A brief explanation is welcome; if undecided, comment without voting. All examples, distinctions and acceptance requirements are in this post. No other document is needed.
Three vocabularies at a glance
test/testscheck/checkscheck/checksobject+capabilitycollectinspectselectwhereselectstateswith direct typed comparisons (nostate:wrapper)expectexpectoperation: equals,valueequalsequalsexistence,matchrequire.collected_items,require.matching_itemsitems.existence,items.matchexistence,matchexpect.*.existence,expect.*.matchfield_values.existence,field_values.matchevaluate/testevaluate/check(proposed)evaluate/checkreported_elementsreport.elementsevidence.fieldscollectdeclaration (proposed)checkdeclaration (proposed)shared_objectsshared_collections(proposed)shared_inspections(proposed)-objectsuffix-collection(proposed)-inspection(proposed)object: users-objectcollect: {ref: users-collection}(proposed)inspect: {ref: users-inspection}(proposed)The terms in B and C are discussion candidates, not approved enumerations.
inspectdenotes read-only assessment intent, not a ban on safe collector execution;expectmeans comparison, not remediation. Current 0.3 structural baseline:capabilityremains on the Test and the inline Object (which owns acquisition). Test-localstatesand Object/Setfilterscontain direct, typed predicates without a separately named State, astate:wrapper, a title, or a duplicate capability. B and C propose going further with a single capability declaration; that still requires safe lexical binding and is not implemented in native 0.3 authoring. Do not choose a vocabulary merely because it has fewer repeated lines. Embedded-predicate contract #208 · architecture research #197.Same real source requirement
RHEL 9 STIG, SV-257851: The
/homefilesystem must use thenosuidmount option. Each version is intended to select the same partition Items and evaluate the same expected mount option. That intended equivalence has not been established for B or C.A — Current SCAP-NG 0.3 (OVAL-aligned, actual syntax)
This is the real converted automated RHEL 9 Assessment excerpt, presented in the active 0.3 direct-predicate form from Assessment examples. Only surrounding metadata is omitted.
B — Ansible-inspired (research-only syntax)
This reads as collect partitions → where the mount point matches → expect
nosuid→ require matching Items → report fields → evaluate the named Check. Thechecks:map and explicitevaluate:root are proposed B syntax; the original one-checkshortcut could not express multi-check OVAL criteria losslessly. Like A/C, B must preserve the six-state OVAL result model rather than assume sequential tasks are an implicit AND. The single-capability shorthand and this Check vocabulary are not supported by the actual 0.3 schema. This is not Ansible playbook syntax.More than one check in B: The following is a separate schematic Assessment illustrating (A AND B) OR C. It is not an alternate evaluation of STIG SV-257851; each named Check would have its own complete
collect/where/expect/require/reportdefinition.The nested
all/anytree must preserve source OVAL criteria semantics, includingtrue,false,error,unknown,not_evaluated, andnot_applicable; it is not equivalent to Ansiblewhen, task ordering, orblock/rescue. An independent conversion/evaluation proof would be required before B could replace A.C — Inspection-oriented (research-only syntax)
This reads as inspect partitions → select the mount → expect the option → apply explicit Item and field-value constraints → retain evidence → evaluate a named check. The single
capabilityon this named Check is proposed to determine the private inline Object's acquisition capability and expected comparison field types; unlike B it also retains an explicit evaluation root. This lexical binding is not yet a valid 0.3 construct and must be proven safe rather than assumed.Named shared acquisition — an important real-world case
SCAP-NG 0.3 retains an Assessment-local named Object whenever several Tests, Variables, Sets or
for_eachcollectors depend on the same acquisition. It is declared inshared_objectsand referenced from a Test asobject: users-object. The B/C terminology should cover this case too, not just private collections embedded within one check.Proposed wording: B calls it a shared collection; C calls it a shared inspection. The acquisition's typed identity and reuse semantics must remain equivalent. For example, the three illustrative reference shapes would be:
These snippets demonstrate declaration and reference syntax, not complete Assessment documents. Only A's names and reference mechanism are part of the current 0.3 model;
shared_collections,shared_inspections, and the B/Crefforms are proposed, not schema-valid or evaluated. Named-collectionand-inspectionsuffixes would require deterministic ID/reference migration if adopted. The original source Object identity and dependency graph must survive conversion.Scope: Sharing here means one Assessment can refer to the same acquisition definition in several places. It does not guarantee the collector runs just once, imply a cache, share a Benchmark-wide global variable, or import collected Items across independent Assessments.
for_each.inwould continue to name its typed source acquisition in each vocabulary. Cross-Assessment shared Observation/Item import is a separate deferred design. Current component naming · currentfor_eachsemantics · local Object scope #165.Non-negotiable acceptance gate: lossless forward conversion from SCAP 1.4
All three candidates (A, B, and C) MUST support lossless, automated forward conversion from the supported SCAP 1.4/XCCDF and OVAL 5.12.3 source constructs. This is an entry requirement, not an optional benefit of A or a feature that members are voting to remove. Any fourth/hybrid proposal must satisfy the same gate. Lossless means equivalent technical and policy meaning, not byte-for-byte identical XML or identical authoring layout.
In particular, conversion SHALL preserve resource selection, collected-Item and Filter/Set membership (including ordering and include/exclude), capabilities, variable computations and bindings, typed values and all quantifiers, Boolean evaluation, applicability, policy/Rule/check selection, manual assessment intent, evidence/redaction requirements, and six technical outcomes (
true,false,unknown,error,not_evaluated,not_applicable). No translator may silently change a check into a weaker or stronger assessment, replace a regex with a literal, or hide an unsupported construct.Proof status is deliberately different from the requirement: A has a source-derived 0.3 converter and active corpus/conformance testing; that is not a blanket claim that its final runtime equivalence gate is complete. B and C are single-rule vocabulary sketches only. They have not demonstrated complete SCAP 1.4 representational coverage or a working lossless converter. Before either could replace A, they would need versioned grammars, precise mapping of complex source constructs, converter implementation, and differential six-state/corpus validation. The short YAML excerpts are not intended to contain every feature by themselves.
Deprecated or deliberately excluded input constructs (for example, legacy
independent.sqlext) SHALL produce explicit source-specific conversion errors, not invented equivalents. This defined exclusion is not permission to drop supported semantics. Backward conversion into SCAP 1.4 is not a requirement.Please choose the clearest authoring direction, assuming it must ultimately satisfy these gates—not merely the shortest one-rule excerpt.
What must not change just because the words change
.*\\/home(two literal backslash characters in the authored YAML). The source-derived regex text must be preserved exactly under YAML decoding and regex processing. Replacing it with a literal/homewould potentially change the selected resources. The prototype must prove exact literal/regex semantics.existence: one_or_moreapplies to collected Items; Testmatch: allcombines Item outcomes. Stateexistence: one_or_moreapplies to observed field instances; Statematch: one_or_morecombines State field comparisons. These cannot be silently collapsed.true,false,error,unknown,not_evaluatedandnot_applicableretain their meanings. A policy pass is not universally identical to an Assessment's technicaltrue.evidencemust not silently become a collection limit or permission to omit required proof.evaluate, named Test results, Set/Filter/Variable algebra, correlatedfor_eachItem populations, and Organizational Input must be representable without inventing Ansible-like procedural semantics.What feedback would help after your vote?
Please vote for the direction you actually support. In explaining an A–C vote, consider whether
inspectorcollectbetter conveys obtain target facts, whetherselectorwheremore clearly means identify relevant resources. Use D if a different vocabulary or hybrid would be preferable; please explain the needed improvement in the comments below. Do not vote A–C merely to avoid choosing D.There is a genuine timing decision: if the Board strongly prefers different terms, should 0.3 wait for a defined, tested change, or should we freeze its current vocabulary and make the improvement in a future version? A straw poll alone cannot settle that technical or release question.
For engineering background (not prerequisite reading), see the feasibility and compatibility study and architecture audit #197. The maintained current Assessment showcase remains the authority for 0.3 authoring.
Implementation / conformance tracking: authoring-vocabulary issue #207 (historical context) · embedded Test/Filter predicates and migration #208 · Board decisions #209 · architecture research #197.
Voting occurs only in the four A/B/C/D comments below this post. This is a request for direction, not ratification or a promise to change 0.3.
All reactions