Defining tree versions

[ahouseholder]

We've talked about this within the team already, but it's time to write some of it down.

I'm going to start with a premise that I think we have agreement on:

Decision Tree identity is the specific set of versioned decision points

Assumption: Decision Points are versioned objects as per #289.
Identity: A Decision Tree's identity is fully defined by the specific names and versions of the decision points it contains.

Because we have not yet explicitly versioned decision points, I can only specify them by name, but a "real" definition would require the version of each decision point to be specified as well.

Note that it's a set not a list, the order does not matter.

For example, the Coordinator Triage tree appears to be identified by:

Coord Triage Tree

• Public?
• Contacted?
• Credible?
• Cardinality,
• Engagement
• Utility
• PublicSafetyImpact

Except that Utility and Public Safety Impact are compound elements for notational convenience only, so we should represent them as their constituent parts.

Coord Triage Tree (all decision point versions TBD)

• Public?
• Contacted?
• Credible?
• Cardinality,
• Engagement
• Automatable
• Value Density
• Safety Impact

The Supplier tree, subject to a similar substitution of constituent decision points for Utility and Public Safety Impact, is defined as:

Supplier Tree (all decision point versions TBD)

• Exploitation
• Automatable
• Value Density
• Technical Impact
• Safety Impact

[ahouseholder]

Outcome Sets are not included

While the set of decision points and their combinations of values define the structure of a tree, an Outcome Set defines the possible labels of each leaf node. However, in order to decide what labels are appropriate for a leaf node, one must also be given a policy that maps each input combination (specific decision point values) to a specific outcome (drawn from an outcome set). But both the policy and the outcome set are actually stakeholder-specific.

The example trees we provide are at best a guess at a reasonable policy for SSVC adopters for each of the decisions we chose to model. And they also make assumptions about the process that the decision supports. So we assumed that suppliers have four options for priority. And we assigned outcome labels to leaf nodes based on what seemed reasonable to us at the time. But we must acknowledge that (a) stakeholders might have an arbitrary number of categories for prioritization (2, 3, 4, 5, ...), and (b) they may have wide variation in what combinations are given which priority.

Thus, the reasoning behind omitting outcome sets and specific outcome labels from the tree identity is as follows:

1. If we were to include specific labels as part of the tree definition, then we'd be saying that even small policy shifts (e.g., relabeling a single row of a large table of combinatorics) is a versioning event. This seems as if it would quickly become unwieldy.
2. If we were to include outcome sets as part of the tree definition, we'd be saying that folks using the same input information to make the same decision but filtering into 3 categories instead of 4 are using a different tree, and that would imply a versioning event. This is equally odd.

Note that 2 subsumes 1, in that 1 maintains the same outcome set and just modifies the mapping, whereas 2 allows the outcome set to change.

The thing we're asserting is that the structure of the tree (as defined by its constituent decision points and their specific versions) is invariant to the above, therefore the tree's identity omits both the outcome set and its specific mapping to the tree structure.

[ahouseholder]

On the naming and versioning of trees

Although the identity of a tree is fully defined in the initial post of this thread, it seems unreasonable to expect anyone to recognize that the object

Tree(Exploitation, Automatable, Value Density, Technical Impact, Safety Impact)

is related to a patch supplier's choice to create a fix, whereas

Tree(Public?, Contacted?, Credible?, Cardinality, Engagement, Automatable, Value Density, Safety Impact)

is related to a Coordinator's choice of how to handle a report.

So the first point to make is:

Trees have names

A tree name should be short, succinct, indicating both the Role and the relevant decision it applies to.

We've already established this principle in practice. The first object above has the name "Supplier Priority" while the second is "Coordinator Triage".

Trees have versions?

This is an unresolved part of the conversation.

The name "Deployer Priority" (or "Deployer Tree", we haven't been super strict on naming consistency) has been used to describe what we're now saying are two distinct tree identities.

SSVC version 1 had

Patch Applier Tree

• Exploitation
• Exposure
• Mission Impact
• Safety Impact

SSVC version 2 added Automatable and Value Density (combined as Utility), and combined Mission and Safety into Human Impact, but since we're sticking with the atomic decision points as identifiers, I'll maintain the expansion here.

Deployer Tree

• Exploitation
• Exposure
• Automatable
• Value Density
• Safety Impact
• Mission Impact

While SSVC version 2.1 dropped Value Density, leaving:

Deployer Tree

• Exploitation
• Exposure
• Automatable
• Safety Impact
• Mission Impact

My argument is that these three objects represent versions 1, 2, and 3 of "the thing" where "the thing" is defined by the role and decision it is modeling. In other words, attributes that make for good tree names (role, decision) make good names for a reason: because they describe an identity.

Above, I deliberately ignored our choice to shift terminology from Patch Applier to Patch Deployer as I think this is a refinement of our understanding but not a meaningful semantic change in who we expected the audience for that tree to be. But it does show up in point 3 below.

So a versioning rule for trees could be:

1. A change in role or the decision being modeled must have a new name.

2. A change (add, remove) in decision points where the role and decision being modeled are held constant is a new version of the existing name

   • Adding or removing a decision point entirely bumps the major version (increment by 1)
   • A change to a constituent decision point version increments the minor version (so if Exploitation v1 goes to v1.1, then trees
     containing Exploitation would increment by 0.1 when they adopt E1.1)

3 . Tree name aliases are permitted and do not trigger version increments

Given the above, the Deployer <=> Patch Applier alias by itself is a non-versioning event, but the addition of Automatable and Value Density (as Utility) implies the tree version should increment by 1.0.

[ahouseholder]

Reproducing the Deployer Tree info from above in tabular form to make it easier to read.

Decision Point	Tree v1	Tree v2	Tree v3
Exploitation	X	X	X
Exposure	X	X	X
Mission Impact	X	X	X
Safety Impact	X	X	X
Automatable		X	X
Value Density		X

[laurie-tyz]

I'm not sure I support the idea of versioning trees.

[ahouseholder]

Could you elaborate on that a bit? What concerns do you have with what's outlined here?

[ahouseholder]

Tree versioning, version 2

A second approach to tree versioning that has been suggested is:

• trees have names
• if necessary, we can refer to the "SSVC version X of tree name", so instead of the version 1, 2, and 3 of a "Deployer Tree" as described in this comment, we'd instead refer to them as:

Decision Point	SSVC v1.0 Patch Applier Tree	SSVC v2.0 Deployer Tree	SSVC v2.1 Deployer Tree
Exploitation	X	X	X
Exposure	X	X	X
Mission Impact	X	X	X
Safety Impact	X	X	X
Automatable		X	X
Value Density		X

Note that this is basically the "status quo" option, as it is how we have to refer to them as it stands because we haven't yet resolved the versioning conversations.

[ahouseholder]

My main concerns with this approach are that

1. It assumes that SSVC "versions" will continue to be a thing in perpetuity. And we have not established that. In fact, the more we move away from SSVC-as-document to SSVC-as-documentation (e.g., as in issue Move away from PDF as the main SSVC documentation vehicle #295), the fuzzier the "versions" of SSVC might get. That's why we started on the versioning discussion in What are the versioned objects in SSVC? #289 and here.
2. Say we adopted a time-based release cadence for the "macro" SSVC version...e.g., 202301, 202302, etc. If the only way we have to refer to a tree is by that version, then there is no immediate way to tell that SSVC 202301 Deployer Tree is the same as SSVC 202503 Deployer Tree. Nor is there a way to indicate the scale of change, if any, between versions.

Yes, the tree versioning described in this comment can be entirely derived as a function of the tree identity from the initial post of this thread, but if we're not going to expect humans to remember which role and decision that Tree(Exploitation, Automatable, Value Density, Technical Impact, Safety Impact) applies to by giving it a name, it seems equally necessary to give it a version, assuming that we're not going to just change tree names every time something small changes (like Exploitation v1.0->1.1).

[laurie-tyz]

So which are we going to version: the trees, the decision points, or both?

Regarding the table: How would someone find the reasoning behind the changes to the trees?
SSVC v1.0 Patch Applier Tree
SSVC v2.0 Deployer Tree (name change and added decision points)
SSVC v2.1 Deployer Tree (removed a decision point)

In this chart the changes are obvious, will the charts be included and updated as part of the documentation?

[ahouseholder]

I'm imagining it as "Using Deployer Priority Tree version 1 with values A,B,C,D and our customized policy, we chose to hide under a rock."

Yeah, it'd be weird if "hide under a rock" were an outcome option, but if we're letting stakeholders define their policy we kind of have to allow that.

But more useful to me is that because they said DPTv1, I know whether their A,B,C, and D are applicable to my decision process or not.

I'm not sure what the bar is that has to be cleared. I don't think what I've described is worse than the status quo (We used the SSVC version 2.0 Deployer Tree with A,B,C,D), and there are situations where it's better (because knowing DPTv1 puts me one lookup away from a precise definition of what A, B, C, and D mean, whereas it's more work to figure out that SSVC version 2.0 Deployer Tree is the one with Utility but not Automatable).

What other alternatives should we consider? Or, different question from a different angle, what's the problem you're concerned about happening?

[laurie-tyz]

A change at a decision point would change the DP version and trigger a change in the Tree version.

If that's the case, then why version the decision points at all? If we only use trees to track versions, then are we (CERT/CC) locking out changes a stakeholder would like to implement?

[ahouseholder]

A change at a decision point would change the DP version and trigger a change in the Tree version.

Yes. Because we're defining a tree's identity as a set of specific versioned decision points.

The idea of a decision tree version is just a convenience label for what would otherwise be (Flavor v1, Container v1) and (Flavor v1.1, Container v1) in the example above. While those tuples are sufficient to fully define what tree we're talking about, they're really inconvenient in everyday use, so if we only assign the name Menu to them, then we have no easy way to indicate that your Menu might be different from my Menu. But with versions, it's easy to recognize that Menu 1.0 and Menu 1.1 are probably close enough to have some shared semantics (indicated by different minor version) in addition to addressing the same decision problem (indicated by same name). The proposed versioning rules for trees are intended to address this: big changes to underlying decision points mean big changes to trees that use them, and small changes imply small changes, etc.

If that's the case, then why version the decision points at all?

Because decision points are the atomic chunks from which decision functions are built. We need to version them so we can tell whether my "Automatable" means the same thing as your "Automatable". That's what we've been discussing in #289.

If we only use trees to track versions, then are we (CERT/CC) locking out changes a stakeholder would like to implement?

I don't see how we'd be locking anyone out of anything.

• Got a new decision point? Use it.
• Got a modification to a decision point? Use it.
• Want to tweak a tree by adding or removing a decision point? Do that.
• Want to stick with the older version of a decision point even though version n+1 is available? You can.
• Want any of those changes to be incorporated into our library of decision points and suggested trees? Submit a PR.

The versioning rules only apply to the things we put into this repository. Users can do whatever customizations they want. So the only time we need to worry about how the versioning rules impact what a user does is when they want to contribute something back to the project. And in that case, we basically ask:

• Is this a new decision point or a modification of a previous one?
• Is this a new tree or a modification of a previous one?

And then we apply the rules and see how to label the contribution accordingly.

Can you describe a scenario/user story that illustrates your concern¹?
I.e., who's trying to do what, and why? (As a (who) I want to (what) because (why).)

Footnotes

1. This and other conversations might be an indication that we should deliberately start collecting more user stories anyway, because I think there is a lot of room for confusion even among the team about what SSVC needs to do for whom. ↩

[laurie-tyz]

Potential user story -
The Decision Point 3.4 changes to 3.5 forcing a change to the decision Tree from 7.0 to 8.0 and forces a change in documentation changes from 2.45 to 2.65.

[j---]

Decision Point	Version	Values
Flavor	1.0	Chocolate, Vanilla
Flavor	1.1	Chocolate, Vanilla, Strawberry

Just to flag, this is not the way Safety Impact currently relates to Public Safety Impact (for example). Up until now, changes to number of options have lead to decision point name changes. The relationship between those two is actually a smaller change than flavor 1.0 to 1.1, because Public safety is just a handy mapping of safety. Whereas there is no down-select from Flavor 1.1 that retains all the semantic options compressed into fewer options for Flavor 1.0.

[j---]

To be discussed, we could mean something else, but this is the kind of thing I have in mind when I think about a user story and what it should contain at what level of detail. https://www.atlassian.com/agile/project-management/user-stories

[ahouseholder]

This should probably be a separate discussion thread.

[ahouseholder]

So here are a few concrete examples of where I'm at on this:

https://github.com/ahouseholder/SSVC/blob/feature/pyjson/src/ssvc/dp_groups/deployer.py
https://github.com/ahouseholder/SSVC/blob/feature/pyjson/src/ssvc/dp_groups/supplier.py
https://github.com/ahouseholder/SSVC/blob/feature/pyjson/src/ssvc/dp_groups/coordinator_publication.py
https://github.com/ahouseholder/SSVC/blob/feature/pyjson/src/ssvc/dp_groups/coordinator_triage.py

[j---]

Thanks. I think this language is great.
Let me try to lay out something that may or may not cause problems for this idea of how trees are named with versions.
Let's say we want to give a system deployer a maturity ramp for trees that might fit their current information consumption and asset management maturity levels.

So tree one is a new decision point and one decision point that is basically: On the KEV -- patch. Not on the KEV -- defer.
Tree two is:
decision_points=[EXPLOITATION_1]

Tree three is:
decision_points=[EXPLOITATION_1, SYSTEM_EXPOSURE_1_0_1]

Tree four is:
decision_points=[EXPLOITATION_1, SYSTEM_EXPOSURE_1_0_1, AUTOMATABLE_1]

Tree five is:
decision_points=[EXPLOITATION_1, SYSTEM_EXPOSURE_1_0_1, AUTOMATABLE_1, MISSION_IMPACT_2]

And tree six is DEPLOYER_3 in your example.

How do we name and version this series of trees in a way that helps people onboard into progressively more mature vulnerability management?
Given that people might actually want to add mission impact before automatable (even though there are reasons for me picking automatable as easier to gather information for than mission impact, people should be able to express that).

[jeroenh]

Do we actually need to name these trees? The enumeration right now is already easily readable, and it could be improved further by choosing some abbreviated forms of decision point names.
At some point those names will also develop naturally when the trees are used a lot.

[ahouseholder]

@jeroenh I think we do not need to name and version them. See #315 for my idea of how I think we can address this as a how to/tutorial documentation exercise rather than as a versioning problem.

[ahouseholder]

This discussion is largely now the subject of Issue #355 and any subsequent PRs that will resolve it. Leaving the discussion open until #355 is closed.

• Create ADR(s) for Decision Point Group versioning #355