Frameworks structural specification

[Robert-MacWha]

My goal here is to define the taxonomy of frameworks, how it should be structured, and how this structure will change as it grows & interact with certs.

Taxonomy

• Frameworks is a collection of structured documentation which can be referenced, either by certs or the general public, focused on blockchain security and actionable steps.
• Certs is a collection of benchmarks and playbooks, categorized by specific domains, which can be used to measure a protocol's alignment with frameworks.

Relationship structure

---
title: Frameworks
---
flowchart LR
MaturityModel{Security Profiles} --Which to follow--> Document
Benchmark{Benchmarks} --Measured by--> Document
Document --Best-practices for--> Benchmark
Benchmark --How to implement--> Playbooks

Loading

---
title: SCF (SEAL Certification framework)
---
flowchart TD
MaturityModel[Security Profile] --> B1[Benchmark 1]
MaturityModel --> B2[Benchmark 2]
MaturityModel --> B3[...]
MaturityModel --> B4[Benchmark n]
B1 --How to Implement--> Playbooks
B2 --> Playbooks
B3 --> Playbooks
B4 --> Playbooks

Loading

Where:

• Security Profiles are metrics by which we categorize different frameworks for different groups. "Who is this for?"
• Documents are textual descriptions of where a protocol should be. "What good looks like?"
• Benchmarks are checklists used to measure a protocol's completion of a framework. "How do you measure good?"
• Playbooks are textual guides on how to accomplish a specific objective / handle a specific scenario. "How do we do this?"

Open Questions

• How strictly should frameworks abide by the above taxonomical structure?
• How should this structure be embedded in documents? Per-document classification, intra-document, or something else?
• Are we ok with adding metadata (IE more toml frontmatter, metadata within documents), or do we want structure to be defined somehow else (structural, via external metadata files).

Current plan

• Phase 1: Certs and frameworks will be semi-seperated. Frameworks will act as a store for playbooks and documents which will be referenced by certs, while certs will store benchmarks and security profiles. Practical example: imagine splitting up multisig best practices into several frameworks (IE security policies, device setup, wallet setup, communication requirements, transaction verification). Certs will then reference these frameworks instead of writing its own content.
• Phase 2: As frameworks becomes a richer repository of content, certs can migrate security profiles and benchmarks into frameworks, and begin generating certs procedurally from frameworks.
  • Procedural generation allows for much greater cert granularity and specificity, keeping documents shorter and more targeted. However, it also comes with more upfront work and less customization, so we should only do this if we're sure it's the right move.

Example

Example for a simplified wallet security cert / framework.

frameworks/
├── wallet-security/
│   ├── index.md                    # Profile matrix definition
│   ├── wallet-security.md     # Framework with scoped sections
│   ├── benchmarks.yaml       # Structured benchmark definitions
│   └── playbooks/
│       └── hardware-wallet-setup.md   # Implementation guides

index.md

# Wallet Security Framework

Comprehensive security practices for wallet operations across different roles and risk levels.

## Security Profiles

### Role Axis
- **signer**: Individual signer
- **admin**: Multisig manager / admin

### Value Axis  
- **low**: <$100k exposure
- **medium**: $100k-1M exposure
- **high**: $1M-10M exposure

### Timescale Axis
- **everyday**: Everyday access, over 12 hour response times acceptable
- **emergency**: Emergency access is required, under 12 hour response time

wallet-security.md

---
document_type: "framework"
---

# Wallet Security Framework

## Hardware Requirements (all)
All wallet operations must use approved hardware wallets. Software wallets create unacceptable security risks for multisig operations.

**Approved Hardware:**
- Ledger Nano S Plus, Ledger Stax
- Trezor Model One, Trezor Safe 3

Purchase only from manufacturers or authorized resellers. Verify tamper-resistant packaging before use.

## Backup Hardware (medium OR high)
Maintain backup hardware wallet with identical seed phrase stored in separate secure location. Test backup device functionality monthly to ensure recovery capability.

Critical requirement: backup device must be kept in geographically separate location from primary device.

## Dedicated Devices (operator OR admin AND high)
Use hardware wallets exclusively for multisig operations. No DeFi interactions, NFT trading, or personal transactions on the same device.

This isolation prevents cross-contamination from potentially compromised dApps or smart contracts.

benchmark.yaml

benchmarks:
  hardware_requirements:
    title: "Use Hardware Wallets"
    description: "All signers must use approved hardware wallets for key operations"
    verification: "Demonstrate hardware wallet setup and test signature"
    evidence: ["Hardware wallet model/serial", "Test transaction signature"]
  
  backup_hardware:
    title: "Maintain Backup Hardware"
    description: "Keep functional backup hardware wallet with same seed"
    verification: "Show backup device can sign test transaction"
    evidence: ["Backup device photo", "Successful backup signature"]
  
  dedicated_devices:
    title: "Dedicated devices" 
    description: "The hardware wallet must exclusively be used for multisig operations"

hardware-wallet-setup.md

---
document_type: "guide"
---

# Hardware Wallet Setup Guide

Step-by-step instructions for secure hardware wallet configuration.

## Initial Setup (all)

### Purchase and Verification
1. Buy only from manufacturer or authorized resellers
2. Verify tamper-resistant packaging is intact
3. Check device for physical tampering

### Firmware and PIN
1. Update to latest firmware before creating accounts
2. Set strong, unique PIN (different from other devices)
3. Never use pre-generated seed phrases

## Backup Configuration (operator OR admin AND high)

### Backup Device Setup
1. Generate seed on primary device
2. Set up backup device with same seed
3. Test both devices can sign transactions
4. Store backup in separate secure location

[Robert-MacWha]

@mattaereal want to hear your thoughts on these proposed structures

[Raiders0786]

Really solid structure @Robert-MacWha

Been thinking about this while implementing similar frameworks, and there's a gap we keep hitting that might be worth addressing.

So here's the thing - when teams actually try to implement these frameworks, they always ask the same questions: "which one do we do first?" and "do we need to hire someone for this?"

What if we baked those answers right into the structure? Like:

Each benchmark could carry its own difficulty score. Picture this in the yaml:

hardware_requirements:
  difficulty: "easy"  # most teams can self-implement
  time_estimate: "30min"
  requires_assistance: false  # no need to call in the cavalry

vs something like:

secure_key_ceremony:
  difficulty: "difficult"
  time_estimate: "2-4 hours"
  requires_assistance: true  # yeah, you probably want an expert here

For the priority question:
Some of these are absolutely critical (like using hardware wallets), while others are more "nice to have if you've got time". Maybe mark them as Essential/Recommended/Additional right in the framework sections?

For example:

Hardware Requirements (all)
Priority: Essential  # Essential/Recommended/Additional
Blocking: true  # Can other items proceed without this?

Been in too many situations where a team spends weeks on some complex additional consideration while missing a basic essential.

The dependency thing is real too
You can't set up backup hardware if you haven't bought the primary hardware yet (obvious, but you'd be surprised). Making these dependencies explicit would save teams from that awkward moment of realizing they're doing things backwards.

backup_hardware:
  depends_on: ["hardware_requirements"]
  blocks: ["multisig_setup", "key_rotation"]

One more thought:
Risk scores might actually drive adoption. If I can tell a team "this 30-minute change prevents 90% of wallet compromises" vs "this 2-day implementation prevents an edge case", they know exactly where to focus.

Example:

hardware_requirements:
  risk_reduction: "high"  # Prevents ~90% of wallet compromises
  attack_vectors_mitigated: ["malware", "clipboard_hijacking", "browser_exploits"]

I'd start simple - just add difficulty and priority metadata first, see how it plays out. Once teams start using it, the patterns for dependencies and verification automation will probably reveal themselves naturally.

The phased approach you're taking makes total sense, btw. Better to have something teams can use now than to wait for the perfect procedural generation system.

These are my thoughts and feedback- happy to brainstorm!

[Robert-MacWha]

For the priority question:
Some of these are absolutely critical (like using hardware wallets), while others are more "nice to have if you've got time". Maybe mark them as Essential/Recommended/Additional right in the framework sections?

Essentially adding similar metadata as https://howtomultisig.com/? Yeah - makes sense! I'm all for adding more context, especially if it's something that'll help teams prioritize / contextualize what should be done. Would it be simplest just to include "priority" as one of the maturity models? It's already essentially a high-dimensional prioritization matrix, so adding a "priority" or "risk_reduction" axis fits pretty well into the existing scope.

In light of this - might be better to conceptualize of maturity models as "Security Profiles" or something. We're still asking "Who is this for", but we're not just considering the protocol's position. We also take into account how important is the implementation. Only downside is now we'll probably need actual metadata instead of just embedding the maturity model in the title :D . But I'm fine with that.

hardware_requirements:
  profiles:
    role: ["individual", "operator", "admin"] 
    value: ["medium", "high"]
    priority: "essential"

[Robert-MacWha]

One more thought:
Risk scores might actually drive adoption. If I can tell a team "this 30-minute change prevents 90% of wallet compromises" vs "this 2-day implementation prevents an edge case", they know exactly where to focus.

Also good point - I definitely want to include what attack vectors are mitigated by different steps for education & incentivisation. If we do include it as metadata, it's probably best to also include it in the plaintext descriptions (something closer to howtomultisig's descriptions). Being able to sort / filter by attack vectors is a cool emergent property.

hardware_requirements:
  profiles:
    role: ["individual", "operator", "admin"] 
    value: ["medium", "high"]
    priority: "essential"
  metadata:
    depends_on: ["hardware_requirements"]
    attack_vectors_mitigated: ["malware", "clipboard_hijacking"]
    time_estimate: "30min"
    difficulty: "easy"
    risk_reduction: "high"

[Robert-MacWha]

The dependency thing is real too
You can't set up backup hardware if you haven't bought the primary hardware yet (obvious, but you'd be surprised). Making these dependencies explicit would save teams from that awkward moment of realizing they're doing things backwards.

I wonder whether we can solve this more cleanly? The documents themselves will be ordered, so task order will be preserved within the markdown. So if you're following the framework / cert as intended, then it wouldn't be an issue. But if you jump in halfway through to solve some specific task, it could mess you up.

No particular harm with adding metadata, I'm just hesitant to create a graph network since that creates more failure points (no longer self-contained). But it is a problem that should be solved.

Can't think of a better solution right now, but I'll ponder it. Worst comes to worst we do make a graph and I make some pretty flowchart visualization for fun / utility (the terror).

Frankly all of this is graphable. Yay for graphs!

[Raiders0786]

Ah, interesting point about the howtomultisig.com (i like it too)

You're right that priority fits naturally into the maturity model concept, and reframing "Maturity Models" as "Security Profiles" is a brilliant idea IMO - it captures what we're really doing here. We're not just measuring maturity, we're defining complete security postures based on multiple dimensions.

Your metadata approach makes this super clean. One thought i've here is - if we're going this route, we might want to distinguish between two types of "priority":

1. Implementation priority (essential/recommended/additional) - "must you do this?"
2. Implementation order (dependencies) - "when should you do this?"

So maybe:

hardware_requirements:
  profiles:
    role: ["individual", "operator", "admin"] 
    value: ["medium", "high"]
    priority: "essential"
    risk_reduction: "high"  # helps with prioritization within same priority tier
 metadata:
  depends_on: ["hardware_requirements"]
  attack_vectors_mitigated: ["malware", "clipboard_hijacking"]
  time_estimate: "30min"
  difficulty: "easy"

This way teams get both "what's critical" and "what order makes sense" without having to figure it out themselves.

The metadata requirement is honestly a small price for the clarity it brings. Better to have an explicit, searchable metadata UI than to try to parse it from titles or hope people read between the lines. And yeah, having attack vectors both in metadata AND plaintext descriptions makes total sense.

On the dependency graph concern - I hear you on the complexity.

You're right that following the document order naturally solves most cases. Maybe we start simple: just add a "prerequisites" field for the non-obvious cases? Like:

backup_hardware:
  prerequisites: "Requires primary hardware wallet already configured"

This way it's human-readable, doesn't create a complex graph structure, but still saves someone from that "oh wait, I need to do X first" moment. If we find ourselves needing more sophisticated dependency tracking later, we can evolve it then. And if something has no dependencies, just leave it empty or omit it entirely.

One practical thought:

As we add all this metadata, might it be worth creating a small validation script to ensure consistency? Nothing fancy, just checking that all required fields are present, attack vectors are from a defined list, etc. Would save us from the inevitable typos and inconsistencies as this grows.

[Robert-MacWha]

This way teams get both "what's critical" and "what order makes sense" without having to figure it out themselves.

Fair point - that's a good distinction to make. I figure we'll probably iron out the specific axes over time as we see what works and what doesn't, but I'd definitely like to note down the kinds of questions we'll be answering and figure out whether they fit as axes or as additional metadata.

• "Is this relevant to me"
• "How should I prioritize these"
• "How hard will this be"
• "What do I need to do before"
• "What risks will this reduce"
• ...

You're right that following the document order naturally solves most cases. Maybe we start simple: just add a "prerequisites" field for the non-obvious cases? Like:

Yeah that works! I'm not strongly opposed to the dependency graph, just wondering whether something simpler could be better. For now text descriptions would make sense, and we can always add detail if it proved helpful. Like you said - better to have explicit metadata :)

As we add all this metadata, might it be worth creating a small validation script to ensure consistency? Nothing fancy, just checking that all required fields are present, attack vectors are from a defined list, etc. Would save us from the inevitable typos and inconsistencies as this grows.

100%. I think @mattaereal currently looking into MDbook alternatives and might want to switch over, so I'm hesitant to build anything quite yet. But will be important for sure.

[relotnek]

If maturity models are just used to answer the questions "who is this for?" I would call that audience instead or Security Profiles like @Robert-MacWha mentioned. A maturity model inherently conveys the maturity of a particular implementer whether that's an individual or an organization within the context of a set of rules or guideiles, whereas it seems like this is identifying who the implementer actually is. I think that puts us in a position of being opinionated about a specific level for a specific audience. In a maturity model the implementer should be making that risk decision

I think what we're lacking here at the top level is a way to measure/compare the state of an implementers current posture vs their target posture. So this applies well to the example in "wallet security" where completion of each "level" is relatively binary. The implementer completes the benchmark or doesn't, but in something more complex like a "Cross-chain Bridge Security Maturity Model" this becomes much more difficult to apply.

I also don't think something like wallet security really needs it's own maturity model. Wallet security is something that benefits from a benchmark, document, and playbook, but isn't something that's looked at at such a high level. I see wallet security as part of a larger framework like an operational security guide. It may be worth thinking of all maturity models being built by benchmarks, documents, and playbooks that are available.

[Robert-MacWha]

I would call that audience instead or Security Profiles

I think you're right, I've updated the original text to include this new name.

a way to measure/compare the state of an implementers current posture vs their target posture

Good point. Benchmarks give you a decent snapshot, but they can't really capture "things people do that aren't benchmarked".

I figure we could partly solve this by just adding lots of benchmarks (so they'll meet some, not meet others, and we can take the average)? Not terribly elegant. If they use a multisig where half of the signers are hardware wallets, and half are hot wallets, do we give them the point? Probably not, but it feels like they should get some credit.

I think this is more of a Certs question, so I'll get Issac's take on it. For frameworks I think it's ok to have binary benchmarks, and to provide context for those benchmarks such that users can come to their own conclusions about how well they fulfill them, even if they don't check the box. Possibly it's as simple as a "for each benchmark you didn't meet, write down an action plan".

don't think something like wallet security really needs it's own maturity model

Yep. What I'm hoping to define is the "maximum possible complexity" case. Many of the frameworks can and should be simpler, and for stuff like wallet security where "best practice" is basically achievable by everyone, there won't be different security profiles nor many documents.

The maturity model categorization is in service of certs being able to pull together many different benchmarks / frameworks into a single "plan of attack" for a protocol / certification.