Conversation
RicoKomenda
left a comment
RicoKomenda
left a comment
There was a problem hiding this comment.
Cross-references all check out. The reworked controls are strictly better than what they replace, and new 3.4.2 is the strongest addition - prompt templates and agent policies being treated informally is exactly the kind of AI-specific gap AISVS is meant to fill.
The most debatable call is dropping old 3.4.5 since it explicitly named RAG indexes and agent-generated fine-tuning data, but C1's scope covers all training data sources so the coverage holds.
Approve.
| | **3.4.4** | **Verify that** model training and fine-tuning occur in isolated environments with controlled network access using egress allow-lists and no access to production tools or MCP resources. | 2 | | ||
| | **3.4.5** | **Verify that** training data sources are validated through integrity checks and authenticated via trusted sources with documented chain of custody before use in model development, including RAG indexes, tool logs, and agent-generated data used for fine-tuning. | 2 | | ||
| | **3.4.1** | **Verify that** AI-specific runtime components (agent orchestration services, tool/MCP servers, model registries, and prompt/policy stores) are not shared across environment boundaries (e.g., development, staging, production). | 1 | | ||
| | **3.4.2** | **Verify that** AI-specific configuration artifacts (prompt templates, agent policies and routing graphs, tool or MCP contracts and schemas, and action catalogs or capability allow-lists) are subject to the same version control and peer review requirements as application code. | 1 | |
There was a problem hiding this comment.
This one is more process than verifiable requirements
There was a problem hiding this comment.
Agreed good point. Now changed language to focus on technical implementation.
| | **3.4.5** | **Verify that** training data sources are validated through integrity checks and authenticated via trusted sources with documented chain of custody before use in model development, including RAG indexes, tool logs, and agent-generated data used for fine-tuning. | 2 | | ||
| | **3.4.1** | **Verify that** AI-specific runtime components (agent orchestration services, tool/MCP servers, model registries, and prompt/policy stores) are not shared across environment boundaries (e.g., development, staging, production). | 1 | | ||
| | **3.4.2** | **Verify that** AI-specific configuration artifacts (prompt templates, agent policies and routing graphs, tool or MCP contracts and schemas, and action catalogs or capability allow-lists) are subject to the same version control and peer review requirements as application code. | 1 | | ||
| | **3.4.3** | **Verify that** model training and fine-tuning environments do not have access to production model endpoints, agent orchestration services, tool/MCP servers, or live RAG data sources. | 2 | |
There was a problem hiding this comment.
Can we re-frame this as a positive? This is what not to do. Can we change this into what TO do? Something like:
Verify that model training and fine-tuning environments enforce strict environment isolation boundaries and are configured to only access explicitly approved non-production resources, excluding production model endpoints, agent orchestration services, tool/MCP servers, and live RAG data sources.
There was a problem hiding this comment.
Very good point, will try to keep this in mind going forward now while working through these.
|
I think now after the comments from Jim, that C03 is release-ready. 👍 |
3 participants
Polishes C03 (Model Lifecycle Management) by removing or reworking requirements that are not AI-specific or duplicate controls already present in ASVS v5 or other AISVS chapters (C1, C4).
Changes
Removed: 3.4.1 (environment separation)
Generic DevOps/security practice ("dev, testing, and production environments are logically or physically separated with distinct access controls"). Not AI-specific - applies equally to any software system. Already covered by:
Removed: 3.4.5 (training data source validation)
Heavily overlaps with C1 (Training Data Integrity & Traceability):
The tail clause about "RAG indexes, tool logs, and agent-generated data used for fine-tuning" is already scoped into C1 by its general applicability to all training data.
Reworked: 3.4.2 (environment isolation)
The original mixed generic infrastructure isolation (already in C4.3.2) with AI-specific concerns. Reworked to focus only on the AI-specific layer: agent orchestration runtimes, tool/MCP servers, and model registries must not be shared across environment boundaries. Removed the generic "infrastructure, data stores" part that C4 already covers.
Reworked: 3.4.3 (version control and peer review)
The original was a compound requirement mixing a generic software practice (use version control + peer review) with AI-specific artifact types. "Stored in version control and require peer review" is standard SDLC, not AI-specific. Reworked to focus on the AI-specific value: ensuring that AI-specific configuration artifacts (prompt templates, agent policies, tool/MCP schemas, capability allow-lists) are subject to the same version control and review rigor as application code. This framing acknowledges these artifact types are often treated informally and highlights the AI-specific gap.
Reworked: 3.4.4 (training environment isolation)
The original overlapped with C4.3.1 (default-deny network policies) and C4.3.2 (isolated network segments). Reworked to focus on the AI-specific isolation concern: training and fine-tuning environments must not have access to production model endpoints, agent orchestration services, tool/MCP servers, or live RAG data sources. The generic network isolation is left to C4.
Requirements retained without changes
All requirements in C3.1, C3.2, C3.3, C3.5, and C3.6 are retained. They are AI-specific (model artifacts, MBOM/AIBOM, model safety evaluations, agent workflow testing, model rollback, hosted model controls, RLHF reward model integrity) and not covered by ASVS v5 or other AISVS chapters.