Request for feedback on ASVS v5 proposal: Make the level definitions more practical

[tghosth]

Guidelines for new definitions

It has been widely discussed that we don’t want to focus on the “pentestable” concept for 5.0. See: #1495 (comment)

We talk about the current levels being “risk based” but practically the actual definitions currently seem to be something like this:

• L1 - Pentestable or so important that we cannot justify L2
• L2 - Not-pentestable or conceptually less urgent
• L3 - Harder to implement or even less urgent.

I think that levels based on the types of requirements makes sense and we just need to be clearer about this.

Alongside this, I think we should consider the levels more around practicality than risk and leave it to users to decide which level is correct for them.

Proposed Levels

Level 1 - Common Protections

These are the minimum security requirements that an application should have in place to prevent common attacks.

We would prioritise them as “common” based on our industry view of what types of attacks are most common or likely. We could also reference the OWASP Top 10 data sets for this.

Some example requirements that I think would fit this definition:

• 5.3.4 - Verify that data selection or database queries (e.g. SQL, HQL, ORM, NoSQL) use parameterized queries, ORMs, entity frameworks, or are otherwise protected from database injection attacks.
• 2.1.1 - [MODIFIED] Verify that user set passwords are at least 8 characters in length.
• 3.4.1 - Verify that cookie-based session tokens have the 'Secure' attribute set. (C6)
• 3.3.1 - Verify that logout and expiration invalidate the session token, such that the back button or a downstream relying party cannot resume an authenticated session. (C6)
• 2.2.9 - [ADDED, SPLIT FROM 2.2.4] Verify that multi-factor authentication is required, that is, the application uses either a multi-factor authenticator or a combination of single-factor authenticators.
• 4.3.1 - Verify administrative interfaces can only be logically accessed from trusted endpoints or locations. For example, restricting access to bastion or jump hosts, trusted admin workstations or endpoints (e.g., device authentication), administrative LANs, etc.
• 5.2.6 - Verify that the application protects against SSRF attacks, by validating or sanitizing untrusted data or HTTP file metadata, such as filenames and URL input fields, and uses allow-lists of protocols, domains, paths and ports.

Level 2 - Wider Protections

This adds additional requirements to provide against a wider range of attacks which might be less common, based on the above assessment.

Some example requirements that I think would fit this definition:

• 4.2.3 - [ADDED] Verify that messages received by the postMessage interface are discarded if the origin of the message is not trusted, or if the syntax of the message is invalid.
• 5.3.10 - Verify that the application protects against XPath injection or XML injection attacks.
• 6.2.5 - [MODIFIED] Verify that known insecure block modes (i.e. ECB, etc.), padding modes (i.e. PKCS#1 v1.5, etc.), ciphers with small block sizes (i.e. Triple-DES, Blowfish, etc.), and weak hashing algorithms (i.e. MD5, SHA1, etc.) are not used.
• 11.1.8 - Verify that the application has configurable alerting when automated attacks or unusual activity is detected.

Level 3 - Comprehensive Defense with Depth

This adds the remaining requirements to provide either more complex protection mechanisms or mechanisms that are more like defense in depth.

Some example requirements that I think would fit this definition:

• 6.4.2 - [MODIFIED] Verify that key material is not exposed to the application (neither the front-end nor the back-end) but instead uses an isolated security module like a vault for cryptographic operations.
• 13.2.2 - Verify that JSON schema validation is in place and verified before accepting input.
• 14.1.5 - [MODIFIED] Verify that deployed environments are short lived and frequently redeployed to a "known good" but updated state. Alternatively, long lived environments should use some form of "drift prevention" to ensure that deployed configurations are not changed to an insecure state.
• 14.5.7 - [ADDED] Verify that the web application warns users who are using an old browser which does not support HTTP security features on which the application relies. The list of old browsers must be periodically reviewed and updated.

Call to Action

What do people think about these definitions? I would welcome any input although I don't want to argue about specific requirements from the examples above but rather just the definitions themselves.

[csfreak92]

consider the levels more around practicality than risk

@tghosth, to help kickoff the discussion for the community, maybe I just want to clarify that by this statement above do you mean practicality of ease of implementing a particular requirement? If yes, then I would agree with that for defining the levels of requirements with L1 being easiest and quick to implement, followed by L2 for wider protection and L3 for defense in depth protections.

[tghosth]

Kind of although I would also relate it to the value the requirement brings compared to complexity to implement.

[EnigmaRosa]

I'd be cautious about making implementation ease the only factor, especially seeing as some companies request ASVS testing against a certain level. Besides the point, getting away from risk feels dangerous to me as a pentester.

[csfreak92]

Gotcha @tghosth, yeah the value the requirement brings also weighs into it. I think what you said there as L1 would be the most common security issues that some particular requirements are addressing, hence the basic security hygiene for any application to have.

How about we still keep risk relation somewhere but not in correlation to the ASVS levels (L1, L2, and L3)? How to do it, I don't know yet. I am open to suggestions @EnigmaRosa. :) In the past, what we did was as pentesters using ASVS-specific level of testing, we do a threat model exercise first (which I know is beyond ASVS usage/functions) with the client to determine their risk appetite and then go from there. I think that can help with that scenario that you were referring to as some companies would want a risk-based testing using ASVS? I understand that getting away from risk would be a bit problematic, but not everyone's tolerance level/risk level is the same. Some applications and their organizations might not care about a particular requirement but it is in L1, but for some it matters to them. I think this was one of the discussions we had at the WG meeting.

[tghosth]

The risk which the requirement addresses is part of the value which the requirement brings. The value of implementing requirements to address common risks is higher than the value to implement less common risks.

I think maybe I was not definite enough in my response to @csfreak92 as it is definitely not just about ease of implementation but rather ease of implementation compared to the value the requirement brings.

This should hopefully reassure @EnigmaRosa.

@csfreak92 companies are always able to provide a perspective on requirements which are more or less important to themselves.

[csfreak92]

Ah, thanks for clarifying @tghosth, yes I totally agree that it should not just boil down to ease of implementation but rather ease of implementation compared to the value the requirement brings. Kind of like a cost vs benefit analysis type of thing.

And yes, ideally companies using ASVS should be able to provide a perspective on the requirements that apply to themselves, else they must consult another party who can guide them on it (this is beyond the scope of ASVS, but just saying that's another way to help them deal with their risk relation).

[jmanico]

I’d like to suggest that we stick only two levels like MASVS. Level 1 - Requirements that every app must follow If they use that type of tech. Level 2 - Requirements for high risk apps like critical infrastructure, if they tech applies. This is a pure risk based approach. It would simplify the levels. AppSec is hard. The beauty of ASVS is its thoroughness. Now that we have many folks who care about ASVS we can drop level 1 since it’s really only a “best effort” and not a complete approach. I feel that “difficulty of testing or developing” a certain issue is relevant but only as metadata, not as a driver for “what must be done”.

[tghosth]

I'm sorry but I strongly disagree.

I think that if nothing else L1 is important as a gateway and a way to ease into ASVS. Also, practically speaking there are some requirements that will be better value than others #1839 (reply in thread).

MASVS seem to have have almost an order of magnitude less requirements than ASVS so I am not sure it is a relevant comparison right now.

[elarlang]

However many we will have those levels, the most important thing is that we must be able to provide a "flowchart" or a "decision tree" to decide, which requirement belongs to which level.

For (really simplified) example:

• L1 - it should cover most of the requirements
  • 1st layer of defense, except if it is complicated to implement compared to provided risk mitigation
  • 2nd layer of defense, only if it is easy to implement and/or mitigates a lot of risk
• L2
  • all that did not meet to be L1

[tghosth]

Agree that we should try and solidify this a bit

[craig-shony]

Is there any wisdom to snapping the levels to NIST's aal1, aal2, aal3? Very similar to @tghosth proposal but L1 would likely be a little more loose

[tghosth]

There is some loose mapping there for V2 and V3 but honestly I think that trying to strictly align with NIST has made ASVS harder to implement, not easier. See separate discussion here: #1833

[craig-shony]

I agree mapping ASVS requirements to NIST requirements would complicate things. I'm curious if ASVS could match the NIST leveling though. There could be a danger in ASVS having something in L1 and NIST have a comparable in aal2, so I see the cons.
Looking over it NIST doesn't have clear use case definitions for it's leveling anyway.