2.2.1 seems too busy for a single requirement

[tghosth]

There seems to be too much going on in this requirement, I think it needs to be trimmed down or split up or something....

#	Description	L1	L2	L3	CWE	NIST §
2.2.1 (Current)	[MODIFIED] Verify that anti-automation controls are effective at mitigating breached credential testing, brute force, and account lockout attacks. Such controls include blocking the most common breached passwords, soft lockouts, rate limiting, CAPTCHA, ever increasing delays between attempts, IP address restrictions, or risk-based restrictions such as location, first login on a device, recent attempts to unlock the account, or similar. More than 5 failed authentication attempts per hour for a single account should trigger some sort of reaction or alert.	✓	✓	✓	307	5.2.2 / 5.1.1.2 / 5.1.4.2 / 5.1.5.2
2.2.1 v4.0.3	Verify that anti-automation controls are effective at mitigating breached credential testing, brute force, and account lockout attacks. Such controls include blocking the most common breached passwords, soft lockouts, rate limiting, CAPTCHA, ever increasing delays between attempts, IP address restrictions, or risk-based restrictions such as location, first login on a device, recent attempts to unlock the account, or similar. Verify that no more than 100 failed attempts per hour is possible on a single account.	✓	✓	✓	307	5.2.2 / 5.1.1.2 / 5.1.4.2 / 5.1.5.2

[belalena]

I'm glad to see this requirement being discussed and refined. I want to share some ideas about it.

1. The part "Such controls include blocking the most common breached passwords" is covered by the following requirements:
   2.1.7 [MODIFIED, SPLIT TO 2.1.14] Verify that passwords submitted during account registration or password change are checked against an available set of, at least, the top 3000 passwords. (C6)
   2.1.14 [ADDED, SPLIT FROM 2.1.7, LEVEL L1 > L3] Verify that passwords submitted during account registration or password changes are checked against a set of breached username and password pairs. (C6)
   I assume it should be removed from 2.2.1

2. For the part about IP address restrictions - it required to utilize the real IP only for L2. But the current requirement is L1.
   1.14.8 [ADDED] Verify that the application is able to discern and utilizes the user's true IP address to provide for sensitive functions, including rate limiting and logging. ✓ ✓ 348
   Seems it should be moved to a new requirement and be required for A2 and A3

3. For about rate limiting, ever increasing delays between attempts,
   Does it mean per user or per IP, or in a different manner?

4. More than 5 failed authentication attempts per hour for a single account should trigger some sort of reaction or alert.
   "Some sort of" :) doesn't it sound too blurred to be a requirement? A clear response should be defined as it's not possible to assess if the requirement is covered or not.

5. Regarding Soft lockouts
   Consider the scenario where all users, including administrators, are locked out.
   Is it sufficient to rely on automated unlocking after a specific period, or should there be a requirement for manual admin intervention?
   What about Admin's account? Is it reasonable to expect a separate requirement to specify the need of a reliable workaround mechanism to unlock Admin, especially if unlocking is possible only by Admin.
   While it's understood that the choice depends on the specific application and its security needs, why not to split it at least to L1, L2 and L3? For instance:

• L1: No lockout.
• L2: A lockout with automated unlocking mechanisms after some time.
• L3: A lockout with manual unlocking by administrators + a reliable workaround mechanism to unlock Admin

[elarlang]

#	Description	L1	L2	L3	CWE	NIST §
2.2.9	[ADDED, SPLIT FROM 2.2.4] Verify that multi-factor authentication is required, that is, the application uses either a multi-factor authenticator or a combination of single-factor authenticators.		✓	✓	308	4.2.1

It all should be reviewed with knowledge, that we require MFA (from level 2):

• in the past - if you guessed someones password, you got access to the application
• now - if you guess someones password, you just know the password for the application (one of this person password) but MFA requirement must give attacker away from accessing the application

This should put everything to different perspective.

[elarlang]

I provide my views and understandings.

1. The part "Such controls include blocking the most common breached passwords" is covered by the following requirements: 2.1.7, 2.1.14

We need to watch it separately - 2.1.7 and 2.1.14 blocks users to use it, which makes it even more suspicious, when someone start to use those "can not be used" passwords for guessing. So it makes sense to block them.

2. For the part about IP address restrictions - it required to utilize the real IP only for L2. But the current requirement is L1.

L2 for 1.14.8 is taken based on "blackbox" logic, that for being able verify the correct IP, verifier probably need more access than just blackbox test.

With guessing password - it needs to block repeatitive attempts from one source (IP). The side effect here can be, that if source IP is taken incorrectly, it blocks too much.

3. For about rate limiting, ever increasing delays between attempts,

Personally I'm not sure that this ever increasing is actually needed. The point for the requirement is to take down online guessing vector, if it is already few attempts per time X, it is too slow to cause the damage. If an attacker is really into it, he/she can find another IP for next few attempts.

The question itself is valid and I asked the same last time, when current 2.2.1 was developed: #906 (comment)

4. some sort of alert

We have requirements to cover this part, like:

• V8.1.4 Verify the application can detect and alert on abnormal numbers of requests, such as by IP, user, total per hour or day, or whatever makes sense for the application.
• V11.1.7 Verify that the application monitors for unusual events or activity from a business logic perspective. For example, attempts to perform actions out of order or actions which a normal user would never attempt.
• V11.1.8 Verify that the application has configurable alerting when automated attacks or unusual activity is detected.

(related issue #1272)

Consider the scenario where all users, including administrators, are locked out.

There should be strong authentication for unlocking.

Also we have requirement like:

• V4.3.1 [MODIFIED] Verify administrative interfaces can only be logically accessed from trusted endpoints or locations. For example, restricting access to bastion or jump hosts, trusted admin workstations or endpoints (e.g., device authentication), administrative LANs, etc.

L3: A lockout with manual unlocking by administrators + a reliable workaround mechanism to unlock Admin

If there is way to lockout accounts, it's easy to lockout entire userbase. I don't think it is a good idea. Basically the requirement itself is against this idea:

Verify that anti-automation controls are effective at mitigating breached credential testing, brute force, and account lockout attacks

For lockouts we need to keep in mind MFA point (#1763 (comment)) - even if attacker is able to guess the password, it still do not provide access to the application.

[belalena]

We need to watch it separately - 2.1.7 and 2.1.14 blocks users to use it, which makes it even more suspicious, when someone start to use those "can not be used" passwords for guessing. So it makes sense to block them.

I have some doubts about it. There are situations to consider, like when a user forgets their password and attempts to guess from a list of their commonly used passwords (even though using the same password for multiple apps isn't secure, some users still do it).
Or there might be cases where the app updates the list of breached passwords or introduces this list for the first time

There should be strong authentication for unlocking.
Also we have requirement like:
V4.3.1 [MODIFIED] Verify administrative interfaces can only be logically accessed from trusted endpoints or locations. For example, restricting access to bastion or jump hosts, trusted admin workstations or endpoints (e.g., device authentication), administrative LANs, etc.

It's great to have this requirement in place. When the requirement is met in the app, it ensures that there's no chance to block the admin account. However, what if, in the process of achieving 100% coverage, this requirement isn't met, and a mechanism for blocking accounts is developed? In that case, it could be a good idea to include a comment about adding a workaround for admin account.
It's mostly the comments about L1 where MFA is not required.

For lockouts we need to keep in mind MFA point (#1763 (comment)) - even if attacker is able to guess the password, it still do not provide access to the application.

MFA is still not required for L1 applications.

[jmanico]

Let's also keep in mind that numerical-based multi-factor can be brute-forced. We're seeing a lot of attacks that successfully pop 6-digit SMS MFA codes with relative ease.

[tghosth]

Ok, I reviewed the useful thoughts from @belalena and @elarlang.

You can skip to the tl;dr below

---

My rough thoughts were as follows:

• Blocking the most common breached password seems out of place. Elar says it is a good signal for detecting attacks but I think that is a little too detailed for the standard.
• IP address restrictions - again too prescriptive and raises a big can of worms about shared IPs.
• Rate limiting concept is good but too much detail will cause confusion
• "Some sort of response" - yeah this sounds bad (I wrote it) Elar points out that we are already covering this elsewhere
• I would rather not touch lockouts at all.

Long story short I hate this requirement, it is far too long and far too specific in a way that I don't think is helpful for implementation.

The original point from NIST was for rate limiting. This current requirement spills into some specific requirements that I want to get rid of and also adaptive responses which NIST does include but I would prefer to rely on the other monitoring controls discussed here for this aspect.

I think we need to have some sort of requirement for this as it is an important area but it needs to be shorter and less prescriptive.

---

tl;dr

As such, I would simplify to:

[MODIFIED] Verify that the application prevents attacks such as credential stuffing, brute force, and account lockout attacks using rate limiting or other anti-automation and adaptive response controls and that these controls are included in security logging and alerting.

Opinions @jmanico @elarlang

[elarlang]

Direction is good - just giving principles without sending the "cheat sheet content" in the requirement, but I think there are 2 separate requirements:

• Verify that you have a anti-automation solution for online authentication form.
• Verify that you don't lock out users or have mechanism how to unlock the account (such as using MFA or strong-authentication)

[tghosth]

Is then 2nd one important enough to deserve its own requirement?

[elarlang]

Is then 2nd one important enough to deserve its own requirement?

yes

[tghosth]

Is then 2nd one important enough to deserve its own requirement?

yes

So like:

Verify that malicious users cannot lock out legitimate users and admins through excessive incorrect login attempts?

[elarlang]

Let's keep the discussion over lockout in a separate issue: #2134 and keep focus here to find the wording for current 2.2.1

[tghosth]

So as I noted here, I would simplify 2.2.1 to:

[MODIFIED] Verify that the application prevents attacks such as credential stuffing, brute force, and account lockout attacks using rate limiting or other anti-automation and adaptive response controls and that these controls are included in security logging and alerting.

Opinions @jmanico @elarlang

[elarlang]

Credential stuffing and brute force are attacker actions to do. Account lockout is (maybe bad) developer choice when such attack is detected - I think those should not be put in the list like those are all the same thing. One is cause and other is impact.

[tghosth]

Are

Credential stuffing and brute force are attacker actions to do. Account lockout is (maybe bad) developer choice when such attack is detected - I think those should not be put in the list like those are all the same thing. One is cause and other is impact.

Ok, check out #2148

[elarlang]

Are
Is there a missing question?

Current proposal:

Verify that the application prevents attacks such as credential stuffing and password brute force using rate limiting or other anti-automation and adaptive response controls and that these controls are included in security logging and alerting.

It is a bit hard to read, 4 times "and" in it. Maybe it should be split to different sentences to provide easier reading.

... and that these controls are included in security logging and alerting.

Hiting some limit is a security event by default and I think it is covered by current 7.2.1 (and it some point is not, it should belong there):

V7.2.1 Verify that all authentication operations are logged including both successful and unsuccessful attempts. Additional metadata such as type of authentication or factors used should also be collected.

[jmanico]

Here are two suggestions.

One with logging:

Verify that the application prevents credential stuffing and password brute-force attacks by using rate limiting, anti-automation measures, or adaptive response controls. Ensure that these security measures are included in the security logging and alerting system.

One without logging

Verify that the application prevents credential stuffing and password brute-force attacks by implementing rate limiting, anti-automation measures, or adaptive response controls.

[elarlang]

Current requirement has

More than 5 failed authentication attempts per hour for a single account should trigger some sort of reaction or alert.

We should keep in mind, that the requirement must be clear for how to test it. If it does not provide clear limits itself, it must ask it to be decided with security decision documentation requirements.

[tghosth]

I think maybe I would just simplify my suggestion to:

Verify that the application prevents attacks such as credential stuffing and password brute force using rate limiting or other anti-automation and adaptive response controls

Thoughts @elarlang @jmanico ?.

[jmanico]

I like it Josh

[elarlang]

I think the requirement can still be misinterpreted - I use brackets to highlight it:

• Verify that the application prevents (attacks such as credential stuffing and (password brute force using rate limiting) or other anti-automation and adaptive response controls).
  • prevents anti-automation and adaptive response controls

Proposal direction:

• Verify that the application uses rate limiting, anti-automation, adaptive response controls, or other defense methods against attacks such as credential stuffing and password brute force.

But this one is not PR-ready as well:

• what do we mean by rate-limiting - testability problem: what is the limit? for what the limit applies? (user-account, source ip)

This is probably one of the most tested requirements for simple applications, we need to be clear here.

[tghosth]

I don't think we can be too prescriptive here about the rate limiting. I think the deciding factor is whether it makes the attack impractical or not. @elarlang

[elarlang]

If it is wide opened it can not be implemented and tested. On option is to use the same way as we did with session - decide your limits, document them, implement them.

[tghosth]

Ok @elarlang so how about:

#	Description	L1	L2	L3	CWE	NIST §
1.2.6	[ADDED, SPLIT FROM 2.2.1] Verify that application documentation defines how controls such as rate limiting, anti-automation, and adaptive response, are used to defend against attacks such as credential stuffing and password brute force. The documentation should make clear how these controls are configured.	✓	✓	✓	307
2.2.1	[MODIFIED, SPLIT TO 1.2.6] Verify that controls to prevent attacks such as credential stuffing and password brute force are implemented according to the application's security documentation.	✓	✓	✓	307	5.2.2