Learning how to implement owasp-appsensor in go.
https://owasp.org/www-pdf-archive/Owasp-appsensor-guide-v2.pdf
Create a detector/response unit for identifying malicious attempts and taking action against them.
Detection can be as simple as logging the user's identifier (IP address, browser fingerprinting, session id, user id .etc). There should be a count for each request patterns that are detected too. For each attempt, the count should be incremented, and once the threshold has been exceeded, the user would be block from accessing the service. Custom rules can be set there, for example blocking the user for 1hr, or permanently disabling the account. To unblock the user, custom rules can also be applied, such as resetting the count after 1 day, or decrement it hourly. Some examples of the rules are:
- for every attempts, increment linearly
- for every attempts, increment exponentially
- once hit threshold, block
- once block, decrement linearly per hour
- once block, will reset if operators decide to whitelist user
Some generic request patters are:
- invalid request
- rate-limit exceeded
- unauthorized
It can also be specific to the business logic:
- invalid logging attempts for 3 times
- block IP
- block user (blacklist/whitelist)
- alert operation staff
- Generate report
- Produce statistics
- Detect patterns/trends
- Searchable request context (based on context logging)
- Flow of attack
In order to test the effectiveness of the system, it is always good to test it by automating the attacks. There could be a bot that are generating random requests to the system with random payloads (e.g. those generated by faker, or quickcheck. Since the IP is known, we can identify other patterns that are not encountered during testing.
Logging is important to understand what is happening in the system. The one question one has to answer is - what needs to be logged? The golden rule is probably to log everything that is essential to give answers to the questions you are seeking. If there are no questions in mind when desiging the system, logging would not be required.
Logs should be separated from application metrics. Logs should be purely the input and output of the system, be it a requests or errors sent to the stderr. Metrics on the other hand gives information on the capacity of the system (memory, CPU allocation, number of connections etc). While they may come hand to hand, when we talk about logs, we focus more on the input/output of the application.
There are some problem with logging everything though. Logging unnecessary information would lead to noise in the system. Also, the information logged may evolve when the system changes (e.g, the log message may change). In order to minimize the changes, especially when we designed dashboards that captures specific log messages, or event sourcing system where the event name should be static, the requirements should be defined carefully. Versioning logs would be helpful.
Requirements are still often subject to changes though. And most of the time, we cannot derive the value at the beginning of the system. We can always choose to log all the raw requests and response (through a middleware if it's a http request), and log more specific requirements later in the system, hence maintaining both copy.
AppSensor design is probably an extension of the idea. We design more funnels (think it as a filter in a pipeline) which filters more specific business requirements that we are interested to know and can derive value from.
It would be interesting to add a scoring system for users too (perhaps a column that keeps track of the user's current score/health etc based on the level of activity). The could be similar to how the credit score system works, it doesn't have to be a value ranging from 0-1 (though the boundaries must be established).
Knowing the user's scores allows us to take more decisive actions like the things to display etc. Think of it as a form or personalization, except that we are probably showing less data to users with bad scores.
References:
- http://credithealthcare.com/blog/how-credit-score-is-calculated/
- https://peoplescu.info/external/pdf/edu/credit-scores.pdf
$ alias dc=docker-compose
$ dc up -d
$ go run main.go
$ curl localhostOutput:
ip is 127.0.0.1%
NOTE: Blocking by clientIP might not be a good solution, because many users can be under the same network. For anonymous users, it is best to generate a unique sessionID and use them to identify the user.
A combination of clientIP + sessionID might work, in the worst case where the sessionID is tampered with, we can fallback to the clientIP.
NOTE: Just using the clientIP might not be a good indicator too. If we want to be specific (e.g. blocking user by the paths they visited), we can include other information such as url path and/or sessionID. This way, we can block users at a more granular level (locally, per path) rather than globally. Of course, if we want to block all users that originate from an IP address, we can just use that as the sole identifier.
Title: Developer add new rule Main success scenarios:
- Developer think of the rule constraints
- Developer writes a new struct for the rule
- Developer supply the max threshold and the block duration for the rule
- Developer supply the ips to be blacklisted
- Developer supply the other conditions for the blacklist