Skip to content
This repository has been archived by the owner on Feb 13, 2024. It is now read-only.

Commit

Permalink
Move documentation to wiki pages
Browse files Browse the repository at this point in the history
  • Loading branch information
@dwisiswant0
dwisiswant0 committed Mar 9, 2021
1 parent 9e71529 commit 411715b
Showing 1 changed file with 5 additions and 332 deletions.
337 changes: 5 additions & 332 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,33 +35,7 @@
- [Features](#features)
- [Why teler?](#why-teler)
- [Demo](#demo)
- [Installation](#installation)
- [from Binary](#from-binary)
- [using Docker](#using-docker)
- [from Source](#from-source)
- [from GitHub](#from-github)
- [Usage](#usage)
- [Flags](#flags)
- [Config](#config)
- [Input](#input)
- [Concurrency](#concurrency)
- [Output](#output)
- [JSON Format](#json-format)
- [Remove Caches](#remove-caches)
- [Configuration](#configuration)
- [Log formats](#log-formats)
- [Apache](#apache)
- [Nginx](#nginx)
- [Nginx Ingress](#nginx-ingress)
- [Amazon S3](#amazon-s3)
- [Elastic LB](#elastic-lb)
- [CloudFront](#cloudfront)
- [Threat rules](#threat-rules)
- [Excludes](#excludes)
- [Whitelists](#whitelists)
- [Notification](#notification)
- [Metrics](#metrics)
- [Prometheus](#prometheus)
- [Documentation](#documentation)
- [Supporting Materials](#supporting-materials)
- [Contributors](#contributors)
- [Resources](#resources)
Expand Down Expand Up @@ -99,312 +73,11 @@ Here is a preview of `teler` with conditions of use as:
|-------------------- |----------------- |
| <a href="https://asciinema.org/a/367616" alt="teler"><img src="https://asciinema.org/a/367616.svg"></a> | <a href="https://asciinema.org/a/367610" alt="teler"><img src="https://asciinema.org/a/367610.svg"></a> |

## Installation
## Documentation

### from Binary

The installation is easy. You can download a prebuilt binary from [releases page](https://github.com/kitabisa/teler/releases), unpack and run! or run with:

```bash
▶ curl -sSfL 'https://ktbs.dev/get-teler.sh' | sh -s -- -b /usr/local/bin
```

### using Docker

Pull the Docker image by running:

```bash
▶ docker pull kitabisa/teler
```

### from Source

If you have go1.14+ compiler installed and configured:

```bash
▶ GO111MODULE=on go get -u ktbs.dev/teler/cmd/teler
```

The same command works for update to the latest version.

### from GitHub

```bash
▶ git clone https://github.com/kitabisa/teler
▶ cd teler
▶ make build
▶ mv ./bin/teler /usr/local/bin
```

## Usage

Simply, teler can be run with:

```bash
▶ [buffers] | teler -c /path/to/config/teler.yaml
# or
▶ teler -i /path/to/access.log -c /path/to/config/teler.yaml
```

If you've built teler with a Docker image:

```bash
▶ [buffers] | docker run -i --rm -e TELER_CONFIG=/path/to/config/teler.yaml kitabisa/teler
# or
▶ docker run -i --rm -e TELER_CONFIG=/path/to/config/teler.yaml kitabisa/teler --input /path/to/access.log
```

### Flags

```bash
▶ teler -h
```

This will display help for the tool.

<p align="center">
<a href="#"><img alt="teler" src="https://user-images.githubusercontent.com/25837540/97946822-164bf300-1dbe-11eb-8f54-3a25ee0eb80e.png" /></a>
</p>

Here are all the switches it supports.

| Flag | Description | Examples |
|---------------------- |------------------------------------------------------------- |--------------------------------------------------------- |
| -c,<br> --config | teler configuration file | kubectl logs nginx \| teler -c /path/to/config/teler.yaml |
| -i,<br> --input | Analyze logs from data persistence rather than buffer stream | teler -i /var/log/nginx/access.log |
| -x,<br> --concurrent | Set the concurrency level to analyze logs<br>(default: 20) | tail -f /var/log/nginx/access.log \| teler -x 50 |
| -o,<br> --output | Save detected threats to file | teler -i /var/log/nginx/access.log -o /tmp/threats.log |
| --json | Display threats in the terminal as JSON format | teler -i /var/log/nginx/access.log --json |
| --rm-cache | Remove all cached resources | teler --rm-cache |
| -v,<br> --version | Show current teler version | teler -v |

#### Config

The `-c` flag is to specify teler configuration file.

```bash
▶ tail -f /var/log/nginx/access.log | teler -c /path/to/config/teler.yaml
```

This is **required**, but if you have defined `TELER_CONFIG` environment you don't need to use this flag, e.g.:

```bash
▶ export TELER_CONFIG="/path/to/config/teler.yaml"
▶ tail -f /var/log/nginx/access.log | teler
# or
▶ tail -f /var/log/nginx/access.log | TELER_CONFIG="/path/to/config/teler.yaml" teler
```

#### Input

Need log analysis incrementally? This `-i` flag is useful for that.

```bash
▶ teler -i /var/log/nginx/access.log
```

#### Concurrency

Concurrency is the number of logs analyzed at the same time. Default value teler provide is 20, you can change it by using `-x` flag.

```bash
▶ teler -i /var/log/nginx/access.log -x 50
```

#### Output

You can also save the detected threats into a file with `-o` flag.

```bash
▶ teler -i /var/log/nginx/access.log -o threats.log
```

#### JSON Format

If you want to display the detected threats as JSON format, switch it with `--json` flag.

```bash
▶ teler -i /var/log/nginx/access.log --json
```

Please note this will also apply if you save it to a file with `-o` flag.

#### Remove Caches

It will removes all stored resources in the user-level cache directory, see [cache](#cache).

```bash
▶ teler --rm-cache
```

## Configuration

`teler` requires a minimum of configuration to process and/or log analysis, and execute threats and/or alerts. See [teler.example.yaml](https://github.com/kitabisa/teler/blob/development/teler.example.yaml) for an example.

### Log Formats

Because we use `gonx` package to parse the log, you can write any log format. As an example:

#### Apache
```yaml
log_format: |
$remote_addr - $remote_user [$time_local] "$request_method $request_uri $request_protocol" $status $body_bytes_sent "$http_referer" "$http_user_agent"
```

#### Nginx
```yaml
log_format: |
$remote_addr $remote_user - [$time_local] "$request_method $request_uri $request_protocol"
$status $body_bytes_sent "$http_referer" "$http_user_agent"
```

#### Nginx Ingress
```yaml
log_format: |
$remote_addr - [$remote_addr] $remote_user - [$time_local]
"$request_method $request_uri $request_protocol" $status $body_bytes_sent
"$http_referer" "$http_user_agent" $request_length $request_time
[$proxy_upstream_name] $upstream_addr $upstream_response_length $upstream_response_time $upstream_status $req_id
```

#### Amazon S3
```yaml
log_format: |
$bucket_owner $bucket [$time_local] $remote_addr $requester $req_id $operationration $key
"$request_method $request_uri $request_protocol" $status $error_code $body_bytes_sent -
$total_time - "$http_referer" "$http_user_agent" $version_id $host_id
$signature_version $cipher_suite $http_auth_type $http_host_header $tls_version
```

#### Elastic LB
```yaml
log_format: |
$time_local $elb_name $remote_addr $upstream_addr $request_processing_time
$upstream_processing_time $response_processing_time $status $upstream_status $body_received_bytes $body_bytes_sent
"$request_method $request_uri $request_protocol" "$http_user_agent" $cipher_suite $tls_version
```

#### CloudFront
```yaml
log_format: |
$date $time $edge_location $body_bytes_sent $remote_addr
$request_method $http_host_header $requst_uri $status
$http_referer $http_user_agent $request_query $http_cookie $edge_type $req_id
$http_host_header $ssl_protocol $body_bytes_sent $response_processing_time $http_host_forwarded
$tls_version $cipher_suite $edge_result_type $request_protocol $fle_status $fle_encrypted_fields
$http_port $time_first_byte $edge_detail_result_type
$http_content_type $request_length $request_length_start $request_length_end
```

### Threat rules

#### Cache

By default, `teler` will fetch external resources every time you run it, but you can switch external resources to be cached or not.

```yaml
rules:
cache: true
```

If you choose to cache resources, it's stored under user-level cache directory of cross-platform and will be updated every day, see [resources](#resources).

#### Excludes

We include resources for predetermined threats, including:
- Common Web Attack
- CVE
- Bad IP Address
- Bad Referrer
- Bad Crawler
- Directory Bruteforce

You can disable any type of threat in the `excludes` configuration _(case-sensitive)_.

```yaml
rules:
threat:
excludes:
- "Bad IP Address"
```

The above format detects threats that are not included as bad IP address, and will not analyze logs/ send alerts for that type.

#### Whitelists

You can also add whitelists to teler configuration.

```yaml
rules:
threat:
whitelists:
- "(curl|Go-http-client|okhttp)/*"
- "^/wp-login\\.php"
```

It covers the entire HTTP request and processed as _regExp_, please write it with caution!

### Notification

We provide alert notification options:
- Slack,
- Telegram
- Discord

Configure the notification alerts needed on:

```yaml
notifications:
slack:
token: "xoxb-..."
color: "#ffd21a"
channel: "G30SPKI"

telegram:
token: "123456:ABC-DEF1234...-..."
chat_id: "-111000"

discord:
token: "NkWkawkawkawkawka.X0xo.n-kmZwA8aWAA"
color: "16312092"
channel: "700000000000000..."
```

You can also choose to disable alerts or want to be sent where the alerts are.

```yaml
alert:
active: true
provider: "slack"
```

### Metrics

`teler` also supports metrics using Prometheus.

#### Prometheus

You can configure the host, port and endpoint to use Prometheus metrics in the configuration file.

```yaml
prometheus:
active: true
host: "localhost"
port: 9099
endpoint: "/metrics"
```

Here are all the metrics we collected & categorized.

| Metric | Description |
| ---------------------------- | ------------------------------------ |
| `teler_threats_count_total` | Total number of detected threats |
| `teler_cwa` | Get lists of Common Web Attacks |
| `teler_cve` | Get lists of CVE threats |
| `teler_badcrawler` | Get lists of Bad Crawler requests |
| `teler_dir_bruteforce` | Get lists of Directories Bruteforced |
| `teler_bad_referrer` | Get lists of Bad Referrer requests |
| `teler_badip_count` | Total number of Bad IP Addresses |
- [Installation](https://github.com/kitabisa/teler/wiki/Installation)
- [Usage](https://github.com/kitabisa/teler/wiki/Usage)
- [Configuration](https://github.com/kitabisa/teler/wiki/Configuration)

## Supporting Materials

Expand Down

0 comments on commit 411715b

Please sign in to comment.