Skip to content

Commit

Permalink
docs: 1.5.0 version docs (#115)
Browse files Browse the repository at this point in the history
* docs for the 1.5.0 version

* wording fix

* explain HTTP request load balancing
  • Loading branch information
rabbitstack committed Apr 29, 2022
1 parent d4c2f6e commit 3fd25bf
Show file tree
Hide file tree
Showing 12 changed files with 535 additions and 20 deletions.
2 changes: 1 addition & 1 deletion docs/_coverpage.md
Expand Up @@ -4,7 +4,7 @@
<img src='logo.png'></img>
</div>

# fibratus <small>1.4.2</small>
# fibratus <small>1.5.0</small>

> A modern tool for the Windows kernel exploration and observability
Expand Down
2 changes: 2 additions & 0 deletions docs/_sidebar.md
Expand Up @@ -37,6 +37,8 @@
* [Null](outputs/null.md)
* [RabbitMQ](outputs/rabbitmq.md)
* [Elasticsearch](outputs/elasticsearch.md)
* [HTTP](outputs/http.md)
* [Eventlog](outputs/eventlog.md)
* <ion-icon name="color-wand-outline"></ion-icon> Transformers
* [Parsing, Enriching, Transforming](transformers/introduction.md)
* <ion-icon name="remove-circle-outline"></ion-icon> [Remove](transformers/remove.md)
Expand Down
20 changes: 20 additions & 0 deletions docs/filters/fields.md
Expand Up @@ -47,7 +47,12 @@ The following tables summarize available field names that can be used in filter
| ps.args | Process command line arguments | `ps.args in ('/cdir', '/-C')` |
| ps.cwd | Process current working directory | `ps.cwd = 'C:\\Users\\Default'` |
| ps.sid | Security identifier under which this process is run | `ps.sid contains 'SYSTEM'` |
| ps.domain | Process domain name | `ps.domain = 'NT AUTHORITY'` |
| ps.username | Process user name | `ps.username = 'SYSTEM'` |
| ps.sessionid | Unique identifier for the current session | `ps.sessionid = 1` |
| ps.access.mask | Process access rights | `ps.access.mask = '0x1000'` |
| ps.access.mask.names | Process access human-readable rights | `ps.access.mask.names in ('TERMINATE', 'QUERY_INFORMATION')` |
| ps.access.status | Process access status | `ps.access.status = 'success'` |
| ps.envs | Process environment variables | `ps.envs in ('MOZ_CRASHREPORTER_DATA_DIRECTORY')` |
| ps.envs[] | Accesses a specific environment variable. Prefix matches are supported | `ps.envs['MOZ_CRASHREPORTER'] = 'C:\\Program Files\\Firefox'` |
| ps.dtb | Process directory table base address | `ps.dtb = '7ffe0000'` |
Expand All @@ -56,16 +61,27 @@ The following tables summarize available field names that can be used in filter
| ps.modules | Modules loaded by the process | `ps.modules in ('crypt32.dll', 'xul.dll')` |
| ps.modules[] | Accesses a specific process module. Prefix matches are supported | `ps.modules['crypt'].size > 1024` |
| ps.parent.name | Parent process name | `ps.parent.name = 'powershell.exe'` |
| ps.parent.pid | Parent process identifier | `ps.parent.pid = 2340` |
| ps.parent.comm | Parent process command line | `ps.parent.comm contains 'attrib'` |
| ps.parent.exe | Full name of the parent process executable | `ps.parent.exe = 'C:\\Windows\\system32\\cmd.exe'` |
| ps.parent.cwd | Parent process current working directory | `ps.parent.cwd = 'C:\\Users\\Default'` |
| ps.parent.sid | Security identifier under which the parent process is run | `ps.parent.sid contains 'SYSTEM'` |
| ps.parent.domain | Parent process domain name | `ps.parent.domain = 'NT AUTHORITY'` |
| ps.parent.username | Parent process user name | `ps.parent.username = 'SYSTEM'` |
| ps.parent.sessionid | Unique identifier for the current session of the parent process | `ps.parent.session = 1` |
| ps.parent.dtb | Parent process directory table base address | `ps.parent.dtb = 'powershell.exe'` |
| ps.parent.envs | Parent process environment variables | `ps.parent.envs in ('PROCESSOR_LEVEL')'` |
| ps.parent.handles | Allocated parent process handles | `ps.parent.handles in ('\\...\\Cor_SxSPublic_IPCBlock')` |
| ps.parent.handle.types | Allocated parent process handles types | `ps.parent.handle.types in ('Key', 'Mutant', 'Section')` |
| ps.ancestor[] | Process ancestry traversing | `ps.ancestor[2].name in ('winword.exe', 'powershell.exe')` |
| ps.sibling.name | Sibling process name | `ps.sibling.name = 'cmd.exe'` |
| ps.sibling.pid | Sibling process identifier | `ps.sibling.id = 6050` |
| ps.sibling.comm | Sibling process command line | `ps.sibling.name contains '/k /v'` |
| ps.sibling.exe | Sibling process executable full path | `ps.sibling.exe = 'C:\\Windows\\system32\\cmd.exe'` |
| ps.sibling.sid | Sibling process security identifier | `ps.sibling.sid contains 'SYSTEM'` |
| ps.sibling.sessionid | Sibling process session identifier | `ps.sibling.sessionid = 1` |
| ps.sibling.domain | Sibling process domain name | `ps.sibling.domain = 'NT AUTHORITY'` |
| ps.sibling.username | Sibling process user name | `ps.sibling.username = 'SYSTEM'` |

### Thread
| Field Name | Description | Example |
Expand All @@ -78,6 +94,10 @@ The following tables summarize available field names that can be used in filter
| thread.ustack.base | Base address of the thread's user space stack | `thread.ustack.base = '7ffe0000'` |
| thread.ustack.limit | Limit of the thread's user space stack | `thread.ustack.limit = '8ffe0000'` |
| thread.entrypoint | Starting address of the function to be executed by the thread | `thread.entrypoint = '7efe0000'` |
| thread.access.mask | Thread access rights | `thread.access.mask = '0x1800'` |
| thread.access.mask.names | Thread access human-readable rights | `thread.access.mask.names in ('QUERY_LIMITED_INFORMATION')` |
| thread.access.status | Thread access status | `thread.access.status = 'success'` |


### Image
| Field Name | Description | Example |
Expand Down
251 changes: 247 additions & 4 deletions docs/filters/functions.md
@@ -1,6 +1,8 @@
# Functions

Functions expand the scope of the filtering language by bringing a plethora of capabilities. The function can return a primitive value, including integers, strings, and booleans. Additionally, some functions may return a collection of values. Function names are case insensitive.
Functions expand the scope of the filtering language by bringing a plethora of capabilities. The function can return a primitive value, including integers, strings, and booleans. Function calls can be nested where the result of one function is used as an input in another function. For example, `lower(ltrim(file.name, 'C:'))`, removes the `C` drive letter from the file path and converts it to a lower case string.

Additionally, some functions may return a collection of values. Function names are case insensitive.

### Network functions

Expand All @@ -18,7 +20,7 @@ Functions expand the scope of the filtering language by bringing a plethora of c

- **Examples**

Assuming the `net.sip` contains the `192.168.1.20` IP address, the following filter
Assuming the `net.sip` field contains the `192.168.1.20` IP address, the following filter
would match on this event.

```
Expand All @@ -40,8 +42,249 @@ Functions expand the scope of the filtering language by bringing a plethora of c

- **Examples**

Assuming the `registry.key.name` contains `HKEY_LOCAL_MACHINE\SYSTEM\Setup\Pid`, the following would filter events for the matching md5 hash.
Assuming the `registry.key.name` field contains `HKEY_LOCAL_MACHINE\SYSTEM\Setup\Pid`, the following would filter events for the matching md5 hash.

```
fibratus run kevt.category = 'net' and md5(registry.key.name) = 'eab870b2a516206575d2ffa2b98d8af5'
```
```

### String functions

#### concat

`concat` concatenates string/integer input arguments.

- **Specification**
```
concat(args: <string|int>...) :: <string>
```
- `args`: Strings or integers to be concatenated. This function requires at least 2 input arguments
- `return` a concatenated string of all input arguments

- **Examples**

Assuming the `ps.domain` field contains `NT_AUTHORITY` and `ps.username` field contains `admin`, the following would filter events for the matching concatenated string.

```
fibratus run concat(ps.domain, '-', ps.username) = 'NT_AUTHORITY-admin'
```

#### ltrim

`ltrim` trims the specified prefix from a string.

- **Specification**
```
ltrim(string: <string>, prefix: <string>) :: <string>
```
- `string`: Input string
- `prefix`: Prefix that is removed from the original input string
- `return` a string with the specified prefix removed

- **Examples**

Assuming the `registry.key.name` field contains `HKEY_LOCAL_MACHINE\SYSTEM\Setup\Pid`, the following filter expression would match on all events where the resulting string is equal to `SYSTEM\Setup\Pid`

```
fibratus run ltrim(registry.key.name, 'HKEY_LOCAL_MACHINE\\') = 'SYSTEM\\Setup\\Pid'
```

#### rtrim

`rtrim` trims the specified suffix from a string.

- **Specification**
```
rtrim(string: <string>, suffix: <string>) :: <string>
```
- `string`: Input string
- `prefix`: Suffix that is removed from the original string
- `return` a string with the specified suffix removed

- **Examples**

Assuming the `registry.key.name` field contains `HKEY_LOCAL_MACHINE\SYSTEM\Setup\Pid`, the following filter expression would match on all events where the resulting string is equal to `HKEY_LOCAL_MACHINE\SYSTEM\Setup`

```
fibratus run rtrim(registry.key.name, '\\Pid') = 'HKEY_LOCAL_MACHINE\\SYSTEM\\Setup'
```

#### lower

`lower` converts the string with all Unicode letters mapped to their lower case.

- **Specification**
```
lower(string: <string>) :: <string>
```
- `string`: Input string
- `return` a string converted to lower case

- **Examples**

Assuming the `registry.key.name` field contains `HKEY_LOCAL_MACHINE\SYSTEM\Setup`, the following filter expression would match on all events where the resulting string is equal to `hkey_local_machine\system\setup`

```
fibratus run lower(registry.key.name) = 'hkey_local_machine\\system\\setup'
```

#### upper

`upper` converts the string with all Unicode letters mapped to their upper case.

- **Specification**
```
upper(string: <string>) :: <string>
```
- `string`: Input string
- `return` a string converted to upper case

- **Examples**

Assuming the `registry.key.name` field contains `HKEY_LOCAL_MACHINE\SYSTEM\Setup`, the following filter expression would match on all events where the resulting string is equal to `HKEY_LOCAL_MACHINE\SYSTEM\SETUP`

```
fibratus run upper(registry.key.name) = 'HKEY_LOCAL_MACHINE\\SYSTEM\\SETUP'
```

#### replace

`replace` replaces all occurrences in the string as given by arbitrary old/new replacement pairs.

- **Specification**
```
replace(string: <string>, old: <string>, new: <string>, ..., old-n: <string>, new-n: <string>) :: <string>
```
- `string`: Input string
- `old`: substring in the original string that is replaced with the `new` string
- `new`: the replacement string
- `return` a string with all occurrences replaced by old/new pairs

- **Examples**

Assuming the `registry.key.name` field contains `HKEY_LOCAL_MACHINE\SYSTEM\Setup`, the following filter expression would match on all events where the resulting string is equal to `HKLM\SYS\Setup`

```
fibratus run replace(registry.key.name, 'HKEY_LOCAL_MACHINE', 'HKLM', 'SYSTEM', 'SYS') = 'HKLM\\SYS\\Setup'
```

#### split

`split` produces a slice of substrings separated by the given delimiter.

- **Specification**
```
split(string: <string>, sep: <string>) :: <[]string>
```
- `string`: Input string
- `prefix`: The separator that is used to split the string
- `return` a slice of substrings

- **Examples**

Assuming the `file.name` field contains `C:\Windows\System32\kernel32.dll`, the following filter expression would match on all events where the `kernel32.dll` or `System32` strings are present in the resulting slice.

```
fibratus run split(file.name, '\\') in ('kernel32.dll', 'System32')
```

#### length

`length` returns the number of characters for string arguments and the size of the slice for slice arguments.

- **Specification**
```
length(string: <string|slice>) :: <int>
```
- `string`: Input string or slice
- `return` the number of characters or array length

- **Examples**

Assuming the `ps.domain` field contains `"こんにちは"`, the following would filter events with 5 symbols in the process domain.

```
fibratus run length(ps.domain) = 5
```

#### indexof

`indexof` returns the index of the instance of substring in a given string depending on the provided search order.

- **Specification**
```
indexof(string: <string>, substring: <string>, order: <string>) :: <int>
```
- `string`: Input string
- `prefix`: Substring for which the search is performed
- `order`: Specifies the string index search order. It can be `first`, `any`, `last`, `lastany`. This is an optional argument.
- `return` the index of the substring

- **Examples**

Assuming the `ps.domain` field contains `NT_AUTHORITY`, the following would filter events for the matching substring index.

```
fibratus run indexof(ps.domain, 'NT') = 0
```

#### substr

`substr` creates a substring of a given string.

- **Specification**
```
substr(string: <string>, start: <int>, end: <int>) :: <string>
```
- `string`: Input string
- `start`: Substring start index
- `end`: Substring end index
- `return` a substring contained within start and end indices

- **Examples**

Assuming the `file.name` field contains `\Device\HarddiskVolume2\Windows\system32\user32.dll`, the following filter expression would match on all events where the substring is equal to `\Device`

```
fibratus run substr(file.name, indexof(file.name, '\\'), indexof(file.name, '\\Hard')) = '\\Device'
```

#### entropy

`entropy` measures the string entropy.

- **Specification**
```
entropy(string: <string>, algo: <string>) :: <int>
```
- `string`: Input string
- `algo`: The algorithm used to calculate the string entropy. `shannon` is the default entropy type. This argument is optional
- `return` the string entropy

- **Examples**

Assuming the `file.name` field contains `\Device\HarddiskVolume2\Windows\system32\user32.dll`, the following filter expression would match on all events where file name entropy is greater than 255.

```
fibratus run entropy(file.name) > 255
```

#### regex

`regex` applies single/multiple regular expressions on the provided string argument.

- **Specification**
```
regex(string: <string>, patterns: <string>...) :: <bool>
```
- `string`: Input string
- `patterns`: Regular expression patterns
- `return` `true` if at least one regular expression matches or `false` otherwise

- **Examples**

Assuming the `ps.name` field contains `powershell.exe`, the following would filter events matching the regular expressions.

```
fibratus run regex(ps.name, 'power.*(shell|hell).dll', '.*hell.exe')
```
13 changes: 13 additions & 0 deletions docs/kevents/process.md
@@ -1,5 +1,7 @@
# Process events

#### CreateProcess and TerminateProcess

Process events are fired up as a stimulus to the process' life-cycle changes. When the kernel puts into motion a process or terminates it, the `CreateProcess` and `TerminateProcess` events are emitted respectively. The following list summarizes all the distinct event parameters that are associated with process events.

- `pid` is the process' identifier. This value is valid from the time a process is created until it is terminated.
Expand All @@ -15,6 +17,17 @@ Process events are fired up as a stimulus to the process' life-cycle changes. Wh
- `status` is the exit status of the stopped process.
- `start_time` designates the instant when the process was started.

#### OpenProcess

`OpenProcess` event is triggered when a process tries to acquire an existing local process object. This event contains the following parameters:

- `desired_access` is the hexadecimal value that represents the desired access to the process object.
- `desired_access_names` is the list of human-readable desired access strings (e.g. `TERMINATE,QUERY_INFORMATION`). For a full list and detailed explanation of available access rights, head to the official [docs](https://docs.microsoft.com/en-us/windows/win32/procthread/process-security-and-access-rights).
- `name` is the name of the local process that was opened.
- `exe` is the full path of the local process object that was open.
- `pid` is the identifier of the local process that was opened.
- `status` contains the result of the process object open operation. (e.g. `success`)

### Process state {docsify-ignore}

Fibratus keeps a snapshot of all running processes including their state such as basic process attributes, allocated file handles, dynamically-linked libraries, PE (Portable Executable) metadata and other resources. The snapshot is updated dynamically as processes get spawn or die. Each time a kernel event is triggered, its process' state is fetched from the snapshot and attached to the event. This state machine semantically enriches each individual event with the aim on providing a powerful context for [filtering](/filters/introduction.md) and [scripting](/filaments/introduction.md).
Expand Down
15 changes: 15 additions & 0 deletions docs/kevents/thread.md
@@ -1,5 +1,7 @@
# Thread events

#### CreateThread and TerminateThread

Thread events notify the creation (`CreateThread`) or termination (`TerminateThread`) of threads within the process' address space. In situations where a process spawns a remote thread that runs in other process' address space, the `CreateThread` event is also triggered.
Thread events are comprised of the following parameters:

Expand All @@ -13,3 +15,16 @@ Thread events are comprised of the following parameters:
- `kstack_base` is the base address of the thread's kernel space stack.
- `kstack_limit` is the limit of the thread's kernel space stack.
- `entrypoint` is the starting address of the function to be executed by the thread.

#### OpenThread

`OpenProcess` event is triggered when a process opens an existing local thread object. This event contains the following parameters:

- `desired_access` is the hexadecimal value that represents the desired access to the thread object.
- `desired_access_names` is the list of human-readable desired access strings (e.g. `QUERY_LIMITED_INFORMATION`). For a full list and detailed explanation of available access rights, head to the official [docs](https://docs.microsoft.com/en-us/windows/win32/procthread/thread-security-and-access-rights).
- `name` is the name of the local process whose thread object was open.
- `exe` is the full path of the local process image whose thread object was open.
- `pid` is the identifier of the local process whose thread object was opened.
- `tid` is the identifier of the local thread that was opened.
- `status` contains the result of the thread object open operation. (e.g. `success`)

0 comments on commit 3fd25bf

Please sign in to comment.