Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

L3af on windows #68

Open
wants to merge 20 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 19 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
85 changes: 85 additions & 0 deletions dev_environment/Prometheus_Grafana_Win.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
## Installing Grafana and Prometheus on Windows System
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Installing Grafana and Prometheus on Windows System
## Installing Grafana and Prometheus on Windows


### Installation of Grafana on Window
evershalik marked this conversation as resolved.
Show resolved Hide resolved

Navigate to https://grafana.com/grafana/download?platform=windows and Download Grafana for windows (https://dl.grafana.com/enterprise/release/grafana-enterprise-9.0.3.windows-amd64.msi)

After installation check `Grafana Service` is running

![Grafana_svc](../images/l3af-on-windows/prometheus_grafana/Grafana_svc.png)

Navigate to http://localhost:3000/login

> Note: The default username and password is `admin`.

![Grafana_Dashboard1](../images/l3af-on-windows/prometheus_grafana/Grafana_Dashboard1.png)

![Grafana_Dashboard2](../images/l3af-on-windows/prometheus_grafana/Grafana_Dashboard2.png)

### Installation of Prometheus as Service in Windows

You can download Prometheus for windows from https://prometheus.io/download/. However, installing prometheus as a service you need to use `NSSM explorer`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
You can download Prometheus for windows from https://prometheus.io/download/. However, installing prometheus as a service you need to use `NSSM explorer`.
You can download Prometheus for Windows from https://prometheus.io/download/. However, installing prometheus as a service you need to use `NSSM explorer`.


So, if you have installed Grafana first then `NSSM explorer` must be downloaded as part of Grafana. You can go to the path where Grafana is installed and can find a folder named as “**svc-9.0.3.0**”. Under this folder you can find `nssm.exe`.

If you have not installed Grafana in your system then you can install Prometheus by downloading `nssm.exe`. You can download from https://nssm.cc/download.

- Navigate the `NSSM.exe` path through command prompt
- Run Below Command:

```bash
nssm.exe install prometheus <The path where prometheus application downloaded>
```

For example:
![Prometheus_Install](../images/l3af-on-windows/prometheus_grafana/Prometheus_Install.png)

Open `service.msc` and you can see prometheus service is installed
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Open `service.msc` and you can see prometheus service is installed
Open `service.msc` and you can see the prometheus service is installed


![service_msc](../images/l3af-on-windows/prometheus_grafana/service_msc.png)

Before starting the prometheus service, install **WMI Exporter**:

For installing WMI Exporter, you need to download `wmi exporter` from https://github.com/prometheus-community/windows_exporter/releases/download/v0.18.1/windows_exporter-0.18.1-amd64.msi

Post installation of `wmi exporter`, you can validate by navigating to http://localhost:9182/

![Win_exporter](../images/l3af-on-windows/prometheus_grafana/Win_exporter.png)

Click on Metrics link

![metrices](../images/l3af-on-windows/prometheus_grafana/metrices.png)

You can also validate `windows_exporter` service is running:

![WMI_exporter_svc](../images/l3af-on-windows/prometheus_grafana/WMI_exporter_svc.png)

Navigate to prometheus `config file path` and open the “**prometheus.yml**”

You need to add job for `wmi exporter` and `l3afd` , as shown below:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
You need to add job for `wmi exporter` and `l3afd` , as shown below:
You need to add a job for `wmi exporter` and `l3afd` , as shown below:


![prometheus_yaml](../images/l3af-on-windows/prometheus_grafana/prometheus_yaml.png)

Now run Run Prometheus service

![prometheus_svc](../images/l3af-on-windows/prometheus_grafana/prometheus_svc.png)

After starting Prometheus service, navigate to http://localhost:9090/
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: sometimes Prometheus is capitalized in this document, sometimes not. be consistent throughout.


![prometheus_dashboard](../images/l3af-on-windows/prometheus_grafana/prometheus_dashboard.png)

Now you can access Prometheus service.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Now you can access Prometheus service.
Now you can access the Prometheus service.


You can also see `l3afd metric` graph using prometheus:

![prometheus_graph](../images/l3af-on-windows/prometheus_grafana/prometheus_graph.png)

## Grafana Dashboard files by l3af:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Grafana Dashboard files by l3af:
## Grafana Dashboard files by L3AF:

- [Check Here](../dev_environment/cfg/grafana/dashboards/)

You can use these `json` files to create dashboard for monitoring of `eBPF programs` in `Grafana`

![grafana_json](../images/l3af-on-windows/prometheus_grafana/grafana_json.png)

#### eBPF program monitoring:
![eBPF_program](../images/l3af-on-windows/prometheus_grafana/eBPF_program.png)
38 changes: 38 additions & 0 deletions dev_environment/cfg/add_payload_win.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
[
{
"host_name": "l3af-local-test",
"iface": "enp0s3",
"bpf_programs": {
"xdp_ingress": [
{
"name": "ratelimiting",
"seq_id": 1,
"artifact": "ratelimiting.exe",
"map_name": "xdp_rl_ingress_next_prog",
"cmd_start": "ratelimiting.exe",
"version": "latest",
"user_program_daemon": true,
"admin_status": "enabled",
"prog_type": "xdp",
"cfg_version": 1,
"start_args": {
"ports": "80,8080,8081",
"rate": "2"
},
"monitor_maps": [
{
"name": "rl_drop_count_map",
"key": 0,
"aggregator": "scalar"
},
{
"name": "rl_recv_count_map",
"key": 0,
"aggregator": "max-rate"
}
]
}
]
}
}
]
11 changes: 11 additions & 0 deletions dev_environment/cfg/delete_payload_win.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
[
{
"bpf_programs": {
"xdp_ingress": [
"ratelimiting"
]
},
"host_name": "l3af-local-test",
"iface": "enp0s3"
}
]
Original file line number Diff line number Diff line change
Expand Up @@ -71,4 +71,4 @@ enabled: false
# san-match-rules: .+l3afd.l3af.io,.*l3af.l3af.io,^l3afd.l3af.io$

[l3af-config-store]
filename: /var/l3afd/l3af-config.json
filename: /var/l3afd/l3af-config.json
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revert?

56 changes: 56 additions & 0 deletions dev_environment/cfg/l3afd_win.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
[DEFAULT]

[l3afd]
pid-file: /var/l3afd/l3afd.pid
datacenter: dc
bpf-dir: /var/l3afd/repo
bpf-log-dir: /var/l3afd/repo
shutdown-timeout: 1s
http-client-timeout: 10s
max-ebpf-restart-count: 3
bpf-chaining-enabled: false
swagger-api-enabled: true
environment: DEV
BpfMapDefaultPath: /sys/fs/bpf

[ebpf-repo]
url: C:/var/l3afd/repo

[web]
metrics-addr: 0.0.0.0:8898
ebpf-poll-interval: 30s
n-metric-samples: 20

[xdp-root]
package-name: xdp-root
artifact: l3af_xdp_root.tar.gz
ingress-map-name: xdp_root_array
command: xdp_root
version: latest
object-file: xdp_root_kern.o
entry-function-name: xdp_root

[tc-root]
package-name: tc-root
artifact: l3af_tc_root.tar.gz
ingress-map-name: tc/globals/tc_ingress_root_array
egress-map-name: tc/globals/tc_egress_root_array
command: tc_root
version: latest
ingress-object-file: tc_root_ingress_kern.o
egress-object-file: tc_root_egress_kern.o
ingress-entry-function-name: tc_ingress_root
egress-entry-function-name: tc_egress_root

[ebpf-chain-debug]
addr: localhost:8899
enabled: false

[l3af-configs]
restapi-addr: localhost:53000

[l3af-config-store]
filename: /var/l3afd/l3af-config.json

[mtls]
enabled: false
50 changes: 50 additions & 0 deletions dev_environment/cfg/prometheus_win.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Sample config for Prometheus.

global:
scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
# scrape_timeout is set to the global default (10s).

# Attach these labels to any time series or alerts when communicating with
# external systems (federation, remote storage, Alertmanager).
external_labels:
monitor: 'example'

# Alertmanager configuration
alerting:
alertmanagers:
- static_configs:
- targets:

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
# - "first_rules.yml"
# - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
# The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
- job_name: 'prometheus'

# Override the global default and scrape targets from this job every 5 seconds.
scrape_interval: 5s
scrape_timeout: 5s

# metrics_path defaults to '/metrics'
# scheme defaults to 'http'.

static_configs:
- targets: ['localhost:9090']

- job_name: "wmi_exporter"
# If prometheus-node-exporter is installed, grab stats about the local
# machine by default.
static_configs:
- targets: ['localhost:9182']

- job_name: "l3afd"
# If prometheus-node-exporter is installed, grab stats about the local
# machine by default.
static_configs:
- targets: ['localhost:9182']
85 changes: 85 additions & 0 deletions dev_environment/l3af_on_windows.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
### Prerequisites:

- [Git](https://github.com/git-for-windows/git/releases/download/v2.41.0.windows.3/Git-2.41.0.3-64-bit.exe)
- Cmake
- Clang
- Nmake
- [Golang](https://go.dev/doc/install)
- C++ using: [MSYS2](https://www.msys2.org/)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to know why MSYS2 is needed, so we can work to remove that dependency

- [Visual Studio Build Tools 2022](https://aka.ms/vs/17/release/vs_buildtools.exe)
evershalik marked this conversation as resolved.
Show resolved Hide resolved

### Steps to Build l3afd on Windows:

1. Clone l3afd repo:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. Clone l3afd repo:
1. Clone the l3afd repository:

```bash
git clone https://github.com/l3af-project/l3afd.git
```

2. Build l3afd using:
```bash
cmake -B build
cmake --build build
```

>l3afd.exe file will be created after this.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
>l3afd.exe file will be created after this.
>l3afd.exe will be created after this.


### Steps to run l3afd on Windows:

3. Update [l3afd_win.cfg](./cfg/l3afd_win.cfg) file:

- Replace the `l3afd.cfg` file in `l3afd` repo with the above `l3afd_win.cfg` file
- Manually create a directory and set the custom path for pid-file: `mkdir C:\var\l3afd`
- Set `swagger-api-enabled` to **true**

4. Run l3afd.exe
```bash
l3afd.exe
```

![l3afd.exe](../images/l3af-on-windows/l3afd.exe_output.png)

### Access Swagger API on the dashboard:
Go to this webpage: http://localhost:53000/swagger/index.html

![SwaggerAPI](../images/l3af-on-windows/SwaggerAPI.png)

### Attaching eBPF program with L3AF:

> **NOTE:** Before moving further, set up [eBPF for Windows](https://github.com/microsoft/ebpf-for-windows/blob/main/docs/GettingStarted.md) on the system.

#### Changes need to be done before proceeding:

Manually set the following paths in [l3afd_win.cfg](../config/l3afd_win.cfg):
- bpf-dir:
- bpf-log-dir:
- BpfMapDefaultPath:
- [ebpf-repo] url:
- set bpf-chaining-enabled to false

#### Using [payload.json](./cfg/port_quota_add_payload.json) to load programs:

- Use this curl command to add an eBPF program using the `payload.json` file:
```bash
curl -X POST http://localhost:53000/l3af/configs/v1/add -d "@cfg/payload.json"
```

- See the `logs` in the window where `l3afd.exe` is running:

![ebppAddProgramLog](../images/l3af-on-windows/ebppAddProgramLog.png)

### Confirm eBPF program loading on Windows:

- Use this command to list all the eBPF programs running on the system:
```bash
netsh ebpf show programs
```
or
```bash
bpftool prog show
```






Binary file added images/l3af-on-windows/SwaggerAPI.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/l3af-on-windows/ebppAddProgramLog.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/l3af-on-windows/l3afd.exe_output.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.