Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adding OpenSearch support, fixing minor typos, and code styling #481

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/source/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,14 @@
:members:
```

## parsedmarc.opensearch

```{eval-rst}
.. automodule:: parsedmarc.opensearch
:members:
```


## parsedmarc.splunk

```{eval-rst}
Expand Down
5 changes: 3 additions & 2 deletions docs/source/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Thanks to all [contributors]!
```

`parsedmarc` is a Python module and CLI utility for parsing DMARC reports.
When used with Elasticsearch and Kibana (or Splunk), it works as a self-hosted
When used with Elasticsearch and Kibana (or Splunk), or with OpenSearch and Grafana, it works as a self-hosted
open source alternative to commercial DMARC report processing services such
as Agari Brand Protection, Dmarcian, OnDMARC, ProofPoint Email Fraud Defense,
and Valimail.
Expand All @@ -40,7 +40,7 @@ and Valimail.
- Consistent data structures
- Simple JSON and/or CSV output
- Optionally email the results
- Optionally send the results to Elasticsearch and/or Splunk, for use with
- Optionally send the results to Elasticsearch/OpenSearch and/or Splunk, for use with
premade dashboards
- Optionally send reports to Apache Kafka

Expand All @@ -52,6 +52,7 @@ installation
usage
output
elasticsearch
opensearch
kibana
splunk
davmail
Expand Down
14 changes: 14 additions & 0 deletions docs/source/opensearch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# OpenSearch and Grafana

To set up visual dashboards of DMARC data, install OpenSearch and Grafana.

## Installation

OpenSearch: https://opensearch.org/docs/latest/install-and-configure/install-opensearch/index/
Grafana: https://grafana.com/docs/grafana/latest/setup-grafana/installation/

## Records retention

Starting in version 5.0.0, `parsedmarc` stores data in a separate
index for each day to make it easy to comply with records
retention regulations such as GDPR.
26 changes: 26 additions & 0 deletions docs/source/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,10 @@ delete = False
hosts = 127.0.0.1:9200
ssl = False

[opensearch]
hosts = https://admin:admin@127.0.0.1:9200
ssl = True

[splunk_hec]
url = https://splunkhec.example.com
token = HECTokenGoesHere
Expand Down Expand Up @@ -238,6 +242,28 @@ The full set of configuration options are:
creating the index (Default: `1`)
- `number_of_replicas` - int: The number of replicas to use when
creating the index (Default: `0`)
- `opensearch`
- `hosts` - str: A comma separated list of hostnames and ports
or URLs (e.g. `127.0.0.1:9200` or
`https://user:secret@localhost`)

:::{note}
Special characters in the username or password must be
[URL encoded].
:::
- `user` - str: Basic auth username
- `password` - str: Basic auth password
- `apiKey` - str: API key
- `ssl` - bool: Use an encrypted SSL/TLS connection
(Default: `True`)
- `timeout` - float: Timeout in seconds (Default: 60)
- `cert_path` - str: Path to a trusted certificates
- `index_suffix` - str: A suffix to apply to the index names
- `monthly_indexes` - bool: Use monthly indexes instead of daily indexes
- `number_of_shards` - int: The number of shards to use when
creating the index (Default: `1`)
- `number_of_replicas` - int: The number of replicas to use when
creating the index (Default: `0`)
- `splunk_hec`
- `url` - str: The URL of the Splunk HTTP Events Collector (HEC)
- `token` - str: The HEC token
Expand Down