Skip to content

Commit

Permalink
Add documentation for building Logstash custom image with opensearch …
Browse files Browse the repository at this point in the history
…plugin

Signed-off-by: Florian Medja <florian.medja@ubisoft.com>
  • Loading branch information
Florian Medja committed Apr 11, 2024
1 parent d60a58f commit ceced46
Show file tree
Hide file tree
Showing 2 changed files with 76 additions and 0 deletions.
4 changes: 4 additions & 0 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
- [Configuration for Logstash Output OpenSearch Plugin](#configuration-for-logstash-output-opensearch-plugin)
- [Submitting Changes](#submitting-changes)
- [Backports](#backports)
- [Dockerfile to build a custom Logstash image](docs/dockerfile_to_build_custom_logstash_image.md)

# Developer Guide

Expand Down Expand Up @@ -215,3 +216,6 @@ original PR with an appropriate label `backport <backport-branch-name>` is merge
run successfully on the PR. For example, if a PR on main needs to be backported to `1.x` branch, add a label
`backport 1.x` to the PR and make sure the backport workflow runs on the PR along with other checks. Once this PR is
merged to main, the workflow will create a backport PR to the `1.x` branch.
## [Dockerfile to build a custom Logstash image](docs/dockerfile_to_build_custom_logstash_image.md)
72 changes: 72 additions & 0 deletions docs/dockerfile_to_build_custom_logstash_image.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Dockerfile to build a custom Logstash image

To build an image that is not available in the [official Docker repository tags](https://hub.docker.com/r/opensearchproject/logstash-oss-with-opensearch-output-plugin/tags), or to add specific plugins to your image, you can create a custom Dockerfile.

The process varies depending on whether you want to build an image with `Logstash 8.x` versions or `Logstash 7.x` versions

## Logstash 8.x

Create this `Dockerfile` to build an image with `Opensearch 2.0.2` for **`Logstash 8.x`**

``` Dockerfile
ARG APP_VERSION

FROM docker.elastic.co/logstash/logstash-oss:${APP_VERSION}
RUN logstash-plugin install --version 7.1.1 logstash-integration-aws
RUN logstash-plugin install --version 2.0.2 logstash-output-opensearch
```

## Logstash 7.x

For **`Logstash 7.x`** , the `logstash output opensearch gem` needs to be build.

### Build logstash output opensearch plugin gem

1. Clone `logstash-output-opensearch repo`

```sh
git clone https://github.com/opensearch-project/logstash-output-opensearch.git
```

2. Checkout the tag for the plugin version you want to build. For the version [2.0.2](https://github.com/opensearch-project/logstash-output-opensearch/tree/2.0.2) for example

```sh
git checkout 2.0.2
```


3. Remove [this line that adds the json version spec](https://github.com/opensearch-project/logstash-output-opensearch/blob/2.0.2/logstash-output-opensearch.gemspec#L49)

4. Build the gem by running the following command:

```sh
gem build logstash-output-opensearch.gemspec
```

The gemfile `logstash-output-opensearch-2.0.2-x86_64-linux.gem` will be generated.

### Dockerfile

Create this Dockerfile to build an image with `logstash version 7.x` and the previously generated `gemfile`:

```Dockerfile
ARG APP_VERSION
FROM docker.elastic.co/logstash/logstash-oss:${APP_VERSION}
USER logstash
# Remove existing logstash aws plugins and install logstash-integration-aws to keep sdk dependency the same
# https://github.com/logstash-plugins/logstash-mixin-aws/issues/38
# https://github.com/opensearch-project/logstash-output-opensearch#configuration-for-logstash-output-opensearch-plugin
RUN logstash-plugin remove logstash-input-s3
RUN logstash-plugin remove logstash-input-sqs
RUN logstash-plugin remove logstash-output-s3
RUN logstash-plugin remove logstash-output-sns
RUN logstash-plugin remove logstash-output-sqs
RUN logstash-plugin remove logstash-output-cloudwatch
RUN logstash-plugin install --version 7.1.1 logstash-integration-aws
COPY logstash-output-opensearch-2.0.2-x86_64-linux.gem /usr/share
RUN logstash-plugin install /usr/share/logstash-output-opensearch-2.0.2-x86_64-linux.gem
```

0 comments on commit ceced46

Please sign in to comment.