Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(examples): add docker compose example #3910

Merged
merged 6 commits into from
Mar 22, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions examples/docker-compose/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
FROM ubuntu:20.04

RUN apt-get update
RUN apt-get install -y curl unzip
RUN curl -fsS https://hub.infinyon.cloud/install/install.sh?ctx=dc | bash

ENV PATH "$PATH:/root/.fluvio/bin"
ENV PATH "$PATH:/root/.fvm/bin"
60 changes: 60 additions & 0 deletions examples/docker-compose/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Docker Compose + Fluvio

You can run Fluvio Clusters in a Docker Compose setup, this could be useful for
local development and POC development.

In order to run a Fluvio Cluster through Docker, you will need to run Fluvio
components separately, we can use Docker Compose `service`s to achieve this.

## Services

- `sc: Streaming Controller`
- `sc-setup: Post-Initialization Commands`
- `spu: Streaming Processing Unit`

> To learn more about Fluvio architecture, please refer to [Fluvio Documentation][1]

## Running Locally

Clone this repo using `git clone https://github.com/infinyon/fluvio.git` and
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might need to add mkdir metadata data. This makes it so starting back up retains both the metadata state and the data of the compose cluster.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docker takes care of creating them! I prefixed with fluvio- in the user side to avoid conflicts with local dirs!

cd into `./fluvio/examples/docker-compose`, then run `docker compose up --build`.
digikata marked this conversation as resolved.
Show resolved Hide resolved

> Optionally you can run on detached mode `docker compose up --build -d` so
> Fluvio runs in the background.

Then use the `fluvio` CLI to connect to the cluster running in Docker, to do
that you must set the _Fluvio Profile_ to point to Docker's container SC:

> If you dont have the Fluvio CLI installed, run the following command
> `curl -fsS https://hub.infinyon.cloud/install/install.sh | bash`.
> Refer to [Fluvio CLI Reference][2] for more details.

```bash
fluvio profile add docker 127.0.0.1:9103 docker
```

> Fluvio Streaming Controller (SC) usually runs on port `9003` but given that our
> SC is running in a Docker Container, internal port `9003` is mapped to `9103`
> in your system's network.

With the profile set, you are now able to perform Fluvio Client operations
like listing topics:

```bash
fluvio topics list
```

## Teardown

In order to shutdown the Fluvio Cluster running in Docker, you must issue the
following `compose` command:

```bash
docker compose down
```

> Remember to run this command in the same directory as the `docker-compose.yml`
> file.

[1]: https://www.fluvio.io/docs/architecture/overview/
[2]: https://www.fluvio.io/cli/
43 changes: 43 additions & 0 deletions examples/docker-compose/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
version: '3'
services:
sc:
image: infinyon/fluvio:stable
container_name: sc
hostname: sc
ports:
- "9103:9003"
environment:
- RUST_LOG=info
command: "./fluvio-run sc --local /fluvio/metadata"
digikata marked this conversation as resolved.
Show resolved Hide resolved
sc-setup:
build:
context: .
dockerfile: Dockerfile
container_name: sc-setup
environment:
- RUST_LOG=info
entrypoint: >
/bin/sh -c "
fluvio profile add docker sc:9003 docker;
fluvio cluster spu register --id 5001 -p spu:9010 --private-server spu:9011;
exit 0;
"
depends_on:
- sc
spu:
image: infinyon/fluvio:stable
container_name: spu
hostname: spu
volumes:
- fluvio-data:/fluvio/data
environment:
- RUST_LOG=info
ports:
- "9110:9010"
- "9111:9011"
command: "./fluvio-run spu -i 5001 -p spu:9010 -v spu:9011 --sc-addr sc:9004 --log-base-dir /fluvio/data"
depends_on:
- sc
volumes:
fluvio-metadata:
fluvio-data:
Loading