Find file History
Pull request Compare This branch is 80 commits ahead of 5.1.0-post.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
docs
README.md
ccloud-delete-all-topics.sh
ccloud-generate-cp-configs.sh
clients
docker-compose.yml
ksql.commands
start-docker.sh
start.sh
stop-docker.sh
stop.sh
submit_replicator_config.sh

README.md

image

Overview

This Confluent Cloud demo showcases Hybrid Kafka Clusters for Bridge to Cloud: moving from Self-Hosted to Confluent Cloud. Data streams into topics both a local cluster and a cluster in Confluent Cloud, and Replicator copies the on-prem data to Cloud so that stream processing can happen in the Cloud.

Note: if you are looking for code examples of producers writing to and consumers reading from Confluent Cloud, or producers and consumers using Avro with Confluent Schema Registry, checkout the clients subdirectory of examples. It provides client examples written in various programming languages.

This automated demo is an expansion of the KSQL Tutorial. Instead of the Kafka cluster backing the KSQL stream processing running on your local install, it runs on your Confluent Cloud cluster. There are also additional Confluent Platform components including Confluent Control Center and Confluent Replicator.

image

End-to-end Solution

This demo showcases:

  • Confluent Cloud
  • KSQL
  • Confluent Replicator
  • Confluent Control Center

Automated Configurations

It also includes a script that reads the Confluent Cloud configuration in $HOME/.ccloud/config and writes delta configuration files into ./delta_configs for Confluent Platform components and clients connecting to Confluent Cloud, including the following:

  • Confluent Platform Components:
    • Confluent Schema Registry
    • KSQL Data Generator
    • KSQL server
    • Confluent Replicator (standalone binary)
    • Confluent Control Center
  • Kafka Clients:

Prerequisites

Local

As with the other demos in this repo, you may run the entire demo end-to-end with ./start.sh, and it runs on your local Confluent Platform install. This requires the following:

To run the local setup:

$ ./start.sh

Docker

To run the Docker setup:

$ ./start-docker.sh

NOTE: Do not run this demo against your production Confluent Cloud cluster. Run this demo only in a development cluster.

Documentation

You can find the documentation for running this demo, and its accompanying playbook, at https://docs.confluent.io/current/tutorials/examples/ccloud/docs/index.html