A Self service Apache Kafka Topic Management portal. A Web application which automates the process of creating and browsing Kafka components, by introducing roles/authorizations to users of various teams of an organization.
Branch: master
Clone or download
kafkawize Merge pull request #5 from muralibasani/jdbc_support_store
Adding placeholder - support for Jdbc rdbms based store
Latest commit 2f00302 Feb 10, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
screenshots Bug fix on activitylog, scripts base folder update, release update Jan 8, 2019
src Adding support for Jdbc rdbms based store Feb 10, 2019
.gitignore Initial Commit Dec 1, 2018
LICENSE
README.md Update README.md Jan 20, 2019
kafkawize.iml Adding support for Jdbc rdbms based store Feb 10, 2019
pom.xml Adding support for Jdbc rdbms based store Feb 10, 2019

README.md

Kafkawize : A Web application which automates the process of creating and browsing Kafka components, by introducing roles/authorizations to users of various teams of an organization

About the Application

Objective :
Automate the process of creating and browsing Kafka components, by introducing roles, teams and users.

Technical Architecture:

ImageFig1

User Interface API directly communicates between users and Cluster API.

Front end is built with AngularJs, HTML, and Java script.

Cluster API acts as middle layer between Kafka brokers and UserInterface API.
Cluster API creates Kafka Admin Client and executes the requests for Topic, Acls or Schema registry.

Apache Cassandra datastore stores all users, teams, topics, request and execution data from all the users., and to maintain source of truth.

Spring Security, Spring Boot frameworks are used to develop this application.

Functionalities:

(Broadly divided into two categories based on user roles.)
ROLE : USER can request for creation of kafka components, and browse kafka components.
ROLE : ADMIN can approve and execute the requests of users to create kafka components.
ROLE : SUPERUSER can synchronize topics meta information with Cassandra Datastore from Kafka Brokers (Source of Truth.)

Browse: (ROLE : USER, ADMIN, SUPERUSER)
All users can Browse Topics
All users can Browse Acls
All users can view the producers and consumers of all topics.

Requests:(ROLE : USER)
Users can request for Kafka Topics
Users can request for Kafka Acl
Users can request for Schemas to be registered on Confluent Schema registry.
Users can view all the requests from his team.

Environments:(ROLE : USER)
Users can view the available environments

Approve - Execute :(ROLE : ADMIN)
Users can appprove requests for creating Kafka Topics
Users can appprove requests for creating Kafka Acls
Users can appprove requests for uploading schemas on topics

Users :(ROLE : ADMIN)
Users can view all user details
Users can add new users

Teams :(ROLE : ADMIN, SUPERUSER)
Users can view all team details
Users can add new team

Environments:(ROLE : SUPERUSER)
Users can add a new environment environments

Synchronize Metadata :(ROLE : SUPERUSER)
Users can synchronize topic information from Brokers with Cassandra datastore. (Update team info.)
Users can synchronize acls information from Brokers with Cassandra datastore. (Update team info.)

My Profile :(ROLE : USER, ADMIN)
Users can view their profile.

Change Password :(ROLE : USER, ADMIN)
Users can change their passwords.

Logout :(ROLE : USER, ADMIN)
Users can logout.

How to Run the application

KafkaWize needs the following applications to be up and running.

  1. Spring boot application KafkaWize https://github.com/kafkawize/kafkawize
  2. Spring boot application KafkaWize ClusterApi https://github.com/kafkawize/kafkawizeclusterapi
  3. Apache Cassandra

Steps to run:

  1. Install Apache Cassandra
  2. Setup project KafkawizeClusterApi and update server.port if necessary in application properties
  3. Start KafkaClusterApi
  4. Setup project KafkaWize, and configure Cassandra running host, Cluster api host, in application properties
  5. Start KafkaWize
  6. Cassandra db setup will be done on the startup of the application. We do not have to create manually.

By default KafkaWize runs on port 9097. Access it by http://localhost:9097

Default Teams Team1 Team2 Team3

Default Users(with pwds)

uiuser1/user    from Team1
uiuser2/user    from Team2
uiuser4/user    from Team1  Admin
uiuser5/user    from Team2  Admin
superuser/user  from Team2  Superuser

Screenshots

ImageFig10

ImageFig8

ImageFig7

ImageFig12

ImageFig14

ImageFig13

ImageFig11

ImageFig6

ImageFig5

ImageFig15

ImageFig16

ImageFig2

ImageFig9

ImageFig3

ImageFig4

ImageFig17