Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documentation update for Tansen #2

Merged
merged 4 commits into from
Apr 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
185 changes: 34 additions & 151 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,166 +1,49 @@
# Tansen
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/ideasbucketlabs/tansen/main/documentation/images/logo-dark.svg">
<source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/ideasbucketlabs/tansen/main/documentation/images/logo-light.svg">
<img alt="Tansen" src="https://raw.githubusercontent.com/ideasbucketlabs/tansen/main/documentation/images/logo-light.svg" width="144" height="58" style="max-width: 100%;">
</picture>
</p>

Easy way to inspect, analyze and visualize Kafka Cluster, Topics and Schema
<p align="center">
Tansen is a light and fast web UI for visualizing Kafka clusters, managing Kafka topic, viewing Kafka topic messages, manage topic schema, view consumer groups and much more.
</p>

---
## Configuration

> **Note:** Only `TANSEN.KAFKA-CLUSTERS.0.BOOTSTRAP_SERVERS` and `TANSEN.KAFKA-CLUSTERS.0.NAME` is a mandatory configuration that must be configured. Others are all optional.

### Endpoints

By default, Tansen listens and uses `HTTP` protocol. However, you can use the `HTTPS` protocol by using the following configuration.

```shell
docker run -d \
-v somepath/with-certificates:/etc/tansen/secrets \
-e 'TANSEN.KAFKA-CLUSTERS.0.NAME=local' \
-e 'TANSEN.KAFKA-CLUSTERS.0.LABEL=Local' \
-e 'TANSEN.KAFKA-CLUSTERS.0.BOOTSTRAP_SERVERS=localhost:9092,localhost:9101' \
-e 'TANSEN.KAFKA-CLUSTERS.0.SCHEMA_REGISTRY_URL=http://localhost:8081' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SERVER_SSL_KEY-ALIAS=tansen' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SERVER_SSL_KEY-PASSWORD=password' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SERVER_SSL_KEY-STORE=/etc/tansen/secrets/tansen.p12' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SERVER_SSL_KEY-STORE-PASSWORD=password' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SERVER_SSL_KEY-STORE-TYPE=PKCS12' \
-e 'SPRING_PROFILES_ACTIVE=prod' \
ideasbucket/tansen:tag
```

### Kafka broker connection

#### PLAIN (no SSL and authentication)

```shell
docker run -d -e 'TANSEN.KAFKA-CLUSTERS.0.BOOTSTRAP_SERVERS=broker:9092' \
-e 'TANSEN.KAFKA-CLUSTERS.0.NAME=local' \
-e 'SPRING_PROFILES_ACTIVE=prod' \
ideasbucket/tansen:tag
```

#### 2-WAY-SSL

```shell
docker run -d \
-v somepath/with-certificates:/etc/kafka/secrets \
-e 'TANSEN.KAFKA-CLUSTERS.0.NAME=local' \
-e 'TANSEN.KAFKA-CLUSTERS.0.BOOTSTRAP_SERVERS=broker:9092' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SECURITY_PROTOCOL=SSL' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SSL_TRUSTSTORE_LOCATION=/etc/kafka/secrets/kafka.tansen.truststore.jks' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SSL_TRUSTSTORE_PASSWORD=confluent' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SSL_KEYSTORE_LOCATION=/etc/kafka/secrets/kafka.tansen.keystore.jks' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SSL_KEYSTORE_PASSWORD=confluent' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SSL_KEY_PASSWORD=confluent' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM=HTTPS' \
-e 'SPRING_PROFILES_ACTIVE=prod' \
ideasbucket/tansen:tag
```

#### SASL-PLAIN
<p align="center">
<a href="https://github.com/ideasbucketlabs/tansen/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License"></a>
<a href="https://hub.docker.com/r/ideasbucket/tansen"><img src="https://img.shields.io/docker/pulls/ideasbucket/tansen" alt="Total Docker Pulls"></a>
</p>

```shell
docker run -d \
-e 'TANSEN.KAFKA-CLUSTERS.0.BOOTSTRAP_SERVERS=broker:9092' \
-e 'TANSEN.KAFKA-CLUSTERS.0.NAME=local' \
-e 'TANSEN.KAFKA-CLUSTERS.0.SCHEMA_REGISTRY_URL=http://schema-registry:8081' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SECURITY_PROTOCOL=SASL_PLAINTEXT' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SASL_JAAS_CONFIG="org.apache.kafka.common.security.plain.PlainLoginModule required username=\"client\" password=\"client-secret\";' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SASL_MECHANISM=PLAIN' \
-e 'SPRING_PROFILES_ACTIVE=prod' \
ideasbucket/tansen:tag
```

#### SASL-SCRAM

```shell
docker run -d \
-e 'TANSEN.KAFKA-CLUSTERS.0.BOOTSTRAP_SERVERS=broker:9092' \
-e 'TANSEN.KAFKA-CLUSTERS.0.NAME=local' \
-e 'TANSEN.KAFKA-CLUSTERS.0.SCHEMA_REGISTRY_URL=http://schema-registry:8081' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SECURITY_PROTOCOL=SASL_PLAINTEXT' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SASL_JAAS_CONFIG="org.apache.kafka.common.security.scram.ScramLoginModule required username=\"client\" password=\"client-secret\";' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SASL_MECHANISM=SCRAM-SHA-256' \
-e 'SPRING_PROFILES_ACTIVE=prod' \
ideasbucket/tansen:tag
```

#### SASL-SSL

```shell
docker run -d \
-v somepath/with-certificates:/etc/kafka/secrets \
-e 'TANSEN.KAFKA-CLUSTERS.0.NAME=local' \
-e 'TANSEN.KAFKA-CLUSTERS.0.BOOTSTRAP_SERVERS=broker:9092' \
-e 'TANSEN.KAFKA-CLUSTERS.0.SCHEMA_REGISTRY_URL=http://schema-registry:8081' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SECURITY_PROTOCOL=SASL_SSL' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SASL_JAAS_CONFIG="org.apache.kafka.common.security.plain.PlainLoginModule required username=\"client\" password=\"client-secret\";' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SASL_MECHANISM=PLAIN' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SSL_TRUSTSTORE_LOCATION=/etc/kafka/secrets/kafka.tansen.truststore.jks' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SSL_TRUSTSTORE_PASSWORD=confluent' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SSL_KEYSTORE_LOCATION=/etc/kafka/secrets/kafka.tansen.keystore.jks' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SSL_KEYSTORE_PASSWORD=confluent' \
-e 'TANSEN.KAFKA-CLUSTERS.0.KAFKA-OPTIONS_SSL_KEY_PASSWORD=confluent' \
-e 'SPRING_PROFILES_ACTIVE=prod' \
ideasbucket/tansen:tag
```
---
## Features

### Kafka schema registry
* Multi-Cluster Management — see all your clusters in one place
* Light and Dark theme based on your device preference
* View Kafka Brokers — view topic and partition assignments, controller status
* View Kafka Topics — view partition count, replication status, and custom configuration
* Topic schema Management for key and message - view, edit and delete your schema
* View Consumer Groups — view per-partition parked offsets, combined and per-partition lag
* Browse Messages — browse messages with JSON, plain text, Protobuf, and Avro encoding
* Topic Configuration — create and configure new topics and edit existing one

#### PLAIN (no SSL and authentication)
---

```shell
docker run -d -e 'TANSEN.KAFKA-CLUSTERS.0.BOOTSTRAP_SERVERS=broker:9092' \
-e 'TANSEN.KAFKA-CLUSTERS.0.NAME=local' \
-e 'TANSEN.KAFKA-CLUSTERS.0.SCHEMA_REGISTRY=localhost:8081' \
-e 'SPRING_PROFILES_ACTIVE=prod' \
ideasbucket/tansen:tag
```
#### 2-WAY-SSL
## Getting started ##

```shell
docker run -d \
-v somepath/with-certificates:/etc/kafka/secrets \
-e 'TANSEN.KAFKA-CLUSTERS.0.BOOTSTRAP_SERVERS=broker:9092' \
-e 'TANSEN.KAFKA-CLUSTERS.0.NAME=local' \
-e 'TANSEN.KAFKA-CLUSTERS.0.SCHEMA_REGISTRY=localhost:8081' \
-e 'TANSEN.KAFKA-CLUSTERS.0.SCHEMA-REGISTRY-OPTIONS_TRUSTSTORE_LOCATION=/etc/kafka/secrets/kafka.tansen.truststore.jks' \
-e 'TANSEN.KAFKA-CLUSTERS.0.SCHEMA-REGISTRY-OPTIONS_TRUSTSTORE_PASSWORD=confluent' \
-e 'TANSEN.KAFKA-CLUSTERS.0.SCHEMA-REGISTRY-OPTIONS_KEYSTORE_LOCATION=/etc/kafka/secrets/kafka.tansen.keystore.jks' \
-e 'TANSEN.KAFKA-CLUSTERS.0.SCHEMA-REGISTRY-OPTIONS_KEYSTORE_PASSWORD=confluent'
-e 'SPRING_PROFILES_ACTIVE=prod' \
ideasbucket/tansen:tag
```
To run Tansen for Apache Kafka, you can use either a pre-built Docker image or build it (or a jar file) yourself. For more information regarding configuration please visit [configuration page](https://github.com/ideasbucketlabs/tansen/blob/main/documentation/configuration.md).

#### Username/Password
---

```shell
docker run -d \
-v somepath/with-certificates:/etc/kafka/secrets \
-e 'TANSEN.KAFKA-CLUSTERS.0.BOOTSTRAP_SERVERS=broker:9092' \
-e 'TANSEN.KAFKA-CLUSTERS.0.NAME=local' \
-e 'TANSEN.KAFKA-CLUSTERS.0.SCHEMA_REGISTRY=localhost:8081' \
-e 'TANSEN.KAFKA-CLUSTERS.0.SCHEMA_REGISTRY_USERNAME=username' \
-e 'TANSEN.KAFKA-CLUSTERS.0.SCHEMA_REGISTRY_PASSWORD=very_secure_password' \
-e 'SPRING_PROFILES_ACTIVE=prod' \
ideasbucket/tansen:tag
```
## Contributing
If you're interested in contributing to Tansen, please read our [contributing docs](https://github.com/ideasbucketlabs/tansen/blob/main/documentation/CONTRIBUTING.md) **before submitting a pull request**.

#### Disable Topic Add/Edit/Delete
---
## Community
* Follow us on [Github](https://github.com/ideasbucketlabs/tansen), [Twitter](https://twitter.com/myideasbucket)

```shell
docker run -d \
-v somepath/with-certificates:/etc/kafka/secrets \
-e 'TANSEN.KAFKA-CLUSTERS.0.BOOTSTRAP_SERVERS=broker:9092' \
-e 'TANSEN.KAFKA-CLUSTERS.0.NAME=local' \
-e 'TANSEN.KAFKA-CLUSTERS.0.SCHEMA_REGISTRY=localhost:8081' \
-e 'TANSEN.KAFKA-CLUSTERS.0.SCHEMA_REGISTRY_USERNAME=username' \
-e 'TANSEN.KAFKA-CLUSTERS.0.SCHEMA_REGISTRY_PASSWORD=very_secure_password' \
-r 'TANSEN.KAFKA-CLUSTERS.0.TOPIC_DELETE_ALLOWED=false' \
-e 'TANSEN.KAFKA-CLUSTERS.0.TOPIC_ADD_ALLOWED=false' \
-e 'TANSEN.KAFKA-CLUSTERS.0.TOPIC_EDIT_ALLOWED=false' \
-e 'SPRING_PROFILES_ACTIVE=prod' \
ideasbucket/tansen:tag
```
---
## License
Copyright © Nirmal Poudyal

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@

import com.fasterxml.jackson.annotation.*;
import jakarta.validation.constraints.AssertFalse;

import java.time.Instant;

@JsonInclude(JsonInclude.Include.NON_NULL)
Expand Down Expand Up @@ -61,7 +60,6 @@ private boolean hasValidPartition() {

@JsonIgnore
public String getCase() {

if (this.partition == null) {
return "";
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -318,7 +318,6 @@ private boolean hasValidRetentionBytes() {
@AssertFalse(message = "Minimum in-sync replica must be equal or less than replication factor.")
@JsonIgnore
private boolean hasMinInsyncReplicas() {

return (this.replicationFactor < this.minInsyncReplicas);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,15 @@
*/
package com.ideasbucket.tansen

import io.swagger.v3.oas.annotations.OpenAPIDefinition
import io.swagger.v3.oas.annotations.info.Info
import org.springframework.boot.autoconfigure.SpringBootApplication
import org.springframework.boot.context.properties.ConfigurationPropertiesScan
import org.springframework.boot.runApplication

@SpringBootApplication @ConfigurationPropertiesScan
@SpringBootApplication
@ConfigurationPropertiesScan
@OpenAPIDefinition(info = Info(title = "Tansen", version = "1.0", description = "Documentation Tansen APIs v1.0"))
class TansenApplication

fun main(args: Array<String>) {
Expand Down
4 changes: 4 additions & 0 deletions backend/main/resources/application.properties
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ spring.kafka.properties.schema.registry.url = ${SCHEMA_REGISTRY_URL:}
spring.kafka.properties.auto.register.schemas = false
spring.kafka.properties.use.latest.version = true

springdoc.api-docs.groups.enabled = true
springdoc.swagger-ui.path = /api-documentation.html
springdoc.swagger-ui.enabled = true
springdoc.swagger-ui.tagsSorter = alpha
#spring.kafka.consumer.properties.offsets.retention.minutes = 10

management.endpoints.web.base-path = /insight
Expand Down
Loading