Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion _config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ jekyll-archives:
category: '/category/:name/'

# Pagination
paginate: 6
paginate: 9
paginate_path: /page:num/

# Other
Expand Down
9 changes: 5 additions & 4 deletions _layouts/default.html
Original file line number Diff line number Diff line change
Expand Up @@ -134,16 +134,17 @@ <h1 class="sitetitle">{{ site.name }}</h1>

</div>


<!-- Categories Jumbotron
================================================== -->
<div class="jumbotron fortags">
<!--<div class="jumbotron fortags">
<div class="d-md-flex h-100">
<div class="col-md-4 transpdark align-self-center text-center h-100">
<div class="d-md-flex align-items-center justify-content-center h-100">
<h2 class="d-md-block align-self-center py-1 font-weight-light">Explore <span class="d-none d-md-inline">→</span></h2>
</div>
</div>
<div class="col-md-8 p-5 align-self-center text-center">
<div class="col-md-8 p-5 align-self-center text-center">
{% assign categories_list = site.categories %}
{% if categories_list.first[0] == null %}
{% for category in categories_list %}
Expand All @@ -155,9 +156,9 @@ <h2 class="d-md-block align-self-center py-1 font-weight-light">Explore <span cl
{% endfor %}
{% endif %}
{% assign categories_list = nil %}
</div>
</div>
</div>
</div>
</div>-->

<!-- Begin Footer
================================================== -->
Expand Down
1 change: 1 addition & 0 deletions _posts/2024-06-18-librdkafka-interceptors.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ categories: [Data, Kafka, Librdkafka]
image: assets/blog-images/librdkafka_interceptors/KafkaInterceptors.png
featured: false
teaser: Building Kafka interceptors for librdkafka clients
toc: true
---

Interceptors in Apache Kafka are plugins that intercept and potentially modify data that is being produced or consumed by Kafka clients. It allows the externalization of common functionality among clients without having to modify the underlying Kafka code. These interceptors can either be producer interceptors or consumer interceptors and act on a single message at a time. Interceptors hook into various stages of a produce or consume request, allowing access to the message and metadata through custom code.
Expand Down
7 changes: 4 additions & 3 deletions _posts/2024-07-11-primer-on-streaming-databases.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ categories: [Data, Kafka, Streaming Databases, KSQL, Materialize, RisingWave]
image: assets/blog-images/streaming_databases/StreamingDatabase.png
featured: false
teaser: Why streaming databases are the rage and what makes them the hottest thing in the world of streaming. A deep dive into important characteristics of a few streaming DBs with a comparative lens.
toc: true
---


Expand All @@ -18,7 +19,7 @@ In the modern era of digitalization, data needs to be made available for making



### Challenges with Stream Processing
## Challenges with Stream Processing

For the rest of the use cases, we can rely on a stream processing system that processes data in “real-time”. For such real-time processing systems, there are constraints within the system, such as constraints on how quickly the data can be made available for processing from various other systems. It is usually the weakest link in the chain that contributes to the constraints. This makes stream processing in “real-time” a hard problem to solve.

Expand Down Expand Up @@ -53,7 +54,7 @@ Although both a stream processing engine and a streaming database serve the same
Real time analytical databases such as Apache Druid, Apache Pinot, or Clickhouse, on the other hand, are meant for querying data in real-time post ingestion. These databases handle ad-hoc and user facing analytics. Whereas a streaming database is meant to work on data before it is ingested into a RTOLAP database for querying.


# Comparing Streaming Databases
## Comparing Streaming Databases

While evaluating a streaming database, asking a few questions about the features, design, and capabilities can help.

Expand Down Expand Up @@ -93,7 +94,7 @@ When evaluating any software, the maturity and age of the software is a basic cr
Finally, understanding the implementation details helps in designing and troubleshooting the stream processing better. An aspect of evaluation is understanding the data flow model, the key primitives, and user facing APIs / constructs.


# Comparison of 3 OSS streaming databases
## Comparison of 3 OSS streaming databases

For this blog post, we will be providing a comparison between three open source databases - KsqlDB, RisingWave, and Materialize. We will briefly compare them across the previously mentioned factors and provide a reference for evaluation.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: post
title: "Tooling and Process: Lessons from running Kafka at scale for mission critical customers"
categories: [Apache Kafka, Mission-Critical Applications, High Availability Systems, Cluster Management, Performance Optimization, Operational Challenges, Kafka at Scale, SRE Practices, ]
categories: [Apache Kafka, Mission-Critical Applications, High Availability Systems, Cluster Management, Performance, Operational Challenges, Kafka at Scale, SRE Practices, ]
teaser: Dive into our journey of deploying Apache Kafka at scale for fintech clients, navigating challenges and implementing solutions for mission-critical success
authors: Arun
featured: false
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: post
title: "Monetizing APIs, Data through usage-based billing"
authors: ashwin
categories: [Data, SaaS, API Metering, Billing, Real-time, Cost, Product]
categories: [Data, SaaS, API Metering, Billing, Real-time, Cost]
image: assets/blog-images/apinomy_blog/ApinomyBlogTitle.webp
featured: true
hidden: true
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: post
title: "From Code to Cluster: Simplifying Infrastructure with IaC and Kafka"
categories: [Kafka Architecture, Infrastructure as Code (IaC), Automation,DevOps]
categories: [Kafka Architecture, Infrastructure, Automation,DevOps]
teaser: Discover how Infrastructure as Code (IaC) can revolutionize your Kafka setup by simplifying management and boosting automation. Learn how to leverage IaC for scalable, error-free Kafka operations and transform your infrastructure practices for optimal efficiency.
author: Shivaprakash
image: assets/blog-images/oauth-oidc-blog/IAC.png
Expand Down
10 changes: 5 additions & 5 deletions _posts/2024-08-02-kafka-on-kubernetes.md
Original file line number Diff line number Diff line change
Expand Up @@ -486,11 +486,11 @@ subgraph KC["Kubernetes Cluster"]
| | CFK | Strimzi | KOperator |
| :--- | :--- | :---: | ---: |
| **Workload Type** | StatefulSet | StrimziPodSet | Pod |
| | | | |

| **CRs supported**|ClusterLink ConfluentRoleBinding Connector Connect ControlCenter KafkaRestClass KafkaRestProxy| Kafka KafkaBridge KafkaConnector KafkaUser KafkaMirrorMaker2 | KafkaCluster KafkaTopic KafkaUser CruiseControlOperation |
| **Networking Models**| Loadbalancer NodePort Ingress | Loadbalancer NodePort Ingress | Loadbalancer (envoy or Istio ingress) NodePort Ingress |
| **Storage** | * Supports Tiered Storage * | Generic CSI with PVs | Generic CSI with PVs |
| | | | |



### **Security**
Expand All @@ -505,7 +505,7 @@ subgraph KC["Kubernetes Cluster"]
| SASL/OAUTHBEARER | No | Yes | No |
| SASL/GSSAPI | No | No | No |
| Kubernetes RBAC | No | No | Yes with K8S namespaces and SA |
| | | | |



### **Authorization**
Expand All @@ -514,7 +514,7 @@ subgraph KC["Kubernetes Cluster"]
| :--- | :--- | :---: | ---: |
| ACL | Yes | Yes | Yes with Envoy |
| RBAC | Yes | No | No |
| | | | |



### **Security**
Expand All @@ -525,7 +525,7 @@ subgraph KC["Kubernetes Cluster"]
| Monitoring add-ons | Control Center, Confluent Health+, JMX | JMX, Cruise Control | JMX, Cruise Control |
| Disaster recovery | Replicator, ClusterLink. Also support Multi-region clusters | MirrorMaker2 | MirrorMaker2 |
| Scale up/out | kubectl scale with Self Balancing | Non-broker components - K8S HPA and KEDA. Broker - Manual scaling with Strimzi Drain Cleaner | Autoscaling with Cruise Control |
| | | | |



# Summary
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,16 @@
layout: post
title: "Integrating APIs and Event Streaming: for fun and profit"
authors: avinash
categories: [Event-Driven Architecture, Kafka, Eventing, command processing, real-time, event driven workflows, saas]
categories: [Event-Driven Architecture, Kafka, Eventing, Command processing, Real-time, Event Driven Workflows, SaaS]
image: assets/blog-images/eventception/EventDrivenArchitectureAbstract.jpeg
featured: true
hidden: true
weight: -10
teaser: "How to progressively get the best out of RESTful and event-driven architectures."
toc: true
---

# How to turn your legacy architecture to an event-driven system
## How to turn your legacy architecture to an event-driven system

In the world of microservices and real-time applications, event-driven architecture is widely adopted for building loosely coupled systems. As a result of this, multiple patterns have emerged, such as Event Notifications, Event Sourcing, CQRS, etc. All of these patterns depend on the system’s ability to produce events and react to events in a timely manner. But what are these events, and why are they important?

Expand Down
3 changes: 2 additions & 1 deletion _posts/2024-08-11-querying-state-store-in-apache-flink.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,12 @@
layout: post
title: "Querying Stream Processing Systems (Apache Flink)"
authors: avinash
categories: [apache flink, data, stream processing, kafka streams]
categories: [apache flink, Data, Stream Processing, Kafka Streams]
image: assets/blog-images/queryable_state_apache_flink/QueryingStateStoreInStreamProcessing.png
featured: false
hidden: false
teaser: "How to query the state store in a stream processing system such as Apache Flink for unifying Operational and Analytical data."
toc: true
---

## Background
Expand Down
15 changes: 14 additions & 1 deletion _sass/_starsnonscss.scss
Original file line number Diff line number Diff line change
Expand Up @@ -89,4 +89,17 @@
.c-rating--small {
font-size: 50%; }
.c-rating--big {
font-size: 150%; }
font-size: 150%; }


p:has(img) {
padding: 1rem 0;
text-align: center; /* Centers content inside <p> tags that contain an <img> */
}

p:has(img) img {
display: inline-block; /* Ensures the image is treated as inline-block */
}
.mermaid{
padding: 2rem 0;
}
13 changes: 13 additions & 0 deletions assets/css/main.scss
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,16 @@
"syntax",
"starsnonscss"
;


.main-content{
table, th, td{
border: 1px solid black;
border-collapse: collapse;
padding: 10px;
}
table{
width: 100%;
margin-bottom: 20px;
}
}
12 changes: 10 additions & 2 deletions assets/css/screen.css
Original file line number Diff line number Diff line change
Expand Up @@ -472,9 +472,11 @@ span.post-read-more a:hover {
}

blockquote {
border-left: 4px solid #00ab6b;
padding: 0 20px;
border: 2px solid #00ab6b;
padding: 20px;
font-style: italic;
background-color: #c5c5c514;
border-radius: 4px;
color: rgba(0, 0, 0, .5);
}

Expand Down Expand Up @@ -1074,4 +1076,10 @@ iframe {

.navbar-collapse{
flex-grow: 0 !important;
}

.more-blog{
font-size: 14px;
padding: 20px;
font-weight: bold;
}