QFlink: Your Quest into Flink Applications

Requirement

• Java 11

Quick Start

• Flink SQL

Module

• QFlink-Common: Commons
• QFlink-Rule: Rule engine
• QFlink-Jobs: Scenario driven jobs
• QFlink-Python: Big data with Python
• QFlink-SQL: SQL Is All Your Need

Branch

• flink.version

Unix Philosophy

• Keep it simple, stupid
• Make it run, then make it right, then make it fast

Articles

• SQL Is All Your Need: Flink SQL
• SQL Is All Your Need: Flink SQL UDF
• SQL Is All Your Need: Flink Dynamic SQL
• Rule Engine

Dive into Deep

• Flink State：RocksDB

• Flink Checkpoint

  • Barrier travel time and alignment, addressed by Unaligned checkpoints and Buffer debloating
  • Snapshot creation time (so-called synchronous phase), addressed by asynchronous snapshots
  • Snapshot upload time (asynchronous phase), addressed by changelog

• Flink DataStream：StreamGraph

• Flink SQL：Lineage

Data Lifecycle

• Data Generation/Collection

  • Flink CDC

• Data Storage

  • Apache Paimon

• Data Process

  • Apache Flink

• Data Application

• Data Management