Skip to content

Updated tp version 1.16 #23

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Nov 10, 2022
Merged

Updated tp version 1.16 #23

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
66f4171
Updated to use local & version 1.16
sandeep540 Nov 10, 2022
c3f309c
Updated to 1.16
sandeep540 Nov 10, 2022
37d52c6
Updated as per comments
sandeep540 Nov 10, 2022
ce8db15
Update Dockerfile
sandeep540 Nov 10, 2022
6620d4e
Update README.md
sandeep540 Nov 10, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
# SQL CLI for Apache Flink® on Docker®


This docker-compose provides an Apache Flink® SQL CLI image updated to the 1.15.0 version. It's inspired by [this](https://github.com/wuchong/flink-sql-demo/tree/v1.11-EN/sql-client).
This docker-compose provides an Apache Flink® SQL CLI image updated to the 1.16.0 version. It's inspired by [this](https://github.com/wuchong/flink-sql-demo/tree/v1.11-EN/sql-client).


It makes use of the `flink:1.15.0-scala_2.12` images and of the `ftisiot/flink-sql-client:1.15.0` which is based on the same `flink:1.15.0-scala_2.12` image.
It makes use of the `flink:1.16.0-scala_2.12` images and of the `ftisiot/flink-sql-client:1.16.0` which is based on the same `flink:1.16.0-scala_2.12` image.

The `sql-client` service maps a `~/kafkacerts/` folder to `/certs` which can be used to create and pass files like Keystores when SSL authentication is needed (e.g. with Apache Kafka®).

Includes the SQL connectors to:
* [Elasticsearch® 7](https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-connector-elasticsearch7/1.15.0/flink-sql-connector-elasticsearch7-1.15.0.jar)
* [Apache Kafka®](https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-connector-kafka/1.15.0/flink-sql-connector-kafka-1.15.0.jar)
* [AVRO](https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-avro-confluent-registry/1.15.0/flink-sql-avro-confluent-registry-1.15.0.jar)
* [JDBC](https://repo.maven.apache.org/maven2/org/apache/flink/flink-connector-jdbc/1.15.0/flink-connector-jdbc-1.15.0.jar)
* [PostgreSQL 42.3.6](https://jdbc.postgresql.org/download/postgresql-42.3.6.jar)
* [Flink Faker](https://github.com/knaufk/flink-faker/releases/download/v0.5.0/flink-faker-0.5.0.jar) allowing to [generate fake data](https://github.com/knaufk/flink-faker)
* [Elasticsearch® 7](https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-connector-elasticsearch7/1.16.0/flink-sql-connector-elasticsearch7-1.16.0.jar)
* [Apache Kafka®](https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-connector-kafka/1.16.0/flink-sql-connector-kafka-1.16.0.jar)
* [AVRO](https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-avro-confluent-registry/1.16.0/flink-sql-avro-confluent-registry-1.16.0.jar)
* [JDBC](https://repo.maven.apache.org/maven2/org/apache/flink/flink-connector-jdbc/1.16.0/flink-connector-jdbc-1.16.0.jar)
* [PostgreSQL 42.5.0](https://jdbc.postgresql.org/download/postgresql-42.5.0.jar)
* [Flink Faker](https://github.com/knaufk/flink-faker/releases/download/v0.5.0/flink-faker-0.5.1.jar) allowing to [generate fake data](https://github.com/knaufk/flink-faker)

## Usage

Expand Down Expand Up @@ -63,9 +63,9 @@ And take note of the `CONTAINER_ID` related to the `ftisiot/flink_sql_cli:1.14.3

```
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
af00e6ef943f ftisiot/flink_sql_cli:1.15.0 "/docker-entrypoint.…" 2 minutes ago Up 2 minutes 6123/tcp, 8081/tcp flink_sql-client_1
22a459232385 flink:1.15.0-scala_2.12 "/docker-entrypoint.…" 2 minutes ago Up 2 minutes 6123/tcp, 8081/tcp flink_taskmanager_1
8b4d5c087800 flink:1.15.0-scala_2.12 "/docker-entrypoint.…" 2 minutes ago Up 2 minutes 6123/tcp, 0.0.0.0:8081->8081/tcp flink_jobmanager_1
af00e6ef943f ftisiot/flink_sql_cli:1.16.0 "/docker-entrypoint.…" 2 minutes ago Up 2 minutes 6123/tcp, 8081/tcp flink_sql-client_1
22a459232385 flink:1.16.0-scala_2.12 "/docker-entrypoint.…" 2 minutes ago Up 2 minutes 6123/tcp, 8081/tcp flink_taskmanager_1
8b4d5c087800 flink:1.16.0-scala_2.12 "/docker-entrypoint.…" 2 minutes ago Up 2 minutes 6123/tcp, 0.0.0.0:8081->8081/tcp flink_jobmanager_1
```

Now execute the following code to enter the docker container in interactive mode
Expand Down
8 changes: 5 additions & 3 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@ version: "2.2"
services:
sql-client:
container_name: 'sql-client'
image: ftisiot/flink_sql_cli:1.15.0
build:
context: .
dockerfile: sql-client/Dockerfile
depends_on:
- jobmanager
environment:
Expand All @@ -11,7 +13,7 @@ services:
- ./settings/:/settings

jobmanager:
image: flink:1.15.0-scala_2.12-java11
image: flink:1.16.0-scala_2.12-java11
ports:
- "8081:8081"
command: jobmanager
Expand All @@ -24,7 +26,7 @@ services:
- ./data/:/data

taskmanager:
image: flink:1.15.0-scala_2.12-java11
image: flink:1.16.0-scala_2.12-java11
depends_on:
- jobmanager
command: taskmanager
Expand Down
22 changes: 11 additions & 11 deletions sql-client/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,26 +21,26 @@
#SQL CLI - inspired by https://github.com/wuchong/flink-sql-demo/tree/v1.11-EN/sql-client
###############################################################################

FROM flink:1.15.0-scala_2.12-java11
FROM flink:1.16.0-scala_2.12-java11

# Create CLI lib folder
COPY bin/* /opt/sql-client/
COPY sql-client/bin/* /opt/sql-client/
RUN mkdir -p /opt/sql-client/lib

# Download connector libraries

RUN wget -P /opt/sql-client/lib/ https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-connector-elasticsearch7/1.15.0/flink-sql-connector-elasticsearch7-1.15.0.jar; \
wget -P /opt/sql-client/lib/ https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-connector-kafka/1.15.0/flink-sql-connector-kafka-1.15.0.jar; \
wget -P /opt/sql-client/lib/ https://repo.maven.apache.org/maven2/org/apache/flink/flink-connector-jdbc/1.15.0/flink-connector-jdbc-1.15.0.jar; \
wget -P /opt/sql-client/lib/ https://jdbc.postgresql.org/download/postgresql-42.3.6.jar; \
wget -P /opt/sql-client/lib/ https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-avro-confluent-registry/1.15.0/flink-sql-avro-confluent-registry-1.15.0.jar; \
wget -P /opt/sql-client/lib/ https://github.com/knaufk/flink-faker/releases/download/v0.5.0/flink-faker-0.5.0.jar;
RUN wget -P /opt/sql-client/lib/ https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-connector-elasticsearch7/3.0.0-1.16/flink-sql-connector-elasticsearch7-3.0.0-1.16.jar; \
wget -P /opt/sql-client/lib/ https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-connector-kafka/1.16.0/flink-sql-connector-kafka-1.16.0.jar; \
wget -P /opt/sql-client/lib/ https://repo.maven.apache.org/maven2/org/apache/flink/flink-connector-jdbc/1.16.0/flink-connector-jdbc-1.16.0.jar; \
wget -P /opt/sql-client/lib/ https://repo.maven.apache.org/maven2/org/apache/flink/flink-json/1.16.0/flink-json-1.16.0.jar; \
wget -P /opt/sql-client/lib/ https://jdbc.postgresql.org/download/postgresql-42.5.0.jar; \
wget -P /opt/sql-client/lib/ https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-avro-confluent-registry/1.16.0/flink-sql-avro-confluent-registry-1.16.0.jar; \
wget -P /opt/sql-client/lib/ https://github.com/knaufk/flink-faker/releases/download/v0.5.1/flink-faker-0.5.1.jar;

# Copy configuration
COPY conf/* /opt/flink/conf/
COPY sql-client/conf/* /opt/flink/conf/

WORKDIR /opt/sql-client
ENV SQL_CLIENT_HOME /opt/sql-client

COPY docker-entrypoint.sh /
COPY sql-client/docker-entrypoint.sh /
ENTRYPOINT ["/docker-entrypoint.sh"]