Skip to content

Latest commit

 

History

History
68 lines (43 loc) · 3.35 KB

flink-web-ssh-on-portal-to-flink-sql.md

File metadata and controls

68 lines (43 loc) · 3.35 KB
Raw
title description ms.service ms.topic ms.date ROBOTS
How to enter the Apache Flink® CLI client using Secure Shell (SSH) on HDInsight on AKS clusters with Azure portal
How to enter Apache Flink® SQL & DStream CLI client using webssh on HDInsight on AKS clusters with Azure portal.
azure-hdinsight-on-aks
how-to
09/20/2024
NOINDEX

Access Apache Flink® CLI client using Secure Shell (SSH) on HDInsight on AKS clusters with Azure portal

[!INCLUDE retirement-notice] [!INCLUDE feature-in-preview]

This example guides how to enter the Apache Flink CLI client on HDInsight on AKS clusters using SSH on Azure portal, we cover both SQL and Flink DataStream.

Prerequisites

  • You're required to select SSH during creation of Flink Cluster

Connecting to the SSH from Azure portal

Once the Flink cluster is created, you can observe on the left pane the Settings option to access Secure Shell

:::image type="content" source="./media/flink-web-ssh-on-portal-to-flink-sql/create-pod-and-connect-to-web-ssh.png" alt-text="Screenshot showing How to create POD and connect to webssh." lightbox="./media/flink-web-ssh-on-portal-to-flink-sql/create-pod-and-connect-to-web-ssh.png":::

Apache Flink SQL

Connecting to SQL Client

You're required to change directory to /opt/flink-webssh/bin and then execute ./sql-client.sh

:::image type="content" source="./media/flink-web-ssh-on-portal-to-flink-sql/sql-client-file.png" alt-text="Screenshot how to find sql client file.":::

:::image type="content" source="./media/flink-web-ssh-on-portal-to-flink-sql/run-sql-client.png" alt-text="Screenshot showing how to run SQL client.":::

You're now on SQL Client on Flink

Refer to this document to perform few more tests.

Apache Flink DataStream

Flink provides a Command-Line Interface (CLI)  bin/flink to run programs that are packaged as JAR files and to control their execution.

The CLI is part Secure Shell (SSH), and it connects to the running JobManager and use the client configurations specified at conf/flink-conf.yaml.

Submitting a job means to upload the job’s JAR to the SSH pod and initiating the job execution. To illustrate an example for this article, we select a long-running job like examples/streaming/StateMachineExample.jar.

Note

For managing dependencies, the expectation is to build and submit a fat jar for the job.

  • Upload the fat job jar from ABFS to webssh.

  • Based on your use case, you’re required to edit the client configurations using Flink configuration management under flink-client-configs.

  • Let us run StateMachineExample.jar

    ./bin/flink run \
    --detached \
    ./examples/streaming/StateMachineExample.jar

Note

Submitting the job using --detached will make the command return after the submission is done. The output contains the ID of the newly submitted job.

Reference