Streamis0.3.0 provides the Streamis-JobManager component, the role of the component is
- Publish streaming applications
- Set streaming application parameters, such as the number of Flink slots, checkpoint related parameters, etc.
- Manage streaming applications (e.g. start and stop)
- Streaming application monitoring
Streamis does not require manual compilation. You can download the installation package directly for deployment. Please click to download the installation package
If you have already obtained the installation package, you can skip this step
- The background compilation method is as follows
cd ${STREAMIS_CODE_HOME}
mvn -N install
mvn clean install
After successful compilation, the installation package will be generated in the 'assembly/target' directory of the project
- The front-end compilation method is as follows
Pre dependency: nodejs, python 2.0
cd ${STREAMIS_CODE_HOME}/web
npm i
npm run buildAfter the compilation is successful, the installation package will be generated in the ${STREAMIS_CODE_HOME}/web directory
The following software must be installed:
- MySQL (5.5+), How to install MySQL
- JDK (above 1.8.0_141), How to install JDK
- The execution of Streamis depends on Linkis, and it needs to be version 1.4.0 and above, so you need to install Linkis above 1.4.0 and ensure that the Flink engine can be used normally.Some functions need to be supported by linkis-1.4.0.
- Datasphere studio (> =1.1.2), the development and debugging of streaming jobs depend on DSS scriptis, and the streaming production center needs to be embedded in the DSS engineering framework system, so it depends on * * dss-1.1.2 * * and above.
Before the formal installation of streamis, please install linkis-1.4.0 and dss-1.1.2 or above, and ensure that the linkis Flink engine and DSS can be used normally. For the installation of DSS and linkis, please refer to the dss & linkis one click installation and deployment document.
How to verify that DSS and linkis are basically available? You can create a flinksql script on DSS scriptis and execute it. If flinksql can execute correctly and return the result set, it means that the DSS and linkis environments are available.
1.installation package preparation
Upload the installation package to the installation directory of the Linux server (currently only supports linux environment deployment), such as /appcom/install/streams, and then extract it:
cd /appcom/Install/streamis
tar -xvf wedatasphere-streamis-${streamis-version}-dist.tar.gz2.Modify the database configuration
vi conf/db.sh
#Configure basic database information3.Modify the basic configuration file
vi conf/config.sh### deploy user
deployUser=hadoop
### ssh port
SSH_PORT=22
##The Max Heap size for the JVM
SERVER_HEAP_SIZE="512M"
##The Port of Streamis
STREAMIS_PORT=9400
### The install home path of STREAMIS，Must provided
STREAMIS_INSTALL_HOME=/appcom/Install/streamis
### Linkis EUREKA information. # Microservices Service Registration Discovery Center
EUREKA_INSTALL_IP=127.0.0.1
EUREKA_PORT=20303
### Linkis Gateway information
GATEWAY_INSTALL_IP=127.0.0.1
GATEWAY_PORT=9001
################### The install Configuration of all Micro-Services #####################
#
# NOTICE:
# 1. If you just wanna try, the following micro-service configuration can be set without any settings.
# These services will be installed by default on this machine.
# 2. In order to get the most complete enterprise-level features, we strongly recommend that you install
# the following microservice parameters
#
STREAMIS_SERVER_INSTALL_IP=127.0.0.1
STREAMIS_SERVER_INSTALL_PORT=9400
STREAMIS_VERSION=0.3.0
STREAMIS_FILE_NAME="STREAMIS-$STREAMIS_VERSION"4.Installation
sh bin/install.sh- The install.sh script will ask you if you need to initialize the database and import metadata.
Because the user is worried that the user repeatedly executes the install.sh script to clear the user data in the database, when the install.sh is executed, the user will be asked if they need to initialize the database and import metadata.
Yes must be selected for the first installation.
5.start up
sh bin/start.sh- Start verification
Verification method, because Streamis and Linkis use a set of Eureka, you need to check whether the Eureka page of Linkis already contains Streamis services, as shown in the figure,
1.Install nginx
sudo yum install -y nginx2.Deploy the front-end package
mkdir ${STREAMIS_FRONT_PATH}
cd ${STREAMIS_FRONT_PATH}
#Place the front-end package
unzip streamis-{streamis-version}.zip
3.Modify the nginx configuration file
cd /etc/nginx/conf.d
vi streamis.conf
# Copy the following template and modify it according to the actual situationserver {
listen 9088;# access port
server_name localhost;
location / {
root ${STREAMIS_FRONT_PAH}; # Please modify it to the appropriate static file directory of Streamis
index index.html index.html;
}
location /api {
proxy_pass http://${Linkis_GATEWAY_IP}:${LINKIS_GATEWY_PORT}; #Back-end Linkis address, please modify it to the ip and port of the Linkis gateway
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header x_real_ipP $remote_addr;
proxy_set_header remote_addr $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_connect_timeout 4s;
proxy_read_timeout 600s;
proxy_send_timeout 12s;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection upgrade;
}
#error_page 404 /404.html;
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
4.Load nginx configuration
sudo nginx -s reloadIf you want to use the streamis0.3.0 front end normally, you also need to install the DSS StreamisAppConn plug-in. Please refer to: StreamisAppConn plug-in installation document
If you want to run streamis0.3.0 normally, you also need to install the linkis Flink engine. Please refer to: linkis Flink engine installation document
If you want to upgrade from a lower version of streamis to streamis0.3.0, please refer to: streamis upgrade document