Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[1.3.6-prepare][Improvement][Docker/K8s] Support more configs, more service access, skywalking configs, improve image for python, update faq and add support matrix #5158

Conversation

chengshiwen
Copy link
Member

@chengshiwen chengshiwen commented Mar 26, 2021

Purpose of the pull request

[1.3.6-prepare][Improvement][Docker/K8s] Support more configs, more service access, skywalking configs, improve image for python, update faq and add support matrix

This closes #4837, closes #5155, closes #5068, closes #5141, closes #402, closes #403

Brief change log

  • Sync all master, worker and common config for the properties, template, ambari, docker and k8s
  • Add master.dispatch.task.num, master.host.selector
  • Remove redundant MASTER_LISTEN_PORT and WORKER_LISTEN_PORT
  • Add HADOOP_SECURITY_AUTHENTICATION_STARTUP_STATE, JAVA_SECURITY_KRB5_CONF_PATH, LOGIN_USER_KEYTAB_USERNAME, LOGIN_USER_KEYTAB_PATH, KERBEROS_EXPIRE_TIME, HDFS_ROOT_USER, YARN_RESOURCEMANAGER_HA_RM_IDS, YARN_APPLICATION_STATUS_ADDRESS for common properties
  • Add SKYWALKING_ENABLE , SW_AGENT_COLLECTOR_BACKEND_SERVICES, SW_GRPC_LOG_SERVER_HOST, SW_GRPC_LOG_SERVER_PORT for skywalking
  • Add MASTER_SERVER_OPTS, WORKER_SERVER_OPTS, ALERT_SERVER_OPTS, API_SERVER_OPTS, LOGGER_SERVER_OPTS
  • Fix mysql check
  • Improve the environment variables and reduce the duplication config
  • Unify annotations, affinity, nodeSelector, tolerations, resources and probe in K8s
  • Support more service access like ClusterIP, NodePort and LoadBalancer
  • Improve https tip in k8s
  • Replace alpine with debian:slim
  • Remove redundant postgresql-client and mysql-client
  • Add faq for python, spark, hadoop, flink and datax
  • Add docs for support matrix on docker/k8s

Verify this pull request

This change added tests and can be verified as follows:

  • Manually verified the change by testing locally.

@chengshiwen chengshiwen force-pushed the 1.3.6-prepare-improve-docker-and-k8s-config branch from 49ba2bd to 05b08cc Compare March 27, 2021 08:18
@chengshiwen chengshiwen force-pushed the 1.3.6-prepare-improve-docker-and-k8s-config branch from 0f5a8af to a331151 Compare March 28, 2021 10:06
@chengshiwen chengshiwen changed the title [1.3.6-prepare][Improvement][Docker/K8s] Improve docker and k8s config [1.3.6-prepare][Improvement][Docker/K8s] Support more configs, more service access and skywalking configs Mar 29, 2021
@chengshiwen chengshiwen changed the title [1.3.6-prepare][Improvement][Docker/K8s] Support more configs, more service access and skywalking configs [1.3.6-prepare][Improvement][Docker/K8s] Support more configs, more service access, skywalking configs and improve image for python Mar 29, 2021
@chengshiwen chengshiwen force-pushed the 1.3.6-prepare-improve-docker-and-k8s-config branch from d3a293d to 1475caf Compare March 29, 2021 10:36
@chengshiwen chengshiwen changed the title [1.3.6-prepare][Improvement][Docker/K8s] Support more configs, more service access, skywalking configs and improve image for python [1.3.6-prepare][Improvement][Docker/K8s] Support more configs, more service access, skywalking configs, improve image for python and update faq Mar 29, 2021
@chengshiwen chengshiwen changed the title [1.3.6-prepare][Improvement][Docker/K8s] Support more configs, more service access, skywalking configs, improve image for python and update faq [1.3.6-prepare][Improvement][Docker/K8s] Support more configs, more service access, skywalking configs, improve image for python, update faq and add support matrix Mar 29, 2021
@chengshiwen
Copy link
Member Author

chengshiwen commented Mar 29, 2021

@dailidong @CalvinKirs @chenxingchun Done, please review

@sonarcloud
Copy link

sonarcloud bot commented Mar 29, 2021

Copy link
Contributor

@davidzollo davidzollo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1
great job

@davidzollo
Copy link
Contributor

@xingchun-chen , please test this PR when you are free, thx

@davidzollo davidzollo merged commit 41fad86 into apache:1.3.6-prepare Mar 30, 2021
@chengshiwen
Copy link
Member Author

This closes #402 and closes #403

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants