How to configure multiple services in the serviceTemplate template

[monadbobo]

How to configure multiple services in the serviceTemplate template?

this was my yaml, I declared a LoadBalancer type, but I also want to add a ClusterIP type service, which means I need two services. How should I do it?

defaults:
        templates:
          dataVolumeClaimTemplate: tcloud-log-clickhouse
          podTemplate: clickhouse:24.3
          serviceTemplate: tcloud-log-clickhouse-service-template            
- 
- name: tcloud-log-clickhouse-service-template
            generateName: "{chi}"
            metadata:
              labels:
                app.kubernetes.io/name: tcloud-log-clickhouse
                app.kubernetes.io/component: tcloud-log-clickhouse-server
            spec:
              ports:
                - name: http
                  port: 8123
                - name: tcp
                  port: 9000
              type: LoadBalancer
              externalTrafficPolicy: Local ```

[alex-zaitsev]

@monadbobo , you may use serviceTemplates instead of serviceTemplate and provide an error, for example

  defaults:
    templates:
      serviceTemplates:
       - service-template-lb
       - service-template-clusterip

Make sure generateName is unique.

[monadbobo]

Hi, I found two problems after making the changes. The first one is that the generated name has an extra prefix "clickhouse". The second problem is that the svc of clusteripd is not generated.

        templates:
          podTemplate: clickhouse:24.3
          serviceTemplates:
            - tcloud-log-clickhouse-service-template
            - tcloud-log-clickhouse-service-inner-template
      templates:
        serviceTemplates:
          - name: tcloud-log-clickhouse-service-template
            generateName: "{chi}"
            metadata:
              labels:
                app.kubernetes.io/name: tcloud-log-clickhouse
                app.kubernetes.io/component: tcloud-log-clickhouse-server
                infra.tce.io/product: tcs
              annotations:
                infra.tce.io/service-id: |-
                  [
                    {
                      "id": "{{ .Values.config.clickhouse.telemetry.domain }}",
                      "port": 8123,
                      "host": "{{ .Values.config.clickhouse.telemetry.domain }}",
                      "protocol": "tcp"
                    }
                  ]
            spec:
              ports:
                - name: http
                  port: 8123
                - name: tcp
                  port: 9000
              type: LoadBalancer
              externalTrafficPolicy: Local
          - name: tcloud-log-clickhouse-service-inner-template
            generateName: "{chi}-inner"
            metadata:
              labels:
                app.kubernetes.io/name: tcloud-log-clickhouse
                app.kubernetes.io/component: tcloud-log-clickhouse-server
                infra.tce.io/product: tcs
              annotations:
                infra.tce.io/service-id: |-
                  [
                    {
                      "id": "{{ .Values.config.clickhouse.telemetry.domain }}-inner",
                      "port": 8123,
                      "host": "{{ .Values.config.clickhouse.telemetry.domain }}-inner",
                      "protocol": "tcp"
                    }
                  ]
            spec:
              ports:
                - name: http
                  port: 8123
                - name: tcp
                  port: 9000
              type: ClusterIP
          - name: tcloud-log-clickhouse-cluster-service-template
            generateName: "{chi}-{cluster}"
            spec:
              ports:
                - name: http
                  port: 8123
                - name: tcp
                  port: 9000
              type: ClusterIP
              clusterIP: None
          - name: tcloud-log-clickhouse-replica-service-template
            generateName: "{chi}-{shard}-{replica}"
            spec:
              ports:
                - name: http
                  port: 8123
                - name: tcp
                  port: 9000
              type: ClusterIP
              clusterIP: None        

[Slach]

could you share your generate CHI manifest without sensitive credentials?

kubectl get chi -n <namespace> <chi-name> -o yaml ?

[monadbobo]

This is my yaml file, and the name of the generated svc is: clickhouse-tcloud-log-clickhouse, operator version: 0.23.3

apiVersion: clickhouse.altinity.com/v1
kind: ClickHouseInstallation
metadata:
  name: tcloud-log-clickhouse
  namespace: tce
  labels:
    infra.tce.io/app-version: <redacted-version>
    infra.tce.io/chart-name: tcloud-log-middlewares
    infra.tce.io/oam-app: tcloud-log-middlewares
    infra.tce.io/oam-comp: tcloud-log-clickhouse
    infra.tce.io/oam-product: application
spec:
  configuration:
    clusters:
      - name: default
        layout:
          shardsCount: 1
          replicasCount: 2
        templates:
          replicaServiceTemplate: tcloud-log-clickhouse-replica-service-template
    profiles:
      distributed_aggregation_memory_efficient: 1
      distributed_background_insert_batch: 1
      distributed_background_insert_split_batch_on_failure: 1
      log_queries: 1
      max_execution_time: 600
      use_uncompressed_cache: 0
    settings:
      logger/level: debug
      logger/file: /var/log/clickhouse-server/clickhouse-server.log
      logger/count: 5
      logger/size: 200M
      background_pool_size: 8
      merge_tree/number_of_free_entries_in_pool_to_execute_mutation: 7
      merge_tree/number_of_free_entries_in_pool_to_execute_optimize_entire_partition: 7
      merge_tree/number_of_free_entries_in_pool_to_lower_max_size_of_merge: 7
      asynchronous_insert_log/flush_interval_milliseconds: 7500
      query_log/flush_interval_milliseconds: 7500
      part_log/flush_interval_milliseconds: 7500
      trace_log/flush_interval_milliseconds: 7500
      error_log/flush_interval_milliseconds: 7500
      macros/layer: "01"
      # 其余 *_log / engines 已省略或可继续保留
    users:
      user_async/settings/async_insert: 1
      user_async/settings/wait_for_async_insert: 0
      user_async/networks/ip: ::/0
      user_default/networks/ip: 127.0.0.1
      operator/networks/ip: ::/0
    zookeeper:
      nodes:
        port: 2181
  defaults:
    templates:
      dataVolumeClaimTemplate: tcloud-log-clickhouse
      podTemplate: clickhouse:24.3
      replicaServiceTemplate: tcloud-log-clickhouse-replica-service-template
  templates:
    podTemplates:
      - name: clickhouse:24.3
        generateName: '{chi}-{shard}-{replica}'
        podDistribution:
          - type: ShardAntiAffinity
            topologyKey: infra.tce.io/hostname
        spec:
          containers:
            - name: clickhouse-pod
              image: <private-registry>/clickhouse-server:<tag>
              env:
                - name: CLICKHOUSE_ALWAYS_RUN_INITDB_SCRIPTS
                  value: "true"
                - name: CLICKHOUSE_SKIP_USER_SETUP
                  value: "1"
              resources:
                requests:
                  cpu: 2000m
                  memory: 2000Mi
                limits:
                  cpu: 8
                  memory: 8Gi
              volumeMounts:
                - name: tcloud-log-clickhouse
                  mountPath: /var/lib/clickhouse
                - name: tcloud-log-clickhouse-config-volume
                  mountPath: /docker-entrypoint-initdb.d
          tolerations:
            - effect: NoSchedule
              key: node-role.kubernetes.io/master
              operator: Exists
          volumes:
            - name: tcloud-log-clickhouse-config-volume
              configMap:
                name: tcloud-log-clickhouse-config
    serviceTemplates:
      - name: tcloud-log-clickhouse-service-template
        generateName: '{chi}'
        spec:
          type: LoadBalancer
          externalTrafficPolicy: Local
          ports:
            - name: http
              port: 8123
            - name: tcp
              port: 9000
      - name: tcloud-log-clickhouse-service-inner-template
        generateName: '{chi}-inner'
        spec:
          type: ClusterIP
          ports:
            - name: http
              port: 8123
            - name: tcp
              port: 9000
      - name: tcloud-log-clickhouse-cluster-service-template
        generateName: '{chi}-{cluster}'
        spec:
          clusterIP: None
          type: ClusterIP
          ports:
            - name: http
              port: 8123
            - name: tcp
              port: 9000
      - name: tcloud-log-clickhouse-replica-service-template
        generateName: '{chi}-{shard}-{replica}'
        spec:
          clusterIP: None
          type: ClusterIP
          ports:
            - name: http
              port: 8123
            - name: tcp
              port: 9000
    volumeClaimTemplates:
      - name: tcloud-log-clickhouse
        spec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 120Gi
            storageClassName: tceinf-csi-loopdevice
            volumeMode: Filesystem

[Slach]

change

  defaults:
    templates:
      dataVolumeClaimTemplate: tcloud-log-clickhouse
      podTemplate: clickhouse:24.3
      replicaServiceTemplate: tcloud-log-clickhouse-replica-service-template

to

  defaults:
    templates:
      dataVolumeClaimTemplate: tcloud-log-clickhouse
      podTemplate: clickhouse:24.3
      replicaServiceTemplate: tcloud-log-clickhouse-replica-service-template
      serviceTemplates:
      - tcloud-log-clickhouse-service-template
      - tcloud-log-clickhouse-service-inner-template