recommended partition_num and replica_factor

[porscheme]

We wanted to ingestion data to our fresh Nebula Graph Cluster.

• Given below details what should be optimal number of partitions and replica factor?
• Does the number of graphd and metad node make sense?

Below are the details
Raw Data Size before ingestion: 1 TB
Cluster Config:

metad	graphd	storage
1	3	9

Each VM Config:

vCPU	Memory	NVMe Disks
16	128GB	2 X 2 TB

@wey-gu

[wey-gu]

@porscheme

• You will have 9 VMs in the initial deployment, right? If that is true, the NVMe Disk is enough,
  • You will have it in bare OS rather than k8s, right? if so, for multiple disks, we could configure them with separated data paths (ref)
• I would recommend metaD to be 3 to have metadata replicated.
• As the partition number cannot be changed when space is created, and a too-small number of partitions could end up as data read/write hotspot while too large will be costly to the communication inside the cluster(each partition will have its own raft log). The "partition per physical disk" may not be larger than 10-20, thus for now you have 2*9 == 18 NVMe disks, maybe no more than 180-360 partitions for now, but we need to plan for the volume that you will scale out to in the future, too. It's better to have i.e. create 200 partitions first and do some performance tests based on your data and traffic pattern to see who it goes before going to production.
• For replica_factor, if HA is required, put 3, else, 1

reference: https://docs.nebula-graph.io/3.1.0/4.deployment-and-installation/1.resource-preparations/#_11

[sunkararp]

@wey-gu

• We are using Nebula helm charts for deployment
• Currently, deployment requires two 2 separate NVEMe disks, data & log; Is this a bug or feature?
• Below is what finally have

metad	graphd	storaged
3	3	9]

[sunkararp]

Do you mean the help chart for nebula operator?

I remember that for metad and storaged, each pod requires two storage-class claimed volumes per log and data, but it's not necessarily physically specifically demanding underlying two NVMe disks.

Could you please help point out where it requires so?

Thanks!

I tried to deploy using a VM that has one NVMe disk; deployment never succeeded. It was waiting for either log or data PVC. On other hand, deployment will succeed if metad & storaged VM has two NVMe disk.

look at this code snippet from nebula-cluster.yaml

    config: {{ toYaml .Values.nebula.metad.config | nindent 6 }}
    logVolumeClaim:
      resources:
        requests:
          storage: {{ .Values.nebula.metad.logStorage }}
    {{- if .Values.nebula.storageClassName }}
      storageClassName: {{ .Values.nebula.storageClassName }}
    {{- end }}
    dataVolumeClaim:
      resources:
        requests:
          storage: {{ .Values.nebula.metad.dataStorage }}
      {{- if .Values.nebula.storageClassName }}
      storageClassName: {{ .Values.nebula.storageClassName }}
      {{- end }}
    labels: {{ toYaml .Values.nebula.metad.podLabels | nindent 6 }}

[sunkararp]

Why does the leader change very frequently?

[wey-gu]

Do you mean the help chart for nebula operator?
I remember that for metad and storaged, each pod requires two storage-class claimed volumes per log and data, but it's not necessarily physically specifically demanding underlying two NVMe disks.
Could you please help point out where it requires so?
Thanks!

I tried to deploy using a VM that has one NVMe disk; deployment never succeeded. It was waiting for either log or data PVC. On other hand, deployment will succeed if metad & storaged VM has two NVMe disk.

look at this code snippet from nebula-cluster.yaml

    config: {{ toYaml .Values.nebula.metad.config | nindent 6 }}
    logVolumeClaim:
      resources:
        requests:
          storage: {{ .Values.nebula.metad.logStorage }}
    {{- if .Values.nebula.storageClassName }}
      storageClassName: {{ .Values.nebula.storageClassName }}
    {{- end }}
    dataVolumeClaim:
      resources:
        requests:
          storage: {{ .Values.nebula.metad.dataStorage }}
      {{- if .Values.nebula.storageClassName }}
      storageClassName: {{ .Values.nebula.storageClassName }}
      {{- end }}
    labels: {{ toYaml .Values.nebula.metad.podLabels | nindent 6 }}

Thanks a lot, @sunkararp , this doesn't look right, would you please raise an issue on nebula-operator?

[wey-gu]

Why does the leader change very frequently?

Dear @sunkararp , would you mind sharing when(how to reproduce) the frequent leader change, please? it could be an issue on this repo, too.

thanks!

[sunkararp]

Why does the leader change very frequently?

Dear @sunkararp , would you mind sharing when(how to reproduce) the frequent leader change, please? it could be an issue on this repo, too.

thanks!

@wey-gu thanks for reply.

Leader change was random but very frequent. I happy to send you logs, please help how to get logs :)