[BUG] multiple "for-cloning-volume" snapshots created after cloning volume

[yangchiu]

Describe the bug (🐛 if you encounter this issue)

After cloning a volume from a source volume, there are multiple (more than one) for-cloning-volume snapshots created for the source volume, need to confirm it's the expected behavior.

To Reproduce

Steps to reproduce the behavior:

1. Create a PVC:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: source-pvc
spec:
  storageClassName: longhorn
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 3Gi

2. Wait for volume to be created and attach it to a node.
3. Write some data to the mount path of the volume
4. Clone the volume by creating the PVC:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: cloned-pvc
spec:
  storageClassName: longhorn
  dataSource:
    name: source-pvc
    kind: PersistentVolumeClaim
  accessModes:
    - ReadWriteOnce
 resources:
    requests:
      storage: 3Gi

5. Check the snapshot list of the source volume, there are 2 (or more) for-cloning-volume snapshots created:

data": [ 3 items

    {
        "actions": { },
        "checksum": "",
        "children": {
            "c8d78d2f-a8db-479e-a36e-57ac44ca39bf": true,
        },
        "created": "2023-05-02T02:52:53Z",
        "id": ["9b966ed2-af19-48e5-a528-c33fe41608a6"](http://10.42.2.3:9500/v1/snapshots/9b966ed2-af19-48e5-a528-c33fe41608a6),
        "labels": {
            "longhorn.io/for-cloning-volume": "pvc-208f0060-3788-446f-8fa5-1d034156a623",
        },
        "links": {
            "self": ["http://10.42.2.3:9500/v1/snapshots/9b966ed2-af19-48e5-a528-c33fe41608a6"](http://10.42.2.3:9500/v1/snapshots/9b966ed2-af19-48e5-a528-c33fe41608a6),
        },
        "name": "9b966ed2-af19-48e5-a528-c33fe41608a6",
        "parent": "",
        "removed": false,
        "size": "134217728",
        "type": "snapshot",
        "usercreated": true,
    },
    {
        "actions": { },
        "checksum": "",
        "children": {
            "volume-head": true,
        },
        "created": "2023-05-02T02:52:54Z",
        "id": ["c8d78d2f-a8db-479e-a36e-57ac44ca39bf"](http://10.42.2.3:9500/v1/snapshots/c8d78d2f-a8db-479e-a36e-57ac44ca39bf),
        "labels": {
            "longhorn.io/for-cloning-volume": "pvc-208f0060-3788-446f-8fa5-1d034156a623",
        },
        "links": {
            "self": ["http://10.42.2.3:9500/v1/snapshots/c8d78d2f-a8db-479e-a36e-57ac44ca39bf"](http://10.42.2.3:9500/v1/snapshots/c8d78d2f-a8db-479e-a36e-57ac44ca39bf),
        },
        "name": "c8d78d2f-a8db-479e-a36e-57ac44ca39bf",
        "parent": "9b966ed2-af19-48e5-a528-c33fe41608a6",
        "removed": false,
        "size": "0",
        "type": "snapshot",
        "usercreated": true,
    },
    {
        "actions": { },
        "checksum": "",
        "children": { },
        "created": "2023-05-02T02:52:54Z",
        "id": ["volume-head"](http://10.42.2.3:9500/v1/snapshots/volume-head),
        "labels": { },
        "links": {
            "self": ["http://10.42.2.3:9500/v1/snapshots/volume-head"](http://10.42.2.3:9500/v1/snapshots/volume-head),
        },
        "name": "volume-head",
        "parent": "c8d78d2f-a8db-479e-a36e-57ac44ca39bf",
        "removed": false,
        "size": "0",
        "type": "snapshot",
        "usercreated": false,
    },

]

Expected behavior

A clear and concise description of what you expected to happen.

Log or Support bundle

If applicable, add the Longhorn managers' log or support bundle when the issue happens.
You can generate a Support Bundle using the link at the footer of the Longhorn UI.

Environment

• Longhorn version: master-head
• Installation method (e.g. Rancher Catalog App/Helm/Kubectl):
• Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version:
  • Number of management node in the cluster:
  • Number of worker node in the cluster:
• Node config
  • OS type and version:
  • CPU per node:
  • Memory per node:
  • Disk type(e.g. SSD/NVMe):
  • Network bandwidth between the nodes:
• Underlying Infrastructure (e.g. on AWS/GCE, EKS/GKE, VMWare/KVM, Baremetal):
• Number of Longhorn volumes in the cluster:

Additional context

Add any other context about the problem here.

[innobead]

@PhanLe1010 could you check this? thanks.

[PhanLe1010]

It is unexpected that there are more than 1 snapshot for 1 cloning operation. Thank you for reporting

[PhanLe1010]

This issue is fixed by the PR longhorn/longhorn-manager#1942

Moving this ticket to ready-for-testing for QA to reverify

[chriscchien]

Verified pass on longhorn master (longhorn-manager ae6b6a) by repeating test steps 10 times