[BUG] Tests related to backup failed on nightly test run

[khushboo-rancher]

Describe the bug
All the tests related to backup are failing on master-head.
https://ci.longhorn.io/job/public/job/master/job/ubuntu/job/amd64/job/longhorn-tests-ubuntu-amd64/477/

Looks like the backups are stuck and didn't complete.

To Reproduce
Steps to reproduce the behavior:

1. Run any test related to backups.

Expected behavior
The backup should complete properly.

Log

def wait_for_backup_completion(client, volume_name, snapshot_name=None,
                                   retry_count=RETRY_BACKUP_COUNTS):
        completed = False
        for _ in range(retry_count):
            v = client.by_id_volume(volume_name)
            for b in v.backupStatus:
                if snapshot_name is not None and b.snapshot != snapshot_name:
                    continue
                if b.state == "complete":
                    assert b.progress == 100
                    assert b.error == ""
                    completed = True
                    break
            if completed:
                break
            time.sleep(RETRY_BACKUP_INTERVAL)
>       assert completed is True
E       AssertionError

[longhorn-manager-jdqgk] time="2021-11-03T21:35:14Z" level=error msg="failed to get backup status" controller=longhorn-engine engine=pvc-1caf0f63-3c4c-4214-b915-c2039d571894-e-cc3e2f3f error="failed to execute: /var/lib/longhorn/engine-binaries/longhornio-longhorn-engine-master-head/longhorn [--url 10.42.2.6:10001 backup status], output , stderr, time=\"2021-11-03T21:35:14Z\" level=fatal msg=\"Error querying backup status: Missing required parameter backupID\"\n, error exit status 1" node=ip-172-31-3-38

Environment:

• Longhorn version: master-head 11/03/2021

[PhanLe1010]

After meeting with @khushboo-rancher , we found out that it was because this engine commit is merged while the Longhorn manager is not merged longhorn/longhorn-manager#1107

[innobead]

longhorn/longhorn-engine#674

[khushboo-rancher]

As per the nightly test run, the problem in fixed.
https://ci.longhorn.io/job/public/job/master/job/ubuntu/job/amd64/job/longhorn-tests-ubuntu-amd64/478/#showFailuresLink