Support setting customized envs for backup and restore jobs

[dragonly]

What problem does this PR solve?

Closes #3831

What is changed and how does it work?

Add Env field in BackupSpec, and add logic/ut cases for backup, restore and clean jobs.

Code changes

• Has Go code change
• Has CI related scripts change

Tests

• Unit test

• E2E test

• Manual test

  1. deploy Backup with spec.env set.

  apiVersion: pingcap.com/v1alpha1
  kind: Backup
  metadata:
    name: demo-backup-gcp
    namespace: default
  spec:
    env:
      - name: test_name
        value: test_value
      - name: GCS_PROJECT_ID
        value: fake_id
    br:
      cluster: basic
      sendCredToTikv: true
    gcs:
      projectId: gcp
      location: us-west2
      bucket: backup
      prefix: test1-demo1
      secretName: gcs-secret

  2. kubectl get job -o yaml and find that env vars are set correctly, and the builtin GCS_PROJECT_ID is overwritten

  apiVersion: batch/v1
  kind: Job
  metadata:
    creationTimestamp: "2021-03-12T06:44:32Z"
    labels:
      app.kubernetes.io/component: backup
      app.kubernetes.io/instance: demo-backup-gcp
      app.kubernetes.io/managed-by: backup-operator
      app.kubernetes.io/name: backup
      tidb.pingcap.com/backup: demo-backup-gcp
    name: backup-demo-backup-gcp
    namespace: default
    ownerReferences:
    - apiVersion: pingcap.com/v1alpha1
      blockOwnerDeletion: true
      controller: true
      kind: Backup
      name: demo-backup-gcp
      uid: 8c3cd5a6-5050-4f49-ab16-380016e8a79d
    resourceVersion: "2630257"
    selfLink: /apis/batch/v1/namespaces/default/jobs/backup-demo-backup-gcp
    uid: 919b20c9-a5ef-4e3c-8187-68f67b5dde7b
  spec:
    backoffLimit: 0
    completions: 1
    parallelism: 1
    selector:
      matchLabels:
        controller-uid: 919b20c9-a5ef-4e3c-8187-68f67b5dde7b
    template:
      metadata:
        creationTimestamp: null
        labels:
          app.kubernetes.io/component: backup
          app.kubernetes.io/instance: demo-backup-gcp
          app.kubernetes.io/managed-by: backup-operator
          app.kubernetes.io/name: backup
          controller-uid: 919b20c9-a5ef-4e3c-8187-68f67b5dde7b
          job-name: backup-demo-backup-gcp
          tidb.pingcap.com/backup: demo-backup-gcp
      spec:
        containers:
        - args:
          - backup
          - --namespace=default
          - --backupName=demo-backup-gcp
          - --tikvVersion=v4.0.10
          env:
          - name: GCS_PROJECT_ID
            value: fake_id
          - name: GCS_OBJECT_ACL
          - name: GCS_BUCKET_ACL
          - name: GCS_LOCATION
            value: us-west2
          - name: GCS_STORAGE_CLASS
          - name: GCS_SERVICE_ACCOUNT_JSON_KEY
            valueFrom:
              secretKeyRef:
                key: credentials
                name: gcs-secret
          - name: BR_LOG_TO_TERM
            value: "\x01"
          - name: test_name
            value: test_value
          - name: TZ
            value: UTC
          image: localhost:5000/pingcap/tidb-backup-manager:latest
          imagePullPolicy: IfNotPresent
          name: backup
          resources: {}
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          volumeMounts:
          - mountPath: /var/lib/br-bin
            name: br-bin
        dnsPolicy: ClusterFirst
        initContainers:
        - args:
          - cp /br /var/lib/br-bin/br; echo 'BR copy finished'
          command:
          - /bin/sh
          - -c
          image: pingcap/br:v4.0.10
          imagePullPolicy: IfNotPresent
          name: br
          resources: {}
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          volumeMounts:
          - mountPath: /var/lib/br-bin
            name: br-bin
        restartPolicy: Never
        schedulerName: default-scheduler
        securityContext: {}
        serviceAccount: tidb-backup-manager
        serviceAccountName: tidb-backup-manager
        terminationGracePeriodSeconds: 30
        volumes:
        - emptyDir: {}
          name: br-bin
  status: {}

• No code

Side effects

• Breaking backward compatibility
• Other side effects:

Related changes

• Need to cherry-pick to the release branch
• Need to update the documentation

Release Notes

Please refer to Release Notes Language Style Guide before writing the release note.

Support setting customized envs for backup and restore jobs

[codecov-io]

Codecov Report

Merging #3833 (c9cdee0) into master (5381164) will decrease coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #3833      +/-   ##
==========================================
- Coverage   62.40%   62.38%   -0.02%     
==========================================
  Files         169      169              
  Lines       17963    17968       +5     
==========================================
+ Hits        11209    11210       +1     
- Misses       5673     5677       +4     
  Partials     1081     1081              

Flag	Coverage Δ
unittest	62.38% <100.00%> (-0.02%)	⬇️

[DanielZhangQD]

@dragonly could you please add the test result?

[DanielZhangQD]

LGTM

[handlerww]

LGTM

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has been approved by:

• DanielZhangQD
• handlerww

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by writing /lgtm in a comment.
Reviewer can cancel approval by writing /lgtm cancel in a comment.

[DanielZhangQD]

/merge

[ti-chi-bot]

This pull request has been accepted and is ready to merge.

Commit hash: 9b5a3b4

[ti-srebot]

cherry pick to release-1.1 in PR #3848