Enable many deployment jobs

[AllanOricil]

Is your proposal related to a problem?

It is just an idea that was bothering me recently regarding doing delta deployments using many deployment jobs, instead of a single one. I'm not really sure if this is a real problem.

• Are there releases that must be done using many deployment jobs because pre/post destructive changes wouldn't work?
• Maybe a release was planned to be done into multiple phases? If so, is it worthy storing this information in git? and how does people usually do it?

Let's assume that there is a use case for breaking a single release into many deployment jobs. For example, the following v1.1.0 build is too complex to be released using a single deployment job, and someone decided to do it using several deployment jobs, using only commits that point to successfull builds, and without adding new tags.

          ______   job 3
          |
         _______   job 2
        | |
       _________  job 1
      | | |
---*--*-*-*--*----   main
   |_________|_____ latest
             |_____ v1.1.0

Describe a solution you propose

Maybe there could exist a reserved keyword that is added to git commits, or even git notes, which sgd could use to break a single deployment into many. For example, considering the following set of commits, and that v1.1.0 has to be released:

a32dasv latest [v1.0.0]      build v1.0.0
23echsd                      [sgd-delete-after]  commit message 1
84jdhe2                      commit message 2
528jded                      [sgd-delta-boundary] [sgd-delete-after] commit messages 3
63ecdsd                      [sgd-delete-after] commit message 4
94jwhe2                      commit message 5
a28jded                      [sgd-delta-boundary] [sgd-delete-after] commit messages 6 
bd231de[v1.1.0]              build v1.1.0

sfdx source:delta -f v1.0.0 -o ./tmp/sgd would output the following set of directories and manifests:

./tmp
└── /sgd
    ├── a28jded
    │   ├── destructiveChangesPre.xml
    │   ├── destructiveChangesPost.xml
    │   └── package.xml
    └── 528jded
        ├── destructiveChangesPre.xml
        ├── destructiveChangesPost.xml
        └── package.xml

and the following output would be sent to stdout:

{
    "deployments": [
        {
              "git_hash" :  "a28jded",
              "has_pre_destructive_changes": true,
              "has_post_destructive_changes": true,
              "has_package_changes": true
        },
        {
              "git_hash" :  "528jded",
              "has_pre_destructive_changes": true,
              "has_post_destructive_changes": true,
              "has_package_changes": true
        }
    ]
}

CICD automations would then use this output to schedule many deployment jobs in a salesforce instance

Describe alternatives you've considered

I could implement an automation to do it as follows:

1. fetch all commits between a from and to git refs.
2. store those commits which have keywords or notes.
3. run sgd N times, one for each marked commit.
4. output results to stdout like shown below:

{
    "deployments": [
        {
              "git_hash" :  "a28jded",
              "has_pre_destructive_changes": true,
              "has_post_destructive_changes": true,
              "has_package_changes": true
        },
        {
              "git_hash" :  "528jded",
              "has_pre_destructive_changes": true,
              "has_post_destructive_changes": true,
              "has_package_changes": true
        }
    ]
}

but I feel that this could be part of sgd's core since it is already iterating over a list of commits.

Additional context

N/A

[AllanOricil]

relates to #583

[scolladon]

Hi @AllanOricil !

Thanks for this enhancement request and thanks for contributing in making this project better!
It is very well described, very clear, thanks.

I had a thought on this one and to me it seems to be possible to achieve the same result using multiple sfdx-git-delta executions with different output folder.

Using the example:

a32dasv latest [v1.0.0]      build v1.0.0
23echsd                      commit message 1
84jdhe2                      commit message 2
528jded                      [sgd-delta-boundary boundary1] commit messages 3
63ecdsd                      [sgd-delete-after] commit message 4
94jwhe2                      commit message 5
a28jded                      [sgd-delta-boundary boundary2] commit messages 6 
bd231de[v1.1.0]              build v1.1.0

It is possible to have the same result with those commands:

sfdx sgd:source:delta -f v1.0.0 -t 84jdhe2 -o ./tmp/sgd/v1
sfdx sgd:source:delta -f 528jded -t 94jwhe2 -o ./tmp/sgd/boundary1
sfdx sgd:source:delta -f a28jded -t v1.1.0 -o ./tmp/sgd/boundary2

I agree it is not derived from the repo automatically. It is more manual to split the boundaries this way than using the repo with proper commit message and let the plugin do the work with only one commands.
I understand the point of the feature is to be able to do that automatically and provide an easy way to reproduce a deployment chain.

Defining the boundaries looks like an advanced CI automation use case to me, which requires work from release manager anyway. This work would certainly involve trial and error before knowing where to put the boundaries which is a similar work to document the chain of commands itself (different output). It can be hard to know the boundary at the PR Level when merging. I would feel more confortable (as a release manager) to write the chain of commands because it seems easier to correct. No need to amend a commit message to change the order (which can be quite annoying for everyone else if it is on a base branch like main).

Let's see if there is traction from the community, as there is an easy way to use the plugin to have the same result I think it is best to see if it solves a big pain point for multiple people before investing in this.

What do you think @AllanOricil? Do you want to shed more light on this?

[AllanOricil]

No, I'm OK with what you said. It was just an idea that I wanted to share. I agree that it does not make sense to work on it yet if there aren't clear use cases. I couldn't find one myself.