Replacements deploy

[mshanemc]

What does this PR do?

string replacements during deploy

define file via glob or filename
define what to replace via exact strings or regex
define new text from env or file
replacements could happen conditional based on one or more envs

provide output in DeployResponse for plugins to use for display

What issues does this PR fix or reference?

@W-11949818@

[WillieRuemmele]

want to keep this env var in?

[mshanemc]

I'm leaning toward yes. It would also enable a use case I don't want to officially encourage (do replacements to create a mdapi folder that you can mdapi deploy)

[mshanemc]

but I renamed it to match sf style

[WillieRuemmele]

should we add a replacement perf test?

[peternhale]

Line 57 above is looking at toReplace checking if it is a string and if so creating a regex from the string. If toReplace needs to be a regex, why not convert the string into the regex here?

[peternhale]

@mshanemc is your work going to include updating sfdx-project.schema.json in @salesforce/schemas?

[mshanemc]

it should. good call.

[mshanemc]

I made a WI for it

[peternhale]

Why carry a string and a regex, when a single property would suffice?

[mshanemc]

exact-match-on-strings is simpler, less error-prone, and require no escaping beyond what json requires for \

considering the common scenarios like emails and urls, user@company.com or https://foo.com/api, it's gonna result in more success for users.

[peternhale]

Given that regexes are loaded from json, there will be nothing that verifies regexes while modifying the project json, allowing an invalid regex to be saved. At runtime the, the regex will fail parse. How will this error bubble out with enough information to properly identify the source of the error?

[mshanemc]

ERROR running force:source:deploy:  Component conversion failed: Invalid regular expression: /BOOM\/: \ at end of pattern

How's that?

[WillieRuemmele]

QA Notes

---

Replace with ENV VAR

    {
      "glob": "force-app/**/*.cls",
      "stringToReplace": "replaceMeWithEnv",
      "replaceWithEnv": "ENVIRONMENT"
    }

✅ : confirmed in the org string was replaced with ENV VAR value
✅ : when ENVIRONMENT not set: ERROR running force:source:deploy: Component conversion failed: "ENVIRONMENT" is in sfdx-project.json as a value for "replaceWithEnv" property, but it's not set in your environment.
✅ : in output of source:push

=== Metadata Replacements

 PROJECT PATH                                        TEXT REPLACED    
 ─────────────────────────────────────────────────── ──────────────── 
 force-app/main/default/classes/GeocodingService.cls replaceMeWithEnv 

😐 : in output of deploy --verbose identical to source:push but not when --verbose is excluded
😐 : with env set, but nothing matching in file, no warning nothing was replaced
✅ : new replacements in json
✅ : present in --json even without specifying --verbose
✅ : source:retrieve no affect - pulls whats in org
😐 : "glob": "force-app/**/abc.cls", with env set, no warning no file was found?
😐 : "stringToReplace": "string not in file", no warning about not finding string?

Optionally Replace

    {
          "glob": "force-app/**/*.cls",
          "stringToReplace": "replaceThis",
          "replaceWithEnv": "ENVIRONMENT",
          "replaceWhenEnv": [
            {
              "env": "ENVIRONMENT",
              "value": "bar"
            }
          ]
        }

✅ : nothing replaced when ENVIRONEMNT=prod
✅ : replaced with bar when ENVIRONEMNT=bar
⁉️ : invalid regex allowed, "glob": "[", should be invalid when parsed as RegEx

Replace with File

        {
          "glob": "force-app/**/*.cls",
          "stringToReplace": "replaceMeWithFile",
          "replaceWithFile": "replacements.txt"
        }

✅ : replaced string with file contents
🏆 : nice error message when file not found The file "nonexistant.csv" specified in the "replacements" property of sfdx-project.json could not be read.

Replace with RegEx

        {
          "glob": "force-app/**/*.cls",
          "regexToReplace": "\\d{4}@W",
          "replaceWithEnv": "ENV"
        }

✅ : replaced // 1234@W in the Apex Class to bar (set at command level)
❓ : had to escape \ as invalid json so the RegEx became \\d to represent \d - Should make a note of this when releasing
🏆 : error message indicates invalid RegEx Invalid regular expression: /[/: Unterminated character class when "regexToReplace": "[",

Replace String with File

        {
          "glob": "force-app/main/default/classes/**/*",
          "stringToReplace": "find this string",
          "replaceWithFile": "replacements.csv"
        }

✅ : replaces all of the matching strings with content of file

Replace String with Env Var

        {
          "glob": "force-app/main/default/classes/**/*",
          "stringToReplace": "find this string",
          "replaceWithEnv": "ENV"
        }

✅ : replaces the string with env var
✅ : replaces all of the strings that match with that env var

[mshanemc]

QA responses:

invalid regex allowed, "glob": "[", should be invalid when parsed as RegEx

globs are not regex.

"glob": "force-app/**/abc.cls", with env set, no warning no file was found

yeah, because these are streamed at the component level, there's not some top-level context where the stream is concerned with "has there ever been any file that matched this?" It's possibly, but not elegantly.

❓ : had to escape \ as invalid json so the RegEx became \d to represent \d - Should make a note of this when releasing

it's going to be in the docs.