Template "diffability"

[elovelan]

Happy to send a PR after a quick discussion on this. It's not a lot of work but there are a few different options.

It would be helpful to see changes between versions of step template scripts. While this is technically achievable by looking at the string in the JSON, it makes for a challenging review. Ideally we could see the changes to the unpacked scripts.

I have a few thoughts on how to do accomplish this:

1. Store the unpacked templates in a subdirectory of step-templates or alongside the JSON files like _unpack.ps1 currently does.
2. Make sure that the subdir is kept in sync with the JSON:
   • Add a placeholder in the JSON instead of storing the script contents. Add a build step to combine them. It might make sense to move the JSON to a metadata subdir as well so the website build still works seamlessly.
   • Keep the JSON as-is and add a build step that validates that the script contents are in-sync. Fail the build if not. This adds more overhead to the development effort (have to remember to run the sync before submitting a PR), but makes the build a bit simpler.

OR

• Don't change anything in the current repo but have the build produce a diff of the unpacked scripts and attach the diff to the PR as a comment. You can't comment on specific lines this way but it at least helps with review.

I also suspect this will make running automated tests a bit easier.

[dlemstra]

I was just thinking about opening an issue for this but it seems you beat me 😄.

Keep the JSON as-is and add a build step that validates that the script contents are in-sync. Fail the build if not. This adds more overhead to the development effort (have to remember to run the sync before submitting a PR), but makes the build a bit simpler.

I would vote for this option because this would have the lowest impact. And someone could just run _pack.ps1 to get the script back into the JSON file before a pull request is send. We could put a script called _validate.ps1 in the tools folder that could be used by the build and fail if the script are not in sync.

I don't mind to keep the script body files in the same folder as the step-templates. Would you prefer a sub-directory @alfhenrik or can the step-templates be stored in the same folder.

Would you be open to a pull request from me or @elovelan to add the _validate.ps1 script and maybe change the pack/unpack if we want the script body in a sub-directory?

[elovelan]

I probably should have indicated my preference!

I'm somewhat torn between the two options I've presented for step 2 above with a slight preference towards the first. My thoughts:

Advantages keeping a placeholder in the JSON over synced option:

• One source of truth for the scripts
• Clear separation of metadata and code
• Encourages best practices:
  • When writing a script, don't just write it in the Octopus interface
  • More likely to encourage testing
  • More likely to keep clean formatting
• No need to remember to sync after development and thus fewer broken builds that need to be cleaned up before a PR can be merged (though maybe this can be prevented in the synced option by having the build commit the synced versions if the developer didn't)

Advantages of keeping JSON and PowerShell scripts in sync over placeholder option:

• Ability to test immediately in Octopus (this could be mitigated in placeholder option by having the build commit the packed version)
• Casual Octopus user that doesn't know much PowerShell or prefers editing in Octopus can just export their step templates from the UI
• Only one copy of JSON is needed, avoiding potential confusion when uploading to Octopus for testing

[elovelan]

@dlemstra, @alfhenrik seems busy, do you want to just submit two PRs and we can decide which we like best? I'm happy to do the first option (the one I prefer) if you are willing to take the second?

[matt-richardson]

Hi @elovelan, @dlemstra - thanks for your comments so far, and sorry we haven't had time to respond - it's been pretty busy at OctopusHQ lately.

While we definitely know we want to improve this story, we are planning on looking at it from end to end, and make sure we can handle it seamlessly from end to end.

In the short term, have you seen https://github.com/ASOS/OctopusStepTemplateCi/? This has a fairly good model that integrates well, and will be a fairly strong inspiration for what we move forward with.

We'll be having a chat internally soon about this, and hopefully we'll have something more concrete to share.

[elovelan]

@matt-richardson, my opening of this issue was inspired by OctopusStepTemplateCi; however I was just trying to make a proposal that changed as little as possible but still added value.

[elovelan]

@matt-richardson, @alfhenrik should I close this since it seems it will be covered by the "fix" for #113?

[hnrkndrssn]

No, let's keep this one open. I'm going to have a chat with @matt-richardson today about what/how we're going to go forward with this.

[DevOpsDerek]

Closing as no comment in 3 years.