Private custom template

[cegekaJG]

Since I've had several problems and security concerns using a public fork of microsoft/AL-Go-PTE, I've decided to create a new one that would be private. Here's what I did:

1. Originally, I've forked microsoft/AL-Go-PTE and modified the repository, adding custom workflows, settings and initial folder structures, simply called CBS-BC-AT/AL-Go. That's when I wanted to "unfork" the repo.
2. To keep all the settings and branches, I just created the new repository CBS-BC-AT/AL-Go-Default, using CBS-BC-AT/AL-Go as the template.
3. I removed the template SHA (I've kept the template URL, which remained "https://github.com/microsoft/al-go-pte@main") from .github/AL-Go-settings.json, ran the "Update AL-Go System Files" action and merged the PR.
4. I've updated the template URL of a repo created using CBS-BC-AT/AL-Go as a template with a link to CBS-BC-AT/AL-Go-Default instead and ran the "Update AL-Go System Files" action again.

But because AL Go is interpreting the new template as an AL Go project rather than a template, I've encountered two problems:

1. Changes to CBS-BC-AT/AL-Go-Default will trigger the build pipeline without any apps, wasting a lot of runner minutes.
2. Updating the system files in an actual Al Go project causes the workflow to fail at this line.

I understand why these two occur - the actions are comparing template URLs to determine whether the repo is a template or a copy. I'd just like to be able to use CBS-BC-AT/AL-Go-Default as a template for our projects, and use the "Update AL Go System Files" workflow to fetch any updates from microsoft/AL-Go-PTE.
Is there a reason why that shouldn't be possible?

[freddydk]

Not sure I understand what you are doing here?, which might be the reason that you are encountering issues - if you are trying to bend things that are not intended to be bend, you will encounter issues and you might at some point be broken by changes in AL-Go.
But,... I would first like to understand your security concern? - in order to guide you to do this the intended way (if there is one)
All GitHub actions are public - why would you think the template repository would need to be private?
The template repository doesn't contain any secrets or like, are you adding code or any secret stuff to the template repo?

[cegekaJG]

The security concern is more about the contents of the repo and the supplementary features on GitHub. We could use issues or discussions to talk about internal information of either the company or our clients, and we might add data that shouldn't be publicly accessible. Of course, we wouldn't store tokens, private URLs or anything like that, but I want the template to allow for mentioning something not intended for public, such as referencing a solution in a private repository or incidents.

[freddydk]

I don't understand that.
using microsoft/AL-Go-PTE@main|preview as your template doesn't prevent you from creating private repositories.
Why would you have your own version of the AL-Go-PTE repo in the first place?

[cegekaJG]

Our version had several configurations already set up to ensure new repositories start with important configurations from the start, such as custom rulesets, workspace settings and AL Go settings.
Using the default template would mean we'd have to manually add those every time we create a new repository.

You can see the old repository here, in case you're curious.

[jonaswre]

@freddydk we've talked about this at the last tech days.

Your Idea was to introduce a .al-go repo in an org. This has basicly the same utily as creating private templates.

I'm currently trying to centralize configurations and script overrides into one template.

I had the same idea to copy the Al-Go-Appsource repo to a private repo in our org and make all the changes I require.

So then a common update Cycle would be

(Offical Template) <- (Org Template) <- (Repo)

There are three things that keep this from working as desired.

Firstly only a few files are updated when I think the following files should also be updated: al.code-workspace (only the settings), .github/*Settings.json (everything not just template url).
Secondly the way you check if something is a template repo, is currently by the template url (as mentioned above). I think i might be better to actually check the template flag in the settings.
.
The check is just to make sure not do accidentily override anything right?
And lastly this happens when the check is run from a build.

TemplateUrl: https://github.com/byndit/AL-Go-AppSource@main
TemplateSha: 2a6909d8b9a4cc59443e22ff72613db[51](https://github.com/byndit/BeyondScheduler/actions/runs/8235387385/job/22519400448#step:4:53)2872b39
DownloadLatest: True
The remote server returned an error: (404) Not Found. 

Not Found

...retrying in 1 minute
Retry failed as well
Error: The remote server returned an error: (404) Not Found.

[cegekaJG]

That's how I expected AL Go to work at first. That being said, there's some things to consider here:

• Not just setting files, but the entire repository can be subject for synchronization. In our case, I've added extra workflows that provide status checks and metadata for PRs, as well as config files
• You definitely don't want to overwrite the entire settings - there might be a good reason why you want to change the default value for a particular repository. The trick would be to discern which properties should be changed, and that's a non-trivial problem.

[jonaswre]

• Not just setting files, but the entire repository can be subject for synchronization. In our case, I've added extra workflows that provide status checks and metadata for PRs, as well as config files

Workflow and Scripts are already copied from the Template.

• You definitely don't want to overwrite the entire settings - there might be a good reason why you want to change the default value for a particular repository. The trick would be to discern which properties should be changed, and that's a non-trivial problem.

This is in theory pretty good. I've created some custom tooling to do that. But in the end its very difficult to find a way that works for everybody. I think the most reasonable solution is to overwrite everything that is present in the template.

If you consider the following config:

{
  "settings": {
    "tools": {
      "ci-tool": "al-go"
    },
    "translations": {
      "de-DE": "Hallo"
    }
  }
}

If you apply the following template

{
  "settings": {
    "tools": {
      "ci-tool": "algo"
    },
    "translations": {
      "en-US": "Hello"
    }
  }
}

The result should be

{
  "settings": {
    "tools": {
      "ci-tool": "algo"
    },
    "translations": {
      "de-DE": "Hallo"
      "en-US": "Hello"
    }
  }
}

This is pretty straight forward to understand and implement.

I've you want it all, you probably need to use a transaction log.
The transaction could be in the JSON Patch format.
This would allow you to controll the exact changes you want be be made.
There are some considerations to be made:

1. You need to store the last transaction that was done so that you don't apply transactions after they have been applied. So that you don't override local changes.
2. You need to write your changes in the json patch format.
3. You need to write your changes in the actual file. (Which is used when "Use this template" is used)
4. You would need a workflow that runs after you used the build in "Use this template" function. Because the transactions in your repo should be deleted.