Skip to content

Divided the workflow content into separate learn and contribute sections #500

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Dec 11, 2024
Merged

Divided the workflow content into separate learn and contribute sections #500

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6fe892d
divided the workflow content into seperate learn and contribute sections
hbisht-RH-ccs Nov 20, 2024
b49e923
Changed menu
hbisht-RH-ccs Nov 20, 2024
dbbd8e5
Addressed review comments
hbisht-RH-ccs Dec 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
190 changes: 190 additions & 0 deletions content/contribute/contributing-and-managing-patterns.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,190 @@
---
menu: contribute
title: Contributing and managing patterns
weight: 50
aliases: /contributing-and-managing-patterns/
---

:toc:
:_content-type: ASSEMBLY
include::modules/comm-attributes.adoc[]

[id="contributing to patterns"]
= Contributing to patterns

To contribute, add the upstream repository as an additional remote and work on a branch other than the main branch. Merge changes into the main branch to align with GitOps workflows. This approach simplifies upstream contributions. Contributions from your forked main branch typically include:

. Customizations to `values-global.yaml` and other installation-specific files.
. Commits made by Tekton and other automated processes unique to your setup.

*To isolate changes for upstream contributions*

. Run the following command to add the upstream repository as a remote:
+
[source,terminal]
----
$ git remote add hcp https://github.com/validatedpatterns/industrial-edge
----

. Fetch branches from all remotes:
+
[source,terminal]
----
$ git fetch --all
----

. Create a branch named `hcp-main` and track `hcp/main`:
+
[source,terminal]
----
$ git branch -b hcp-main -t hcp/main
----

. Make your changes on the `hcp-main` branch as needed.

. Push your changes to your fork’s `hcp-main` branch:

+
[source,terminal]
----
$ git push origin hcp-main
----

*To update `hcp-main` branch with upstream changes*

. Run the following command to check out the `hcp-main` branch:

+
[source,terminal]
----
$ git checkout hcp-main
----

. Pull the latest changes from the upstream repository and rebase:
+
[source,terminal]
----
$ git pull --rebase
----

. Reflect these changes in your forked repository (useful for submitting a PR later):
+
[source,terminal]
----
$ git push origin hcp-main
----

*To integrate upstream pattern changes into your local GitOps process*

. Run the following command to check out the main branch:
+
[source,terminal]
----
$ git checkout main
----

. Merge changes from the `hcp-main` branch into your main branch:
+
[source,terminal]
----
$ git merge hcp-main
----

. Push the merged changes to your fork’s main branch:
+
[source,terminal]
----
$ git push origin main
----

This workflow ensures that the `hcp-main` branch:

. Remains separate from changes made by your local GitOps processes.
. Can be merged or cherry-picked into your local main branch for use in GitOps workflows (useful for tracking submodule updates, such as common).
. Serves as a solid basis for Pull Requests upstream, as it will not include local configuration differences or local GitOps commits.

[id="changing-subtrees"]
== Changing subtrees

The Git subtree feature promotes modularity by enabling multiple patterns to share a common base. Over time, functionality is moved into the common directory to isolate pattern-specific components as standard usage practices emerge. This approach strengthens tools in the common directory, adds features, and simplifies the development of new patterns. The common subtree is typically maintained during routine updates, and pulling changes from upstream includes any updates from the common directory.

You only need to modify subtrees if you want to test changes in the common area of the pattern repositories or contribute to the common repository alongside one of the patterns. Using the pattern alone does not require changing subtrees.

For most cases involving the use and consumption of the pattern, users do not need to know that the pattern uses a subtree.

[source,terminal]
----
$ git clone https://github.com/<your-workspace>/industrial-edge
----

If you want to modify and track your version of common, fork and clone the common repository separately:


[source,terminal]
----
$ git clone https://github.com/<your-workspace>/common
----

Make changes in your fork’s main branch or create a new branch for your modifications.

To track these changes in your fork of the pattern repository (example, `industrial-edge`), replace the common subtree with your forked version. To simplify this process use:

[source,terminal]
----
$ common/scripts/make_common_subtree.sh <subtree_repo> <subtree_branch> <subtree_remote_name>
----

This script sets up a new remote in your local working directory with the specified repository. It replaces the common directory with the specified fork and branch, commits the change, but does not push it.

For example:

[source,terminal]
----
$ common/scripts/make_common_subtree.sh https://github.com/mhjacks/common.git wip-main common-subtree
----

This replaces the common directory in the current repository with the wip-main branch from mhjacks's common repository and names the remote `common-subtree`.

To pull changes from mhjacks's wip-main branch into the parent repository:

[source,terminal]
----
$ git subtree pull --prefix common common-subtree wip-main
----

Running the script without arguments defaults to:

[source,terminal]
----
$ common/scripts/make_common_subtree.sh https://github.com/validatedpatterns/common.git main common-subtree
----

These are the default settings for the repository configuration.

[id="subtree-compared-to-submodule"]
== Subtree compared to Submodule

Ensuring patterns are easily shareable across multiple projects has been a key priority. While technically possible, sharing changes between unrelated Git repositories often involves a manual and error-prone process. The goal is to provide a seamless "pull" experience, where a single git pull action updates shared pattern components. For repository sharing, two main strategies were considered: submodules and subtrees. Initially, submodules were used, but the approach later transitioned to subtrees.

Subtrees integrate the history of another repository into a parent repository, offering many benefits of submodules without most of their drawbacks. Atlassian provides useful documentation on subtrees, which explains their advantages in more detail. For more information see, link:https://www.atlassian.com/blog/developer[Developer's blog] and link:https://www.atlassian.com/git/tutorials/git-subtree[Git subtree].

Submodules presented unavoidable challenges, such as:

* Remembering to check out repositories with `--recurse-submodules` or running `git submodule init && git submodule sync`. The common directory often appeared empty without these steps.
* Ensuring compatibility between submodule-based repositories and other tools. While tools like ArgoCD and Tekton Pipelines supported submodules well, compatibility concerns remained.
* Switching branches where different revisions of submodules were referenced often resulted in out-of-sync states. This behavior, although is correct, could be confusing.
* Submodules required mirroring more repositories in disconnected environments.
* Working with forked submodules meant maintaining two separate repositories and multiple branches in each.

Subtrees also have challenges. For example, it is easier to diverge from the upstream version of the subtree, and users may not realize that a subtree is in use when cloning a repository. This can be considered a feature, but might lead to conflicts when updating to a newer version. Additionally, subtrees are less commonly understood, which can result in unexpected issues, such as:


* Cherry picking from a subtree commit into the parent puts the change in the parent location, not the subtree.

[id="contributing-to-patterns-using-common-subtrees"]
== Contributing to Patterns using Common Subtrees

After forking the common repository and modifying your subtree for testing, you can propose changes to [https://github.com/validatedpatterns/common.git] for integration into other patterns. Making changes to the upstream common for a specific pattern involves two steps:

. Submit a PR to update the upstream common.
. Submit a PR to update the pattern repository with the modified common.
165 changes: 22 additions & 143 deletions content/learn/workflow.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,174 +11,53 @@ include::modules/comm-attributes.adoc[]

= Workflow

These patterns are designed to be composed of multiple components, and for those components to be used in gitops
workflows by consumers and contributors. To use the first pattern as an example, we maintain the link:/industrial-edge[Industrial Edge] pattern, which uses a https://github.com/validatedpatterns/industrial-edge[repo] with pattern-specific logic and configuration as well as a https://github.com/validatedpatterns/common[common repo] which has elements common to multiple patterns. The common repository is included in each pattern repository as a subtree.
Patterns are designed to be composed of multiple components, that consumers and contributors can integrate into GitOps workflows. For instance, the link:/industrial-edge[Industrial Edge] pattern comprises a https://github.com/validatedpatterns/industrial-edge[repository] with pattern-specific logic and configuration, alongside a https://github.com/validatedpatterns/common[common repository] containing elements common to various patterns. This common repository is included as a subtree within each pattern repository, streamlining consistency and reusability across different patterns.


[id="consuming-a-pattern"]
== Consuming a pattern

. Fork the pattern repository on GitHub to your workspace (GitHub user or organization). It is necessary to fork because your fork will be updated as part of the GitOps and DevOps processes, and the main branch (by default) will be used in the automated workflows.
. Clone the forked copy
. Fork the pattern repository on GitHub to your workspace (GitHub user or organization). Forking is necessary because your fork updates as part of the GitOps and DevOps processes, and the main branch (by default) is used in the automated workflows.
. Clone the forked repository:
+
`git clone git@github.com:<your-workspace>/industrial-edge.git`

. Create a local copy of the Helm values file that can safely include credentials

DO NOT COMMIT THIS FILE
You do not want to push personal credentials to GitHub.
. If you want to customize credentials, create a local copy of the secrets values file. Do not push the secrets file to GitHub.

.. Create a local copy of the secrets template file:
[source,terminal]
+
----
$ cp values-secret.yaml.template ~/values-secret.yaml
$ vi ~/values-secret.yaml
----

. Customize the deployment for your cluster
.. Edit the file and securely add your cluster-specific credentials:
+
[source,terminal]
----
$ vi values-global.yaml
$ git commit values-global.yaml
$ git push
----

[id="contributing"]
== Contributing

For contributions, we recommend adding the upstream repository as an additional remote, and making changes on a
branch other than main. Changes on this branch can then be merged to the `main` branch (to be reflected in the GitOps
workflows) and will be easier to make upstream, if you wish. Contributions from your forked `main` branch will contain, by design:

. Customizations to `values-global.yaml` and other files that are particular to your installation
. Commits made by Tekton and other automated processes that will be particular to your installation

To isolate changes for upstreaming (`hcp` is "Validated Patterns", you can use a different remote and/or branch name
if you want):

[source,terminal]
----
$ git remote add hcp https://github.com/validatedpatterns/industrial-edge
$ git fetch --all
$ git branch -b hcp-main -t hcp/main
<make changes on the hcp-main branch>
$ git push origin hcp-main
----

To update branch `hcp-main` with upstream changes:

[source,terminal]
----
$ git checkout hcp-main
$ git pull --rebase
----

To reflect these changes in your forked repository (such as if you would like to submit a PR later):

[source,terminal]
----
$ git push origin hcp-main
----

If you want to integrate upstream pattern changes into your local GitOps process:

[source,terminal]
----
$ git checkout main
$ git merge hcp-main
$ git push origin main
----

Using this workflow, the `hcp-main` branch will:

. Be isolated from any changes that are being made by your local GitOps processes
. Be merge-able (or cherry-pick-able) into your local main branch to be used by your local GitOps processes
(this is especially useful for tracking when any submodules, like common, update)
. Be a good basis for submitting Pull Requests to be integrated upstream, since it will not contain your local configuration differences or your local GitOps commits

[id="changing-subtrees"]
== Changing subtrees

Our patterns use the git subtree feature as a mechanism to promote modularity, so that multiple patterns can use the
same common basis. Over time we will move more functionality into common, to isolate the components that are
particular to each pattern, and standard usage conventions emerge. This will make the tools in common more powerful and featureful, and make it easier to develop new patterns. Normally, we will maintain the common subtree in the normal course of updates, and pulling changes from upstream will include any changes from common.

You only need to change subtrees if you want to test changes in the common/ area of the pattern repositories, or if you wish to contribute to the common/ repository itself in conjunction with one of the patterns. Using the pattern by itself _does not_ require changing subtrees.

For the common cases (use and consumption of the pattern), users do not need to be aware that the pattern uses a subtree at all.

[source,terminal]
----
$ git clone https://github.com/<your-workspace>/industrial-edge
----

If you want to change and track your own version of common, you should fork and clone our common repository separately:

[source,terminal]
----
$ git clone https://github.com/<your-workspace>/common
----

Now, you can make changes in your fork's main branch, or else make a new branch and make changes there.

If you want to track these changes in your fork of the _pattern_ repository (industrial-edge in this case), you will need to swap out the subtree in industrial-edge for the version of common you forked. We have provided a script to make this a bit easier:
$ vi ~/values-secret.yaml
----

. Customize the deployment for your cluster. Customization involves updating the `values-global.yaml` file with configurations specific to your cluster environment.
.. Edit the `values-global.yaml` file:
+
[source,terminal]
+
----
$ common/scripts/make_common_subtree.sh <subtree_repo> <subtree_branch> <subtree_remote_name>
$ vi values-global.yaml
----

This script will set up a new remote in your local working directory with the repository you specify. It will replace the common directory with a new common from the fork and branch you specify, and commit it. The script will _not_ push the result.

For example:

.. Commit the updated `values-global.yaml` file to the repository:
+
[source,terminal]
+
----
$ common/scripts/make_common_subtree.sh https://github.com/mhjacks/common.git wip-main common-subtree
$ git commit values-global.yaml
----

This will replace common in the current repository with the wip-main branch from the common in mhjacks's common repository, and call the remote common-subtree.

From that point, changes from mhjacks's wip-main branch on mhjacks's fork of common can be pulled in this way:

.. Push the committed changes:
+
[source,terminal]
----
$ git subtree pull --prefix common common-subtree wip-main
----

When run without arguments, the script will run as if it had been given the following arguments:

[source,terminal]
----
$ common/scripts/make_common_subtree.sh https://github.com/validatedpatterns/common.git main common-subtree
$ git push
----

Which are the defaults the repository is normally configured with.

[id="subtree-vs-submodule"]
== Subtree vs. Submodule

It has always been important to us to be have a substrate for patterns that is as easy as possible to share amongst
multiple patterns. While it is possible to share changes between multiple unrelated git repositories, it is an almost
entirely manual process, prone to error. We feel it is important to be able to provide a "pull" experience (i.e. one git "pull" type action) to update the shared components of a pattern. Two strategies exist for repository sharing in this way: submodule and subtree. We started with submodules but have since moved to subtree.

Atlassian has some good documentation on what subtree is https://blog.developer.atlassian.com/the-power-of-git-subtree/[here] and https://www.atlassian.com/git/tutorials/git-subtree[here]. In short, a subtree integrates another repository's history into a parent repository, which allows for most of the benefits of a submodule workflow, without most of the caveats.

Earlier versions of this document described the usage of patterns with submodules instead of subtrees. In the earliest stages of pattern development, we used submodules because the developers of the project were familiar with submodules and had used them previously, but we had not used subtrees. User feedback, as well as some of the unavoidable complexities of submodules, convinced us to try subtrees and we believe we will stick with that strategy. Some of the unavoidable complexities of submodules include:

* Having to remember to checkout repositories with `--recurse-submdules`, or else doing `git submodule init && git submodule sync`. Experienced developers asked in several of our support channels early on why common was empty.
* Hoping that other tools that are interacting with the repository are compatible with the submodule approach. (To be fair, tools like ArgoCD and Tekton Pipelines did this very well; their support of submodules was one of the key reasons we started with submodules)
* When changing branches on a submoduled repository, if the branch you were changing to was pointed to a different revision of the submoduled repository, the repository would show out of sync. While this behavior is correct, it can be surprising and difficult to navigate.
* In disconnected environments, submodules require mirroring more repositories.
* Developing with a fork of the submoduled repository means maintaining two forked repositories and multiple branches in both.

Subtrees have some pitfalls as well. In the subtree strategy, it is easier to diverge from the upstream version of the subtree repository, and in fact with a typical `git clone`, the user may not be aware that a subtree is in use at all. This can be considered a feature, but could become problematic if the user/consumer later wants to update to a newer version of the subtree but local changes might conflict. Additionally, since subtrees are not as well understood generally, there can be some surprising effects. In practice, we have run into the following:

* Cherry picking from a subtree commit into the parent puts the change in the parent location, not the subtree

[id="contributing-to-patterns-using-common-subtrees"]
== Contributing to Patterns using Common Subtrees

Once you have forked common and changed your subtree for testing, changes from your fork can then be proposed to [https://github.com/validatedpatterns/common.git] and can then be integrated into other patterns. A change to upstream common for a particular upstream pattern would have to be done in two stages:

. PR the change into upstream's common
. PR the updated common into the pattern repository
Loading