diff --git a/.github/actions/spelling/allow.txt b/.github/actions/spelling/allow.txt index be8e85b1..9f45ef19 100644 --- a/.github/actions/spelling/allow.txt +++ b/.github/actions/spelling/allow.txt @@ -284,4 +284,4 @@ nonprod plpgsql pg_stat_statements pg_buffercache - +JIRA diff --git a/source/cjs-common-platform/index.html.md.erb b/source/cjs-common-platform/index.html.md.erb index 35b9db47..8743deec 100644 --- a/source/cjs-common-platform/index.html.md.erb +++ b/source/cjs-common-platform/index.html.md.erb @@ -1,6 +1,6 @@ --- title: CJS Common Platform -weight: 4 +weight: 11 --- # <%= current_page.data.title %> diff --git a/source/cloud-native-platform/images/infrastructure-overview.png b/source/cloud-native-platform/images/infrastructure-overview.png deleted file mode 100644 index e2ce3ec8..00000000 Binary files a/source/cloud-native-platform/images/infrastructure-overview.png and /dev/null differ diff --git a/source/cloud-native-platform/index.html.md.erb b/source/cloud-native-platform/index.html.md.erb deleted file mode 100644 index d64caf07..00000000 --- a/source/cloud-native-platform/index.html.md.erb +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: Cloud Native Platform -weight: 3 ---- - -# <%= current_page.data.title %> - -The Cloud Native Platform is an Azure based platform that hosts CFT, SDS and Legacy services. - -The core principles of the CNP are: - - - open source by default - - self-service platform - - fully automated - - [GitOps](./new-component/gitops-flux.html) - - continuous delivery - - microservices - - automated testing - - devops - ownership and responsibilities - -## Infrastructure architecture - -![Cloud Native Platform](./images/infrastructure-overview.png) diff --git a/source/cloud-native-platform/troubleshooting/index.html.md.erb b/source/cloud-native-platform/troubleshooting/index.html.md.erb deleted file mode 100644 index 20ecf968..00000000 --- a/source/cloud-native-platform/troubleshooting/index.html.md.erb +++ /dev/null @@ -1,315 +0,0 @@ ---- -title: Troubleshooting issues -last_reviewed_on: 2023-03-24 -review_in: 6 months -weight: 98 ---- - -# <%= current_page.data.title %> -## GitHub - -### Adding a new user on GitHub - - - See [GitHub Onboarding](/cloud-native-platform/onboarding/team/github.html#access) for how to get access - - Make sure the user has accepted the invite received on the email setup in Github or by visiting [github.com/hmcts](https://github.com/hmcts). - -### Creating a new GitHub repository - -See [create a GitHub repository](../new-component/github-repo.html#) - -### Access to GitHub Repo - - - All repositories should be administered by the team who owns it. - - Access should be managed through GitHub teams and not individuals. - - If no one from your team have access, follow [asking for help](/cloud-native-platform/asking-for-help/#asking-for-help) to request org admins in one of the community/ support channels. - - -## Receiving an email regarding inactive account - -You may have received an email from our scheduled pipeline which deletes guest user accounts if they're inactive for longer than 31 days. - -### Account close to being deleted - -Users who are within a week of being deleted get notified via email on the lead up to the deletion date, starting 7 days before. - -If you have received an email about your account being close to being deleted, because of inactivity, that is because you haven't logged into the `CJS Common Platform` Azure Tenant for a while and need to do so now to keep your access. - -Please don't ignore this email if you use the `CJS Common Platform` Tenant or GitHub as you will lose access to both if you don't log in. - -To extend your access, visit the [CJS Common Platform Azure portal](https://portal.azure.com/531ff96d-0ae9-462a-8d2d-bec7c0b42082). - -Login link: [CJS Common Platform]( https://portal.azure.com/531ff96d-0ae9-462a-8d2d-bec7c0b42082) - -*Note:* We're looking into fixing this soon so that as long as you're active in either GitHub or the `CJS Common Platform` Tenant you will not lose access. - -### Account has been deleted - -If your account has been deleted and you need it re-enabled, you need to create a ticket via slack in the [#platops-help](https://hmcts-reform.slack.com/app_redirect?channel=platops-help) channel and someone will send you another guest invite. - -Once that is done you will then be able to get your GitHub access back by asking one of your team members to re-add you following [person onboarding](/cloud-native-platform/onboarding/person). - -## Jenkins - -### Jenkins is unavailable - - - Check if there is a planned outage in [#cloud-native-announce](https://hmcts-reform.slack.com/archives/CA4F2MAFR). - - Please note Jenkins could be temporarily unavailable while rolling out a change, give it a few minutes before you raise an issue. - -### Cannot login to Jenkins - - - Login to Jenkins is managed by Azure AD, please make sure user is added to right [Azure AD groups](/cloud-native-platform/onboarding/person/#azure-ad-groups) - -### You are now logged out of Jenkins/ Infinite loop when trying to login to Jenkins - - - Please try clearing browser cookies on [login.microsoft.com](https://login.microsoftonline.com/) - -### Cannot see my new repo in Jenkins org / Dashboard - - - See [Jenkins Onboarding section](/cloud-native-platform/onboarding/team/jenkins.html#jenkins) to add your app to Jenkins Org / Dashboard - -### Cannot find default branch - -You may get the error below, when the pipeline has not ran on the master/main branch first. - -When you run it on the master/main branch it will setup the default branch and then the pull request build will start working. - -``` -[2021-09-16T10:59:54.154Z] Execution failed for task ':sonarqube'. -[2021-09-16T10:59:54.154Z] > Could not find a default branch to fall back on. -``` -### Cannot see my branch/PR or This project is currently disabled in Jenkins - - - Any branches which are also filed as PRs are not listed as a branch, they will only be listed in pull requests section. - - Branch/ PR is not listed if its last commit creation date is older than 30 days. - -### Build / Docker Build / Unit Test failure - - - If your build is failing in these stages, it's most likely to fail in your local as well. Look at the first line of the Jenkins step that fails and try run the same command Jenkins is running. - - Try on a colleague's machine as you might have cached something locally. - -### Sonar scan timeout - - - Please see [sonarcloud status](https://sonarcloud.statuspage.io/) for any known issues with sonar cloud. - - Remember that Platform Operation do not maintain SonarCloud, issues are usually discussed on community forums. - -### Helm Upgrade Failed, Helm Release timed out waiting for condition, Helm Release Failure - - - See [Connecting to AKS Clusters](#connecting-to-aks-clusters) and connect to the relevant cluster. - - These errors usually mean your pods didn't start as expected in time. - - It could be that they are stuck in `Pending`, `ContainerCreating` status or might be failing to startup leading to `CrashLoopBackOff` status. - - Follow [Debug Application Startup issues in AKS](#debug-application-startup-issues-in-aks) to troubleshoot further - -### Jenkins managed helm releases / pods are automatically deleted - - - To maintain the health of the cluster, it is important to cleanup unwanted pods regularly. - - Helm release is cleared for PRs which are merged or closed. - - Helm release of PRs raised by dependency bots (based on `dependencies` label) is cleared once the functional tests pass. - - Helm release on AAT Staging is also cleared once functional tests pass. - - For optimal usage, You can also configure your pipeline to [clear Helm releases on successful build](https://github.com/hmcts/cnp-jenkins-library#clear-helm-release-on-successful-build). - - A scheduled pipeline runs every hour to clear any helm releases which are not updated in last 3 days. Teams can do more frequent cleanup by overriding in the [cleanup script](https://github.com/hmcts/cnp-aks-pipelines/blob/0fe733120f78b6dabcdd5895bb16134085631842/scripts/delete-inactive-helm-releases.sh#L5) - -### Smoke / Functional test failure - - - Jenkins only sets secrets as environment variables and runs the `gradle` / `yarn` task to run tests. - - Access the URL on VPN and try running tests manually using the `TEST_URL` printed in the logs. - - You can also run tests locally by setting the required secrets while on the VPN. - - To add additional logging, see [Example config](#example-2). - -### Terraform/ Build Infrastructure failure - - - It is important that you need to review your plan in a pull request before applying it to an environment. - - Please check if its an intermittent failure with Azure as a retry could fix it. - - Also, see if there are any open issues/discussions on community channels for the failure. - - There could be open GitHub issues on terraform/ azurerm, so googling it could help as well. - -### Using branches to troubleshoot issues - -If your pipeline is throwing an error, you may be able to more easily troubleshoot the issue by using a branch in the cnp-jenkins-library repo. - -#### Example 1 -Gradle is failing because a plugin cannot be found in artifactory. You've checked the plugin exists and there are no typos in your code. -You can check if the issue lies with artifactory by temporarily bypassing it using a [branch](https://github.com/hmcts/cnp-jenkins-library/blob/073ad8587b7281d62bac705ed984e739a0911c83/resources/uk/gov/hmcts/gradle/init.gradle#L3). - -And then referencing that branch within your repo's [Jenkinsfile](https://github.com/hmcts/sds-toffee-recipes-service/pull/12/files) - -In this example, the issue is network related. This may be down to routing in Azure or traffic being blocked by a firewall. - -#### Example 2 -Gradle is failing a functional test. To help get more information on why, you could update the Gradle logging level in a branch of [cnp-jenkins-library](https://github.com/hmcts/cnp-jenkins-library/blob/cfb31f3a2699b2a1dafd66fed0b525ae145d627d/src/uk/gov/hmcts/contino/GradleBuilder.groovy#L62) - -And then reference this branch within your repo's [Jenkinsfile](https://github.com/hmcts/document-management-store-app/blob/646593336377fd59112b0b6c84fd223d0cb7832c/Jenkinsfile_CNP#L12) - -### Error message `channel_not_found` - -When you want to send Jenkins pipeline notifications to a slack channel, you must make sure the Jenkins app has been added to the channel. - -If you don't do this, you will receive the `channel_not_found` error message and the pipeline will fail. - -See the instructions on the [slack](/cloud-native-platform/onboarding/team/slack.html) section of the onboarding guide to find out how to do this. - -If you are getting this message on a PR pipeline and the error also says `Failed to notify @U1234ABCD`, that means your github username has been mapped to an invalid slack id. - -Find your slack id by clicking on `View profile` within the slack app, then click on the three dots and click `Copy member ID`. - -Update your github to slack user mapping in [this file](https://github.com/hmcts/github-slack-user-mappings/blob/master/slack.json) and try running the pipeline again. - -## Debug Application Startup issues in AKS - -- There could be many reasons why applications could fail to startup like : - - A secret referred in helm chart is missing in keyvaults - - Pod identity is not able to pull keyvault secrets due to missing permissions - - There is not enough space in the cluster to fit in a new pod. - - Pod is scheduled, but fails to pass readiness (`/health/readiness`) or liveness (`/health/liveness`) checks. - - A misconfigured environment variable, example - incorrect URL of a dependent service. - -- Below are some handy kubectl commands to debug the issues - - To check latest events on your namespace: - - ```shell - kubectl get events -n - ``` - - To check status of pods: - - ```shell - kubectl get pods -n | grep - - #Examples - # kubectl get pods -n ccd | grep ccd-data-store-api - # kubectl get pods -n ccd | grep pr-123 - ``` - - To check status of a specific pod which is not running - - ```shell - kubectl describe pod -n - ``` - - To check logs of pods which is not starting - - ```shell - kubectl logs -n - - #To follow logs - kubectl logs -n -f - - # To check previous pod logs if its restarting - kubectl logs -n -p - - ``` - -## Flux / Gitops - - > Always check __why__ your release or pod has failed in the first instance. - > Although you may have permissions to delete a helm release or pod in a non-production environment, use this privilege wisely as you could be _hiding a potential bug_ which could also _occur in production_. - -### Latest image is not updated in cluster - -- Start with checking [cnp-flux-config](https://github.com/hmcts/cnp-flux-config) to make sure flux has updated/ committed the image. -- If image hasn't been committed to Github, see [ Flux did not commit latest image to Github](#flux-did-not-commit-latest-image-to-github). -- If flux has committed the new image to Github, check if the `HelmRelease` has been updated by Flux. Run below command and check that the image tag has been updated in the output - - ```shell - kubectl get hr -n -o yaml - ``` -- If Image is not updated in above, [Change in git is not applied to cluster](#change-in-git-is-not-applied-to-cluster). -- If the image tag is updated and still application pods are not deployed, see [Updated HelmRelease is not deployed to cluster](#updated-helmrelease-is-not-deployed-to-cluster) - -### Flux did not commit latest image to Github - - - Image automation is run from management cluster (CFTPTL). Please login to cftptl cluster before further troubleshooting. - - Image reflector controller keeps polling ACR for new images, but it should generally update the new image in 10 minutes. - - Check status of `imagerepositories` and verify the last scan. - - ```shell - kubectl get imagerepositories -n flux-system - ``` - - If the last scan doesn't update, check image reflector controller logs to see if there any logs related to the helm repo. - - ```shell - kubectl logs -n flux-system -l app=image-reflector-controller --tail=-1 - # search for specific image - kubectl logs -n flux-system -l app=image-reflector-controller --tail=-1 | grep - ``` - - If the last scan is latest, check `imagepolicy` status to verify that the image returned matches the expectation. - - ```shell - kubectl get imagepolicies -n flux-system - ``` - - If it doesn't match the expected tag, verify image reflector controller logs as described above. - - If the `imagepolicy` object returned shows the expected image, but it didn't commit to Github, check image automation controller logs. - - ```shell - kubectl logs -n flux-system -l app=image-automation-controller - # search for specific image - kubectl logs -n flux-system -l app=image-automation-controller | grep - ``` - -### Updated HelmRelease is not deployed to cluster - - - Helm operator queues all the updates, so it could take up to 20 minutes sometimes to be picked up. - - Check HelmRelease status to see the status. - - ```shell - kubectl get hr -n - ``` - - Look at helm operator logs to see if there are any errors specific to your helm release - - ```shell - kubectl logs -n flux-system -l app=helm-controller --tail=1000 | grep - ``` - - If you see any errors like, `status 'pending-install' of release does not allow a safe upgrade"`. You need to delete `HelmRelease` for fixing this, request help from Platform Operations if you do not have permissions. - - ```shell - kubectl delete hr -n - ``` - - In most cases, helm release gets timed out with an error in log similar to ` failed: timed out waiting for the condition`. This usually means application pods didn't startup in time and you need to look at your pods to know more. - - Check the latest status on helm release and if it has already been rolled back to previous release. - - ```shell - kubectl describe hr -n - ``` - - If you are looking at pods after a long time, `HelmRelease` might have been rolled back and you won't have failed pods. Easiest way is to add a simple change like a dummy environment variable in flux-config to re-trigger the release and debug the issue when it occurs. - - - If your old pods are still running when you check, follow [Debug Application Startup issues in AKS](#debug-application-startup-issues-in-aks) to troubleshoot further. - -### Change in git is not applied to cluster - - - To check if latest github commit has been downloaded by checking status - - ```shell - kubectl get gitrepositories flux-config -n flux-system - ``` - - If the commit doesn't match latest id, verify source controller logs to see any related errors - - ```shell - kubectl logs -n flux-system -l app=source-controller - ``` - - If commit id is recent, verify status of flux kustomization for your namespace to get the version of git applied. - - ```shell - kubectl get kustomizations.kustomize.toolkit.fluxcd.io -n flux-system - ``` - - If the above status doesn't show latest commit/ show any error , see kustomize controller logs to find relevant errors. - - ```shell - kubectl logs -n flux-system -l app=kustomize-controller - # search for specific image - kubectl logs -n flux-system -l app=kustomize-controller | grep - ``` - -## Connecting to AKS Clusters - -- By Default, all developers have read access to non-prod AKS clusters and slightly higher privileges to their namespaces. -- You can connect to AKS clusters using `az aks get-credentials`. Below are some handy commands: - -<%= partial 'clusters' %> - -Once you have logged in, you can switch between clusters using [kubectx](https://github.com/ahmetb/kubectx) or below kubectl commands: - -```shell -kubectl config use-context cft-perftest-00-aks -kubectl config use-context cft-aat-00-aks -``` diff --git a/source/concepts-standards/angular.html.md.erb b/source/concepts-standards/angular.html.md.erb new file mode 100644 index 00000000..f4c5bef5 --- /dev/null +++ b/source/concepts-standards/angular.html.md.erb @@ -0,0 +1,425 @@ +--- +title: Angular +weight: 1 +--- + +# <%= current_page.data.title %> + +Follow Conventions +------------------ + +One of the core arguments to choose a framework over traditional way with [Vanilla JavaScript](http://vanilla-js.com/) is its clearly defined way of how things are supposed to be done. That defined way enables the creation of a uniform code base across an organization, without worrying about defining certain rules. Following them does not only increase the uniformity and therefore the quality of the code. This strict approach also comes in handy across cooperation borders. It enables new developers to integrate into a new team very quickly, because of the high familiarity with the code. We should follow the [Angular Style Guide](https://angular.io/guide/styleguide) to get the most out of the Angular Framework. It will make our life a lot easier, when coming into new projects and will increase the quality of our code almost automatically. + +Use Angular CLI +--------------- + +The [Angular CLI](https://cli.angular.io/) is the best way to build Angular Applications. Angular CLI can be installed easily via terminal by typing in the following command: + +``` +npm install -g @angular/cli +``` + +The CLI has [scaffolding tools](https://angular.io/guide/schematics) for creating new projects and generating new code, such as, modules, services, components, directives and pipes. However, this isn't the main benefit. The main benefit of the CLI is the way it automates the build pipeline for both live development with `ng serve`, as well as for production code that we would ship down to browsers with `ng build --prod`. Other common commands available are `ng lint`, `ng test` and `ng e2e`. + +Isolate API hacks +----------------- + +Sometimes, we need to add some logic in the code to make up for bugs in the APIs. Instead of having the hacks in components where they are needed, it is better to isolate them in one place like in a function or a service and use them. When fixing the bugs in the APIs, it is easier to look for them in one file rather than looking for the hacks that could be spread across the codebase. + +Prefer Observables over Promises +-------------------------------- + +[Observables](https://rxjs-dev.firebaseapp.com/guide/observable) partly overlaps the standard JavaScript [Promise](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Promise) functionality. Both are meant to handle asynchronous code. However, Observables can do so much more than Promises. Observables can resolve to more than just one value, because they are a stream of values. Observables are everywhere in Angular Framework. We can see that, by looking at the Angular [HttpClient](https://angular.io/guide/http). It returns Observables, even when it is clear, that a HTTP call can never result in more than one response. Mixing Observables with Promises isn't a good solution. That way, we have completely different implementations, that are hardly compatible with each other in different parts of our application. + +Subscribe in template +--------------------- + +Avoid subscribing to observables from components and instead subscribe to the observables from the template. To consume a stream, we need to subscribe that stream, that's simply how observables work. The [async](https://angular.io/api/common/AsyncPipe) pipe unsubscribe themselves automatically and it makes the code simpler by eliminating the need to manually manage subscriptions. + +``` +@Component({ + selector: 'my-component', + template: `` +}) +export class MyComponent { + data$: Observable; +} +``` + +Using of the [subscribe() instead of the async pipe](https://medium.com/angular-in-depth/angular-question-rxjs-subscribe-vs-async-pipe-in-component-templates-c956c8c0c794) introduces complementary need to unsubscribe at the end of the component life-cycle to avoid memory leaks. Subscribing to the observable manually in the `ngOnInit()` doesn't work with `OnPush` Change Detection Strategy. The [OnPush](https://angular.io/api/core/ChangeDetectionStrategy) is great for performance, so, we should use `async` pipe as much as possible. + +Avoid memory leaks +------------------ + +When subscribing to observables, always unsubscribe from them appropriately by using operators like `take`, `takeUntil`, ... or calling `unsubscribe()`. To consume a stream, we need to subscribe to that stream. That subscription will keep on living until the stream is completed or until we unsubscribe manually from that stream. While Angular takes care of unsubscribing when using the `async` pipe, it quickly becomes a mess when we need to do it ourselves. Failing to unsubscribe from observables will lead to unwanted [memory leaks](https://itnext.io/angular-rxjs-detecting-memory-leaks-bdd312a070a0) as the observable stream is left open, potentially even after a component has been destroyed or the user has navigated to another page. This risk can also be mitigated by using a [lint rule](https://www.npmjs.com/package/rxjs-tslint-rules) to detect unsubscribed observables. + +Avoid nested subscriptions +-------------------------- + +Nesting subscribes is something that needs to be avoided as much as possible. It makes the code unreadable, complex, and introduces side effects. It basically forces us to not think reactively. The next implementation is considered a bad practice: + +``` +this.route.params + .pipe(map(param => param.id)) + .subscribe(id => this.userService.fetchById(id) + .subscribe(user => (this.user = user))); +} +``` + +Do not pass streams to components directly +------------------------------------------ + +Passing streams to child components is a bad practice because it creates a very close link between the parent component and the child component. A component should always receive an object or value and should not even care if that object or value comes from a stream or not. It is better to handle the subscription in the parent component itself. Angular has a feature called the [async](https://angular.io/api/common/AsyncPipe) pipe that can be used for this. + +Do not pass streams to services +------------------------------- + +By passing a stream to a service we don't know what's going to happen to it. The stream could be subscribed to, or even combined with another stream that has a longer lifecycle, that could eventually determine the state of our application. Subscriptions might trigger unwanted behavior. It's recommended to use higher order streams in the components for these situations. + +Use pipeable operators +---------------------- + +Use [pipeable operators](https://github.com/ReactiveX/rxjs/blob/master/doc/pipeable-operators.md) when using RxJS operators. A pipeable operator is basically any function that returns a function with the signature: `(source: Observable) => Observable`. Pipeable operators are [tree-shakable](https://medium.com/@netxm/what-is-tree-shaking-de7c6be5cadd) meaning only the code we need to execute will be included when they are imported. This also makes it easy to identify unused operators in the files. + +Use pure functions +------------------ + +RxJS follows the concepts of functional reactive programming which basically means that we will use pure functions to create our reactive flow. In the beginning it might seem pragmatic to use side effects, but that mostly means we aren't fully thinking reactively. Therefore, avoid side effects at much as possible. + +Avoid manual subscribes +----------------------- + +If a component needs values from different streams, we need to subscribe all those streams and manually map all the values to unique properties. Angular has a feature called the `async` pipe. The `async` pipe unsubscribe themselves automatically and it makes the code simpler by eliminating the need to manually manage subscriptions. It also reduces the risk of accidentally forgetting to unsubscribe a subscription in the component, which would cause a memory leak. It triggers change detection automatically and cleans up the code a lot. + +Do not share all subscriptions +------------------------------ + +Since most streams are cold by default, every subscription will trigger the producer of these streams. However, subscribing to Angular its `http.get()` multiple times will actually perform multiple XHR calls. In some cases, we may want to share the subscriptions: + +``` +users$ = this.http.get(...).pipe(share()); +``` + +It works, but it is a common mistake to share everything. Angular also provides an alternative that can reduce the sharing of streams to a minimum by using the `async as else` syntax. + +``` +@Component({ + selector: 'my-component', + template: ` +
+ Number of users: + +
+ Loading... + ` +}) +class MyComponent { + users$ = this.http.get(...); +} +``` + +When to Subscribe? +------------------ + +The answer to that question is, "Only when we absolutely have to." Because if we don't subscribe, we don't have to unsubscribe. In Services, it's recommended never to subscribe to Observables. In components, we can use the [async](https://angular.io/api/common/AsyncPipe) pipe. So, we will subscribe when we absolutely have to, and we almost never have to. + +When to use Subjects? +--------------------- + +A [Subject](https://rxjs-dev.firebaseapp.com/guide/subject) is both a hot observable and an observer at the same time. This gives us the opportunity to next values into the stream ourselves. Subjects tend to be overused by people that didn't make the mind switch towards reactive programming yet. Only use them when really needed, for instance it's fine to use Subjects when we are mocking streams in tests, when we want to create streams from outputs in Angular or when handling circular references. For most other cases an `operator` or `Observable.create` might be enough. + +Do not expose subjects +---------------------- + +There is a pretty common practice to use Observable Data Services in Angular: + +``` +export class DataService { + private data: BehaviorSubject = new BehaviorSubject(0); + + readonly data$: Observable = this.data.asObservable(); + + increment(): void { + this.data.next(this.data.getValue() + 1); + } +} +``` + +Here we're exposing data stream as observable. Just to make sure it can be changed only through a data service interface. + +Handle RxJS errors +------------------ + +Error handling is an essential part of RxJS. By default, if something goes wrong with an Observable, it just dies. If we don't deal with such errors, it will happen silently, and we won't know why we are not receiving data anymore. [RxJS Error Handling - Complete Practical Guide](https://blog.angular-university.io/rxjs-error-handling/) provides the most common error handling strategies. + +RxJS clean-code practices +------------------------- + +Consistent code indentation and formatting can improve the readability of complex streams: + +- Align operators below each other; +- Extract into different streams when it becomes unreadable; +- Put complex functionalities in private methods; +- Avoid the use of brackets for readability. + +Use marble diagrams +------------------- + +[Marble Diagrams](https://rxmarbles.com/) provide a visual way for us to represent the behavior of an Observable. We can use them to assert that a particular Observable behaves as expected, as well as to create [hot and cold observables](https://medium.com/@benlesh/hot-vs-cold-observables-f8094ed53339) we can use as mocks. Marble diagram is a domain specific language for RxJS to help us to model the interactions and the values of one or more observable in our test. Most of the documentation about [marble testing](https://rxjs-dev.firebaseapp.com/guide/testing/marble-testing) can be found on official documentation. + +Use appropriate operators +------------------------- + +[RxJS Operators](https://rxjs-dev.firebaseapp.com/guide/operators) are the essential pieces that allow complex asynchronous code to be easily composed in a declarative manner. There are operators for different purposes, and they may be categorized as: `creation`, `transformation`, `filtering`, `joining`, `multicasting`, `error handling`, `utility`, etc. When using [flattening operators](https://medium.com/angular-in-depth/switchmap-bugs-b6de69155524) with our observables, use the appropriate operator for the situation. Using a single operator when possible instead of chaining together multiple other operators to achieve the same effect can cause less code to be shipped to the user. Using the wrong operators can lead to unwanted behavior, as different operators handle observables in different ways. + +Use a state management library +------------------------------ + +When building large and complex applications that has lots of information coming from and going to the database, and where state is shared across multiple components, we might start considering adding a [state management](https://medium.com/@2muchcoffee/angular-state-management-a-must-have-for-large-scale-angular-apps-8b98e5a761c7) library. By using a state management library, we can keep the application state in one single place, which reduces the communication between components and keeps our app more predictable and easier to understand. A [Store](https://ngrx.io/guide/store/) isolates all state related logic in one place and makes it consistent across the application. It also has memorization mechanism in place when accessing the information in the Store leading to a more performant application. A Store combined with the change detection strategy of Angular leads to a faster application.\ +There is however, an over use of, for example NgRx. The need for state management increases the complexity of the application. Large stores increase the memory footprint of an application at run time.\ +Don't automatically reach for a store solution if most of what you are doing could be stored in a service or component to manage local state.  An injectable Angular service singleton provided in the might suffice in many cases if the service is set up correctly using function programming paradigms, as long as that service is always the single source of truth for that data and the only way to mutate that data. Consider the issues around data security and  make sure you clear state and date from the store on application exit.\ +See discussion on NgRx Pros and Cons here.  [A Deep Dive Into NgRx Advantages and Features](https://www.toptal.com/angular/why-use-ngrx) + +Avoid any. Type everything +-------------------------- + +Always declare variables or constants with a `type` other than `any`. When declaring variables or constants in Typescript without a typing, the typing of the variable/constant will be deduced by the value that gets assigned to it. This will cause unintended problems. Another advantage of having good typings in our application is that it makes refactoring easier and safer. The any type isn't necessarily a bad thing and, in fact, does still come in useful sometimes. However, in most cases, there is a better alternative that leads to having better defined types overall. In new projects, it is worth setting `strict:true` in the `tsconfig.json` file to enable all strict type checking options. + +Make use of lint rules +---------------------- + +Having lint rules in place means that we will get a nice error when we are doing something that we should not be. This will enforce consistency in our application and readability. Besides checking style, linters are also excellent tools for finding certain classes of bugs, such as those related to variable scope. We can combine [TSLint](https://palantir.github.io/tslint) with [Prettier](https://prettier.io/). TSLint is an extensible static analysis tool that checks TypeScript code for readability, maintainability, and functionality errors. Prettier is an amazing tool that enforces a consistent style by parsing our code and re-printing it, with its own rules in place. Having Prettier setup with TSLint gives us a strong foundation for our applications, as we no longer need to maintain our code-style manually. Combined with [husky](https://www.npmjs.com/package/husky) makes for an excellent workflow. + +LIFT Principle +-------------- + +The [LIFT Principle](https://angular.io/guide/styleguide#lift) helps us to find code quickly. Structuring the app such that we can Locate code quickly, Identify the code immediately, keep the flattest structure we can, and Try to be DRY. Keep a flat folder structure as long as possible. It's very important to give good names for methods, variables, and parameters. 5 seconds rule says that if we can't understand in 5 seconds, we probably need a refactor. We must organize the class code by putting the most important things first, properties followed by methods, grouped and sorted with consistent naming and spelling matter. + +Do not Repeat Yourself +---------------------- + +Make sure we do not have the same code copied into different places in the codebase. Extract the repeating code and use it in place of the repeated code. Having the same code in multiple places means that if we want to make a change to the logic in that code, we must do it in multiple places. This makes it difficult to maintain and is prone to bugs where we could miss updating it in all occurrences. It takes longer to make changes to the logic and testing it is a lengthy process as well. In those cases, extract the repeating code and use it instead. This means only one place to change and one thing to test. Having less duplicate code shipped to users means the application will be faster. + +Avoid long functions +-------------------- + +Long functions generally indicate that they are doing too many things. Small functions are better to read and faster to understand the purpose. If our function has more than 10 lines, we need to ask ourselves if it would be better to break it into smaller functions. The best functions or methods are from 5 to 10 lines of code. Try to use the [Single Responsibility Principle](https://codeburst.io/understanding-solid-principles-single-responsibility-b7c7ec0bf80). The method itself might be doing one thing, but inside it, there are a few other operations that could be happening. We can extract those methods into their own method and make them do one thing each and use them instead. This is sometimes measured as "cyclomatic complexity". There are also some TSLint rules to detect [cyclomatic/cognitive complexity](https://palantir.github.io/tslint/rules/cyclomatic-complexity/), which we could use in your project to avoid bugs and detect code smells and maintainability issues. + +Avoid change the DOM directly +----------------------------- + +It's important to know that Angular uses Lifecycle Hooks that determine how and when components will be rendered and updated. Direct DOM access or manipulation can corrupt these lifecycle hooks, leading to unexpected behavior of the whole app. Direct access to the DOM can make our application more vulnerable to [XSS attacks](https://owasp.org/www-community/attacks/xss/). Use [ElementRef](https://angular.io/api/core/ElementRef) as the last resort when direct access to DOM is needed. Use templating and data-binding provided by Angular instead. Alternatively we can take a look at [Renderer2](https://angular.io/api/core/Renderer2) which provides API that can safely be used even when direct access to native elements is not supported. + +Avoid computing values in the template +-------------------------------------- + +Sometimes in Angular templates, we may be tempted to bind a method in the HTML template. The problem is that the methods are constantly getting called during the [Angular Change Detection Cycle](https://blog.angular-university.io/how-does-angular-2-change-detection-really-work/). + +``` +@Component({ + selector: 'my-component', + template: `

{{ getTitle() }}

` +}) +export class MyComponent { + getTitle(): string { + return 'Page Title'; + } +} +``` + +It's highly [recommended not to use methods calls](https://medium.com/showpad-engineering/why-you-should-never-use-function-calls-in-angular-template-expressions-e1a50f9c0496) in Angular template expressions. While method calls in Angular templates are super convenient and technically valid, they may cause serious performance issues because the method is called every time change detection runs. Instead, we can use pure [pipes](https://angular.io/guide/pipes) or manually calculate the values with [Lifecycle Hooks](https://angular.io/guide/lifecycle-hooks). + +Avoid logic in templates +------------------------ + +If we have any kind of logic in our templates, even if it is a simple && clause, it is good to extract it out into its component. Having logic in the template means that it is not possible to test with isolated unit tests it and therefore it is more prone to bugs when changing template code. Logic should be contained in one place (the component class) instead of being spread in two places. Keeping the component's presentation logic in the class instead of the template improves testability, maintainability, and reusability. + +Use mandatory inputs +-------------------- + +To make the requirement explicit we can use the selector in the `@Component` decorator to require that the attribute on our component must exist. + +``` +@Component({ + selector: 'my-component[items]', +}) +export class MyComponent { + @Input() + items: object[]; +} +``` + +Resulting an error, when we start the application or at compile time when the application is built Ahead of Time (AoT), if the `MyComponent` doesn't have a items attribute. This approach improves the readability of the code because helps other developers to integrate this component into their projects, throwing errors, and we don't need to define explicit validations to check if it exists. + +Provide private services +------------------------ + +Most [providers](https://angular.io/guide/providers) in angular are designed to act on a global scope. They are then provided at an application level (AppModule). This makes sense if the use of the [global-singleton-pattern](https://angular.io/guide/singleton-services) is required. However, there are services that do not need to be provided globally. Especially if they are used by just one component. In that case, it can make sense to provide that service inside of the component, instead of globally. That is, if the service is directly tied to that component, as shown below. + +``` +@Component({ + selector: 'my-component', + providers: [MyService] +}) +export class MyComponent {} +``` + +Providers are [tree-shakable](https://angular.io/guide/dependency-injection-providers#tree-shakable-providers), the Angular compiler removes the associated services from the final output when it determines that our application doesn't use those services. It also minimizes the risk of dead code and reduces the size of our bundles. + +Use OnPush +---------- + +By default, Angular uses the `ChangeDetectionStrategy.Default` [change detection strategy](https://angular.io/api/core/ChangeDetectionStrategy). The Default strategy doesn't assume anything about the application, therefore every time something changes in our application, as a result of various user events, timers, XHR, promises, etc., a change detection will run on all components. This means anything from a click event to data received from an ajax call causes the change detection to be triggered. Now, imagine a big application with thousands of expressions. If we let Angular check every single one of them when a change detection cycle runs, we might encounter a performance problem. We can set the `ChangeDetectionStrategy` of our component to `ChangeDetectionStrategy.OnPush`. This tells Angular that the component only depends on its `@inputs()` and needs to be checked only in the following cases: + +- Input reference of the component changes; +- An event originated from the component or one of its children; +- Emission of an observable event subscribed with `async` pipe; +- Change detection is manually run. + +This practice is even more important for large and complex applications as the number of components skipped by the change detection is substantial. We need to note that when we are working with observable subscriptions, there is no away for Angular to know that we are updating a component attribute. In such cases, it is recommended to subscribe with `async` pipe. + +Use trackBy +----------- + +When using `ngFor` to loop over an array in templates, use it with a `trackBy` function which will return an unique identifier for each item. When an array changes, Angular re-renders the whole DOM tree. But if we use `trackBy`, Angular will know which element has changed and will only make DOM changes for that element. + +``` +@Component({ + selector: 'my-component', + template: `
  • {{ item.name }}
    • ` +}) +export class MyComponent { + @Input() + items: object[]; + + trackByFn(index: number, item: object): number { + return item.id; + } +} +``` + +Use virtual scrolling +--------------------- + +In [virtual scrolling](https://dev.to/adamklein/build-your-own-virtual-scroll-part-i-11ib), we don't display the entire content on the screen, to reduce the amount of DOM node rendering and calculations. We "fool" the user to think the entire content is rendered by always rendering just the part inside the window, and a bit more on the top and bottom to ensure smooth transitions. We can use `` directive from [Angular CDK](https://material.angular.io/cdk/scrolling/). + +Use lazy loading +---------------- + +When possible, try to lazy load the modules in Angular application. [Lazy loading](https://angular.io/guide/lazy-loading-ngmodules) helps keep initial bundle sizes smaller, which in turn helps decrease load times. + +``` +const routes: Routes = [{ + path: "customer-list", + loadChildren: () => import("./customers/customers.module").then(m => m.CustomersModule) +}]; +``` + +Lazy, or "on demand", loading is a great way to optimize our site or application. This practice essentially involves splitting our code at logical breakpoints, and then loading it once the user has done something that requires, or will require, a new block of code. This speed up the initial load of the application and lightens its overall weight as some blocks may never even be loaded. + +Avoid poorly structured CSS +--------------------------- + +Common mistakes are excessive use of deep selectors and inline styles. [Inline styles](https://www.w3schools.com/css/css_howto.asp) are considered as bad practice due to poor scalability and maintainability. As a rule of thumb, define all styles in the CSS files. Usage of [::ng-deep](https://angular.io/guide/component-styles#deprecated-deep--and-ng-deep) to overwrite styles in other components is incredibly popular. Despite being a working solution, it's marked as deprecated. The main reason for that is that this mechanism for piercing the style isolation sandbox around a component can potentially encourage bad styling practices. Though, it isn't going away until Angular implements `::part()` and `::theme()` from the [CSS Shadow Parts](https://drafts.csswg.org/css-shadow-parts/) spec, as there is no better alternative. + +Lack of meaningful unit tests +----------------------------- + +Angular CLI encourages to write unit tests by spanning out `*.spec.ts` files with every created component. However, don't leave them empty or be satisfied by configuring the [TestBed](https://angular.io/api/core/testing/TestBed) with component initialization without actual tests. If developers don't write tests, then absence of a test file would clearly indicate the state of affairs to other developers, rather than misleading them by giving a false sense of security with a rudimental `*.spec.ts` file. We need to cover with tests the most fragile parts, rather than covering what's easier to test. + +Add caching mechanisms +---------------------- + +When making API calls, responses from some of them do not change often. In those cases, we can add a caching mechanism and store the value from the API. When another request to the same API is made, we checked if there is a value for it in the cache and if so, use it. Otherwise, make the API call and cache the result. Having a caching mechanism means avoiding unwanted API calls. By only making the API calls when required and avoiding duplication, the speed of the application improves as we do not have to wait for the network. It also means we do not download the same information repeatedly. + +Avoid magic numbers +------------------- + +Magic numbers are values that appear in source code without any explanation of what they mean. This makes the code difficult to understand and maintain. Magic numbers should be avoided as they often lack documentation. Forcing them to be stored in variables gives them implicit documentation. With [no-magic-numbers](https://palantir.github.io/tslint/rules/no-magic-numbers/) lint rule, we make code more readable and refactoring easier by ensuring that special numbers are declared as constants to make their meaning explicit. + +Avoid useless code comments +--------------------------- + +Comments are considered a best practice, but if we are adding a comment, it's because it's not self-explanatory and we should choose a better way to implement it. + +Good comments are informative comments, when be useful to provide basic information. For example, a comment that contains legal information, or are a warning, when we are working with multiple developers on a project, we could use a comment to warn other developers about certain consequences, or are a to-do comments for tasks a developer thinks should be done, but for some reason can't be done at this moment. + +Bad Comments are commented-out code is a common practice, but we shouldn't do it, because other developers will think the code is there for a reason and won't have the courage to delete it. Just delete the code. We have got version control, so the code isn't lost forever. Another case is noise comments. Some comments that we see are just noise. They restate the obvious and serve no real purpose. Redundant comments are comments that are not more informative than the code. These comments only clutter the code. + +Remove unused code +------------------ + +Unused code or [dead code](https://dev.to/apastuhov/dead-code-problem-3o2) is any code which will never be executed. It may be some condition, loop or any file which was simply created but wasn't used in our project. It is a problem because that code has no sense and we can drop it. [Dead-code Elimination](https://webpack.js.org/guides/tree-shaking/) also reduces the size of our bundles and repositories. Less code also increases maintenance, IDE performance and makes it easier to understand. Common mistakes in TypeScript projects are unused imports, variables, functions and private class members. With [no-unused-variable](https://palantir.github.io/tslint/rules/no-unused-variable/) lint rule are automatically remove unused imports, variables, functions, and private class members, when using TSLint's --fix option. + +Separation of concerns +---------------------- + +Angular is built around separation of concerns. This is a [design-pattern](https://medium.com/@rkay301/programming-fundamentals-part-5-separation-of-concerns-software-architecture-f04a900a7c50) that makes our code easier to maintain and extend, and more reusable and testable. It helps us encapsulate and limit the logic of components to satisfy what the template needs, and nothing more. Separation of concerns is the core of writing clean code in Angular. It's important follow the [Angular Style Guide](https://angular.io/guide/styleguide). It will make our life a lot easier, when coming into new projects and will increase the quality of our code almost automatically. + +Use aliases for imports +----------------------- + +Sometimes, we may use imports three folders deep or more, so the following import is not the ideal solution for the project. + +``` +import { LoaderService } from '../../../loader/loader.service' +``` + +In TypeScript, we can avoid these "bad" looking imports with the help of [path aliases](https://www.typescriptlang.org/docs/handbook/module-resolution.html#path-mapping). With path aliases we can declare aliases that map to a certain absolute path in our application. Path aliases are defined in the `compilerOptions` section in the `tsconfig.json` file. + +``` +import { LoaderService } from ' @app/loader/loader.service' +``` + +Develop in a modular way +------------------------ + +When we start developing in Angular, we may be tempted to disregard creation of modules for the sake of using components solely. That approach might be fine for smaller apps, but as our app starts to grow, development will become cumbersome. That's when separation of concerns steps in, which is fueled by a modular Angular app. Splitting our app into core, shared and multiple feature modules will make our life much easier. Each module can have its own components, services, directives and pipes. Modules help to organize our code into smaller bundles to make finding things easier. With the help of lazy-loaded modules, we can also improve the user experience by only downloading the parts of the application, that are required at that moment. + +Components should only deal with display logic +---------------------------------------------- + +We want our components to be as simple as possible. Avoid having any logic other than the display logic in the components whenever possible and make the component deal only with the display logic. Components are designed for presentational purposes and control what the view should do. This means if our component needs to do some complex logic we need to decide if that logic belongs to the component or not. Any business logic should be extracted into its own methods/services where appropriate, separating business logic from view logic. Business logic is usually easier to unit test when extracted out to a service and can be reused by any other components that need the same business logic applied. + +Small reusable components +------------------------- + +Ideally, a single component should render a specific bit of our page or modify a behavior. It means, we should keep them small, so that one component corresponds to one function. Make the component as dumb as possible, as this will make it work in more scenarios. Making a component dumb means that the component does not have any special logic in it and operates purely based on the inputs and outputs provided to it. Dumb components are simpler, so they are less likely to have bugs. Dumb components make us think harder about the public component API and help sniff out mixed concerns. Generally, the last child in the component tree will be the dumbest of all. Reusable components reduce duplication of code therefore making it easier to maintain and make changes. + +Smart and dummy components +-------------------------- + +Most common use case of developing Angular's components is a separation of [smart and presentation components](https://blog.angular-university.io/angular-2-smart-components-vs-presentation-components-whats-the-difference-when-to-use-each-and-why/). A 'dumb' component is a component used for presentation purposes only, meaning that the component doesn't know where the data came from. For that purpose, we can use one or more smart components that will inherit dummy's component presentation logic. Making a component dumb means that the component does not have any special logic in it and operates purely based on the inputs and outputs provided to it. Dumb components are simpler, so they are less likely to have bugs. A smart component does not have to be a top-level router component only. We can have other components further down the tree and don't necessarily get their data only from `@Input()`.\ +All component templates should be dumb, maybe a better metaphor here is parent child or shell host, the parent or host handles dependencies and hands data to the child this makes the child component easy to test, since dependencies can be stubbed.  If the outer (parent or shell) component handles the data fetch and keeps the children hidden until it has been hydrated, the child component can make certain assumptions about its role and state.  + +Base component classes +---------------------- + +Create a base class component may come in handy when we have lots of reused stuff and don't want to pollute each component with the same code all over. Common situations are when we are creating form components, when we have pages with the same behavior, such as a list with the create, read, update and delete methods, or pages with HTML forms. In these examples, having the same code in multiple places means that if we want to make a change to the logic in that code, we must do it in multiple places. We can create a base class with the common data and methods. Thus, we don't have the same duplicate code in different locations in the code base. + +Do not remove view encapsulation +-------------------------------- + +In Angular, component CSS styles are encapsulated into the component's view and don't affect the rest of the application. To control how this encapsulation happens on a per component basis, we can set the [view encapsulation](https://angular.io/guide/view-encapsulation) mode in the component metadata. The default is Emulated and it emulates the behavior of [Shadow DOM](https://developer.mozilla.org/en-US/docs/Web/Web_Components/Using_shadow_DOM) by preprocessing the CSS code to effectively scope the CSS to the component's view. In the None mode, styles from the component propagate back to the main HTML and therefore are visible to all components on the page. We can use this option, but we need to be careful and adopt other strategies like nested CSS or naming conventions like [BEM](http://getbem.com/introduction/). + +Use reactive forms +------------------ + +Angular presents two different methods for creating forms: [template-driven](https://angular.io/guide/forms) and [reactive forms](https://angular.io/guide/reactive-forms). Reactive forms provide a model-driven approach to handling form inputs whose values change over time. The Reactive approach removes validation logic from the template, keeping the templates clean of validation logic. Reactive forms use an explicit and immutable approach to managing the state of a form at a given point in time. Each change to the form state returns a new state, which maintains the integrity of the model between changes. + +Bibliography +------------ + +- [Angular coding style guide](https://angular.io/guide/styleguide) +- [Angular Performance Advice](https://medium.com/slackernoon/angular-performance-enhancement-413c6f950ebb) +- [Angular Bad Practices](https://medium.com/angular-in-depth/angular-bad-practices-revisited-4f607fcb75da) +- [Angular Reactive Templates with ngIf and the Async Pipe](https://blog.angular-university.io/angular-reactive-templates/) +- [Angular Smart Components vs Presentational Components](https://blog.angular-university.io/angular-2-smart-components-vs-presentation-components-whats-the-difference-when-to-use-each-and-why/) +- [A Comprehensive Guide to Angular onPush Change Detection Strategy](https://netbasal.com/a-comprehensive-guide-to-angular-onpush-change-detection-strategy-5bac493074a4) +- [Best practices for a clean and performant Angular application](https://medium.com/free-code-camp/best-practices-for-a-clean-and-performant-angular-application-288e7b39eb6f) +- [Best Practices in Angular](https://itnext.io/best-practices-in-angular-a8926fa02ae2) +- Clean Code Checklist in Angular +- [Don't follow RxJS Best Practices](https://dev.to/nikpoltoratsky/don-t-follow-rxjs-best-practices-4893) +- [RxJS in Angular: When To Subscribe?](https://indepth.dev/posts/1279/rxjs-in-angular-when-to-subscribe-rarely) +- [5 Things that Improve your Angular Codebase Right Now](https://malcoded.com/posts/improve-your-angular-codebase/) +- [6 most common mistakes of Angular devs revealed on code reviews](https://alex-klaus.com/angular-code-review/) diff --git a/source/cloud-native-platform/api-docs/index.html.md.erb b/source/concepts-standards/apis/api-docs/index.html.md.erb similarity index 100% rename from source/cloud-native-platform/api-docs/index.html.md.erb rename to source/concepts-standards/apis/api-docs/index.html.md.erb diff --git a/source/cloud-native-platform/standards/apis.html.md.erb b/source/concepts-standards/apis/index.html.md.erb similarity index 94% rename from source/cloud-native-platform/standards/apis.html.md.erb rename to source/concepts-standards/apis/index.html.md.erb index be7f88b2..dd53b869 100644 --- a/source/cloud-native-platform/standards/apis.html.md.erb +++ b/source/concepts-standards/apis/index.html.md.erb @@ -3,7 +3,6 @@ title: API design last_reviewed_on: 2023-08-16 review_in: 12 months weight: 2 -hide_in_navigation: true --- # <%= current_page.data.title %> diff --git a/source/cloud-native-platform/standards/db-schema-change.html.md.erb b/source/concepts-standards/db-schema/index.html.md.erb similarity index 99% rename from source/cloud-native-platform/standards/db-schema-change.html.md.erb rename to source/concepts-standards/db-schema/index.html.md.erb index 14d9368b..38a5841e 100644 --- a/source/cloud-native-platform/standards/db-schema-change.html.md.erb +++ b/source/concepts-standards/db-schema/index.html.md.erb @@ -3,7 +3,6 @@ title: Database schema changes last_reviewed_on: 2023-02-22 review_in: 6 months weight: 3 -hide_in_navigation: true --- # <%= current_page.data.title %> diff --git a/source/concepts-standards/images/overview.png b/source/concepts-standards/images/overview.png new file mode 100644 index 00000000..2d829f02 Binary files /dev/null and b/source/concepts-standards/images/overview.png differ diff --git a/source/concepts-standards/index.html.md.erb b/source/concepts-standards/index.html.md.erb new file mode 100644 index 00000000..48bb39fc --- /dev/null +++ b/source/concepts-standards/index.html.md.erb @@ -0,0 +1,40 @@ +--- +title: Concepts and standards +last_reviewed_on: 2023-02-24 +review_in: 6 months +weight: 2 +--- + +# <%= current_page.data.title %> +Standards are a low level specification of how a particular technology should be used. Where possible, standards should be implemented through tooling and libraries rather than documentation. + + +# Principles, practices and standards + +DTS define the principles, practices and standards that apply to all digital services in HMCTS. Principles are abstract +ideas and values which DTS holds independent of the specific area of the business. For example, Crime and CFT may have +different Practices or Standards in areas but they hold the same shared Principles. + +![DTS Overview](./images/overview.png) + +### Principles + +Principles are high level concepts that are used to guide the design of the system. They are not specific to any particular component or feature, and can be applied in a broad manner. + +### Practices + +Practices are concepts that are applicable within specific contexts such as a particular language, framework, or technology. + +### Standards + +Standards are a low level specification of how a particular technology should be used. Where possible, standards should be implemented through tooling and libraries rather than documentation. + + +- Apps must adhere to [Twelve-Factor](https://12factor.net) principles +- All pipelines must follow the [common build pipeline](../common-pipeline/) +- Use standardised [environments](../environments/) +- Testing +- Deployment +- Release + - [Feature flags](../new-component/feature-flags.html#feature-flags) +- [Branching strategies](../new-component/feature-flags.html#trunk-based-development) diff --git a/source/concepts-standards/practices/apis.html.md.erb b/source/concepts-standards/practices/apis.html.md.erb new file mode 100644 index 00000000..86dad5f7 --- /dev/null +++ b/source/concepts-standards/practices/apis.html.md.erb @@ -0,0 +1,24 @@ +--- +title: APIs +weight: 3 +--- + +# <%= current_page.data.title %> + +### API Design and implementation + +Use the [HMCTS RESTful API standards](https://hmcts.github.io/restful-api-standards/). + +### Versioning + +Use content type negotiation or endpoint naming to version the API. + +Do not make backward incompatible changes to existing endpoints. + +See the official guidance in the [TGL](https://tools.hmcts.net/confluence/pages/viewpage.action?pageId=1611204737&__ncforminfo=TeykTRO7uMi-mDnly8SmjuKluB1u3rrwwlwGkfYwyuBtTvzhtdiq1QymGuTr5WAl0bULwsGc4_Kv4lvc4ja_Tf4s4wSlGOiC) + +### Documentation + +Use [OpenAPI](https://swagger.io/specification/) for API documentation. + +Maintain a central repository of API documentation. diff --git a/source/concepts-standards/practices/backends.html.md.erb b/source/concepts-standards/practices/backends.html.md.erb new file mode 100644 index 00000000..0d3aea76 --- /dev/null +++ b/source/concepts-standards/practices/backends.html.md.erb @@ -0,0 +1,39 @@ +--- +title: Backend applications +weight: 2 +--- + +# <%= current_page.data.title %> + +APIs should be written using Java, preferably with an industry standard framework such. + +### Java + +Use the current LTS version. Ensure that there is a process in place to upgrade it as new versions released. + +### Dependency management + +Ensure dependabot or renovate has been configured so that dependencies are kept up to date. + +### Build tools + +Use gradle or maven to build the application. Ant is not recommended. + +### Code style + +Use the [Google Java Style Guide](https://google.github.io/styleguide/javaguide.html) or a derivative. + +Set up tooling to enforce coding standards such as checkstyle or spotless. + +### Security + +Use [OWASP](https://owasp.org/) or [Snyk](https://snyk.io/) for dependency scanning. + +### Documentation + +Use [OpenAPI](https://swagger.io/specification/) for API documentation. + +Using JavaDoc is not necessary. + +Documentation should not explain what the code is doing, but can be used to clarify why it is doing it. + diff --git a/source/concepts-standards/practices/certificates.html.md.erb b/source/concepts-standards/practices/certificates.html.md.erb new file mode 100644 index 00000000..4bd2af94 --- /dev/null +++ b/source/concepts-standards/practices/certificates.html.md.erb @@ -0,0 +1,19 @@ +--- +title: Certificates +weight: 4 +--- + +# <%= current_page.data.title %> + +### Automate + +Ensure that SSL certificates are automated and renewed regularly. + +Use services such as [Microsoft managed certificates](https://docs.microsoft.com/en-us/azure/frontdoor/front-door-custom-domain-https#option-1-default-use-a-certificate-managed-by-front-door) or [LetsEncrypt](https://letsencrypt.org/) to automate the process of obtaining and renewing SSL certificates. + +### TLS + +Use TLS 1.3 with a 2048-bit RSA or 256-bit ECDSA key. + +Do not use TLS 1.0 or 1.1. + diff --git a/source/concepts-standards/practices/frontend.html.md.erb b/source/concepts-standards/practices/frontend.html.md.erb new file mode 100644 index 00000000..e4f42f52 --- /dev/null +++ b/source/concepts-standards/practices/frontend.html.md.erb @@ -0,0 +1,54 @@ +--- +title: Frontend applications +weight: 1 +--- + +# <%= current_page.data.title %> + +Citizen frontend applications should be built using a node.js based Server Side Rendering (SSR) framework such as Express. + +Professional frontend applications should be built using a node.js based Single Page Application (SPA) framework such as Angular. + +Both citizen and professional applications should be based on the [GOV.UK Design System](https://design-system.service.gov.uk/) and use +[GOV.UK Frontend](https://frontend.design-system.service.gov.uk/) or a derivative such as [HMCTS Frontend](https://hmcts.github.io/hmcts-frontend/). + +### Node.js + +Use the current LTS version. Ensure that there is a process in place to upgrade it as new versions released. + +Deno is not currently recommended. + +### Dependency management + +Ensure dependabot or renovate has been configured so that dependencies are kept up to date. + +NPM or Yarn is acceptable. + +### TypeScript + +TypeScript is recommended for all new projects. + +### Browser support + +Applications should support the latest version of Chrome, Edge, Firefox and Safari. + +Where possible applications should aim to use a responsive layout in order to work on mobile devices. + +### Code style + +Use an opinionated code linter such as eslint and code formatter such as prettier to eliminate discussions about subjective coding preferences. + +### Accessibility + +Use Pa11y for accessibility testing to WCAG 2.1 AA. + +### Cross-browser and UI testing + +Use playwright or cypress to test applications in the browser. + +### Security + +Configure the Content Security Policy headers to prevent XSS attacks. + +Add CSRF protection to forms to ensure that they cannot be submitted by a third party. + diff --git a/source/concepts-standards/practices/index.html.md.erb b/source/concepts-standards/practices/index.html.md.erb new file mode 100644 index 00000000..7cb1b79f --- /dev/null +++ b/source/concepts-standards/practices/index.html.md.erb @@ -0,0 +1,13 @@ +--- +title: Practices +weight: 2 +# hide_in_navigation: true +--- + +# <%= current_page.data.title %> + +Practices are concepts that are applicable within specific contexts such as a particular language, framework, or technology. + +<% current_page.children.each do |page| %> +- [<%= page.data.title %>](<%= page.url %>) +<% end %> diff --git a/source/concepts-standards/principles/continuous-delivery.html.md.erb b/source/concepts-standards/principles/continuous-delivery.html.md.erb new file mode 100644 index 00000000..edaf0931 --- /dev/null +++ b/source/concepts-standards/principles/continuous-delivery.html.md.erb @@ -0,0 +1,44 @@ +--- +title: Continuous delivery +weight: 2 +--- + +# <%= current_page.data.title %> + +All teams should use a Continuous Integration and Continuous Delivery pipeline. + +[Trunk Based Development](https://trunkbaseddevelopment.com/) is the recommended branching strategy for continuous delivery: + +- All changes integrate into the trunk +- If branches from the trunk are used: + - They originate from the trunk + - They re-integrate to the trunk + - They are short-lived and removed after the merge + +Git-flow is [not recommended](https://nvie.com/posts/a-successful-git-branching-model/), even by the original author. + +## Principles of Continuous Delivery + +### Early feedback + +Ensure automated tests are run as often as possible (preferably when ever a chance is made) to the branch which is to be merged in. + +### Early integration + +Integrating early and often will help avoid complicated merge conflicts and ensure that the code is always in a releasable state. + +### Early business input + +To ensure smooth flow of work being deployed, it is important that The Business/Product Owner sign off tickets to be released to production as soon as possible. Before work begins, it needs to be agreed whether or not a Feature Flag is needed to allow for code to be deployed into production, prior to a business release being approved. This could be to allow for time for staff training or legislation to be announced. + +### Release as frequently as possible + +The more often a change is released, the lower the risk of a change being deployed for a long time and the easier it is to rollback. Releasing often gives developers confidence in, and understanding of the deployment process. Pipelines should be well maintained and reliable. + +### Release as small as possible + +The smaller the change, the lower the risk, the easier to rollback and the simpler to debug any issues which may arise from the deployment. + +### Backward compatible only + +Avoid making any breaking changes to ensure rollbacks are very simple (or fix forward). diff --git a/source/concepts-standards/principles/devops.html.md.erb b/source/concepts-standards/principles/devops.html.md.erb new file mode 100644 index 00000000..3f119a5e --- /dev/null +++ b/source/concepts-standards/principles/devops.html.md.erb @@ -0,0 +1,26 @@ +--- +title: DevOps +weight: 3 +--- + +# <%= current_page.data.title %> + +DevOps is not a job title. It's a culture. It is not just about infrastructure. It's about the people, processes, and tools that make it all work. + +### Infrastructure as code + +Infrastructure should be provisioned from code. This allows for the infrastructure to be versioned, tested, and deployed. It also allows for the infrastructure to be easily recreated in the event of a disaster. + +### You build it, you run it + +Delivery team's should be responsible for their own infrastructure. This means team's should be able to troubleshoot and fix issues with the infrastructure. + +Applications should be run, monitored and scaled by the team that built it. + +### Automate everything + +Everything should be automated. This includes provisioning, configuration, deployment, and monitoring. + +### Containers + +Use containers to package applications. This allows for applications to be easily deployed and scaled. diff --git a/source/concepts-standards/principles/images/testing-pyramid.png b/source/concepts-standards/principles/images/testing-pyramid.png new file mode 100644 index 00000000..333a44af Binary files /dev/null and b/source/concepts-standards/principles/images/testing-pyramid.png differ diff --git a/source/concepts-standards/principles/index.html.md.erb b/source/concepts-standards/principles/index.html.md.erb new file mode 100644 index 00000000..cb989a4c --- /dev/null +++ b/source/concepts-standards/principles/index.html.md.erb @@ -0,0 +1,12 @@ +--- +title: Principles +weight: 2 +--- + +# <%= current_page.data.title %> + +Principles are high level concepts that are used to guide the design of the system. They are not specific to any particular component or feature, and can be applied in a broad manner. + +<% current_page.children.each do |page| %> +- [<%= page.data.title %>](<%= page.url %>) +<% end %> diff --git a/source/concepts-standards/principles/programming.html.md.erb b/source/concepts-standards/principles/programming.html.md.erb new file mode 100644 index 00000000..35ab2699 --- /dev/null +++ b/source/concepts-standards/principles/programming.html.md.erb @@ -0,0 +1,23 @@ +--- +title: Programming +weight: 2 +--- + +# <%= current_page.data.title %> + +## General principles + +- YAGNI - [You Aren't Gonna Need It](https://en.wikipedia.org/wiki/You_aren%27t_gonna_need_it) +- KISS - [Keep It Simple, Stupid](https://en.wikipedia.org/wiki/KISS_principle) +- Security baked-in not bolted on - [Cloud Security Principles](https://www.ncsc.gov.uk/collection/cloud/the-cloud-security-principles) +- Transparency - [work in the open](https://www.gov.uk/guidance/be-open-and-use-open-source) +- Peer review - [nothing gets to production without peer review](https://en.wikipedia.org/wiki/Peer_review) +- Egoless programming - [you are not your code](https://blog.codinghorror.com/the-ten-commandments-of-egoless-programming/) +- Domain Driven Design - [design around the domain](https://en.wikipedia.org/wiki/Domain-driven_design) + +## Object oriented programming + +- SOLID - [Single Responsibility, Open/Closed, Liskov Substitution, Interface Segregation, Dependency Inversion](https://en.wikipedia.org/wiki/SOLID) +- Composition over inheritance - [prefer composition over inheritance](https://en.wikipedia.org/wiki/Composition_over_inheritance) +- Law of Demeter - [talk to friends, not strangers](https://en.wikipedia.org/wiki/Law_of_Demeter) + diff --git a/source/concepts-standards/principles/test.html.md.erb b/source/concepts-standards/principles/test.html.md.erb new file mode 100644 index 00000000..ee4d6f7c --- /dev/null +++ b/source/concepts-standards/principles/test.html.md.erb @@ -0,0 +1,63 @@ +--- +title: Testing +weight: 2 +--- + +# <%= current_page.data.title %> + +Your tests should be: + +### Automated + +Tests should be automated. They should be run as part of the build process. They should be run as part of the deployment process. + +### Reliable + +Tests should be reliable. They should not fail randomly. They should not fail when the system is under load. + +### Fast + +Tests should be fast. If they are slow, they will be ignored. + +### Repeatable + +Tests should be repeatable. They should be able to run in any environment. They should be able to run in parallel. + +### Maintainable + +Tests should be maintainable. They should be easy to understand. They should be easy to write. They should be easy to maintain. + +## Types of testing + +![Testing pyramid](./images/testing-pyramid.png) + +UI tests are more valuable as they are closer to the user but they are more expensive to write and maintain. Unit tests are faster and easier to write. + +The number of tests should be a structured as a pyramid: more tests to the lower layer of the pyramid and less at the end-to-end layer. + +### Unit Tests + +Write lots of small and fast unit tests. Unit tests are targeted at an individual class level, each class generally have a corresponding test class that tests the internals of that class. + +They should ensure that all your non-trivial code paths are tested (including happy path and edge cases). + +### Integration tests + +Integration tests test the integration of your application with all the parts that live outside of your application. + +Write integration tests for each integrated application in isolation. + +Use the Arrange, Act, Assert pattern: + +| | Integration with REST API | Integration with DB | +|---|---|---| +| Arrange | 1. Start your application and stubs (e.g. WireMock) | 1. Start your application and database | +| Act | 2. Invoke the function in your application that requests the response from the 3rd party | 2. Invoke the function in your application that writes the data to the DB | +| Assert | 3. Verify that your application deserialise the response correctly | 3. Inspect the DB whether the data being written correctly | + +### UI tests + +Have a small number of tests that run with the full stack deployed in a real environment that is as close to production as possible. + +Put extra effort into making sure the tests are reliable. + diff --git a/source/cloud-native-platform/standards/pull-requests.html.md.erb b/source/concepts-standards/pull-requests/index.html.md.erb similarity index 98% rename from source/cloud-native-platform/standards/pull-requests.html.md.erb rename to source/concepts-standards/pull-requests/index.html.md.erb index 7c50dc81..e51bf943 100644 --- a/source/cloud-native-platform/standards/pull-requests.html.md.erb +++ b/source/concepts-standards/pull-requests/index.html.md.erb @@ -3,7 +3,7 @@ title: Pull requests last_reviewed_on: 2023-02-23 review_in: 6 months weight: 1 -hide_in_navigation: true +# hide_in_navigation: true --- # <%= current_page.data.title %> @@ -18,7 +18,7 @@ As a PR reviewer I will: - approve, as long as it’s better than before - favour suggestions over rejections, particularly when a multipart PR is indicated -We practise code review on all changes at HMCTS to: +We practice code review on all changes at HMCTS to: - improve code quality - get team buy in on changes diff --git a/source/concepts-standards/technology-stack/index.html.md.erb b/source/concepts-standards/technology-stack/index.html.md.erb new file mode 100644 index 00000000..3e4e3af0 --- /dev/null +++ b/source/concepts-standards/technology-stack/index.html.md.erb @@ -0,0 +1,13 @@ +--- +title: Technology stack +weight: 4 +--- + +# <%= current_page.data.title %> + +| Layer | Technology | +| --- | --- | --- | +| Frontend - citizen | Node.js (Express / other SSR) | +| Frontend - professional user | Angular | +| API | Java | +| Database | PostgresSQL | diff --git a/source/cloud-native-platform/glossary/index.html.md.erb b/source/glossary/index.html.md.erb similarity index 100% rename from source/cloud-native-platform/glossary/index.html.md.erb rename to source/glossary/index.html.md.erb diff --git a/source/hmcts-overview/index.html.md.erb b/source/hmcts-overview/index.html.md.erb index 74a3769b..0fcbdb02 100644 --- a/source/hmcts-overview/index.html.md.erb +++ b/source/hmcts-overview/index.html.md.erb @@ -1,6 +1,7 @@ --- title: HMCTS overview -weight: 2 +weight: 10 +hide_in_navigation: true --- # <%= current_page.data.title %> diff --git a/source/cloud-native-platform/common-pipeline/common-pipeline.html.md.erb b/source/new-components/common-pipeline/common-pipeline.html.md.erb similarity index 100% rename from source/cloud-native-platform/common-pipeline/common-pipeline.html.md.erb rename to source/new-components/common-pipeline/common-pipeline.html.md.erb diff --git a/source/cloud-native-platform/common-pipeline/index.html.md.erb b/source/new-components/common-pipeline/index.html.md.erb similarity index 98% rename from source/cloud-native-platform/common-pipeline/index.html.md.erb rename to source/new-components/common-pipeline/index.html.md.erb index 95a5b03b..e457b6c8 100644 --- a/source/cloud-native-platform/common-pipeline/index.html.md.erb +++ b/source/new-components/common-pipeline/index.html.md.erb @@ -2,7 +2,7 @@ title: The common pipeline last_reviewed_on: 2022-05-11 review_in: 12 months -weight: 3 +weight: 1 --- # <%= current_page.data.title %> diff --git a/source/cloud-native-platform/common-pipeline/publishing-libraries/index.html.md.erb b/source/new-components/common-pipeline/publishing-libraries/index.html.md.erb similarity index 100% rename from source/cloud-native-platform/common-pipeline/publishing-libraries/index.html.md.erb rename to source/new-components/common-pipeline/publishing-libraries/index.html.md.erb diff --git a/source/cloud-native-platform/common-pipeline/publishing-libraries/java.html.md.erb b/source/new-components/common-pipeline/publishing-libraries/java.html.md.erb similarity index 100% rename from source/cloud-native-platform/common-pipeline/publishing-libraries/java.html.md.erb rename to source/new-components/common-pipeline/publishing-libraries/java.html.md.erb diff --git a/source/cloud-native-platform/common-pipeline/publishing-libraries/nodejs.html.md.erb b/source/new-components/common-pipeline/publishing-libraries/nodejs.html.md.erb similarity index 100% rename from source/cloud-native-platform/common-pipeline/publishing-libraries/nodejs.html.md.erb rename to source/new-components/common-pipeline/publishing-libraries/nodejs.html.md.erb diff --git a/source/cloud-native-platform/environments/aks-auto-shutdown.html.md.erb b/source/new-components/environments/aks-auto-shutdown.html.md.erb similarity index 100% rename from source/cloud-native-platform/environments/aks-auto-shutdown.html.md.erb rename to source/new-components/environments/aks-auto-shutdown.html.md.erb diff --git a/source/cloud-native-platform/environments/external-ip-addresses.html.md.erb b/source/new-components/environments/external-ip-addresses.html.md.erb similarity index 100% rename from source/cloud-native-platform/environments/external-ip-addresses.html.md.erb rename to source/new-components/environments/external-ip-addresses.html.md.erb diff --git a/source/cloud-native-platform/environments/index.html.md.erb b/source/new-components/environments/index.html.md.erb similarity index 100% rename from source/cloud-native-platform/environments/index.html.md.erb rename to source/new-components/environments/index.html.md.erb diff --git a/source/cloud-native-platform/environments/sandbox-cleardown.html.md.erb b/source/new-components/environments/sandbox-cleardown.html.md.erb similarity index 100% rename from source/cloud-native-platform/environments/sandbox-cleardown.html.md.erb rename to source/new-components/environments/sandbox-cleardown.html.md.erb diff --git a/source/new-components/guides/afd-waf.html.md.erb b/source/new-components/guides/afd-waf.html.md.erb new file mode 100644 index 00000000..96fe739d --- /dev/null +++ b/source/new-components/guides/afd-waf.html.md.erb @@ -0,0 +1,32 @@ +--- +title: Azure Frontdoor Web Application Firewall Debug +last_reviewed_on: 2022-10-10 +review_in: 6 months +weight: 10 +hide_in_navigation: true +--- + +# <%= current_page.data.title %> + +When testing through the Azure Frontdoor you may be presented with an error page saying 'The request is blocked', which normally results in being a Web Application Firewall(WAF) rule being blocked. + +![Azure Frontdoor WAF Error Page](/images/waf-error.png) + +## Investigate + +You can find out what the blocked rule is by following these steps: + +1. Go to the spective Azure Frontdoor `hmcts-{env}` for CFT and `sdshmcts-{env}` for SDS. [Azure Frontdoors](https://portal.azure.com/#view/HubsExtension/BrowseResource/resourceType/microsoft.cdn%2Fprofiles) +2. Go down the side menu to `Monitoring > Logs` +3. Go to the query and add the query below, updating `{Request URL}` with the URL you are requesting. example `toffee.aat.platform.hmcts.net`

    +``` +AzureDiagnostics +| where ResourceProvider == "MICROSOFT.NETWORK" and Category == "FrontdoorAccessLog" and requestUri_s contains "{Request URL}" +``` +4. You can change the `Time range` to scope the search down. + +For more information you can look at the [Azure Documentation](https://learn.microsoft.com/en-us/azure/web-application-firewall/afds/waf-front-door-monitor?pivots=front-door-standard-premium#logs-and-diagnostics) + +## Exclude Rules + +If you need to allow this block through then you can update the exclusion rules for your frontdoor configuration following the [Azure Platform Terraform ReadMe](https://github.com/hmcts/azure-platform-terraform#WAF-Exclusions) diff --git a/source/new-components/guides/connect-via-vpn.html.md.erb b/source/new-components/guides/connect-via-vpn.html.md.erb new file mode 100644 index 00000000..e6317df4 --- /dev/null +++ b/source/new-components/guides/connect-via-vpn.html.md.erb @@ -0,0 +1,30 @@ +--- +title: Make a new Virtual Network accessible over the VPN +last_reviewed_on: 2023-02-24 +review_in: 6 months +weight: 10 +hide_in_navigation: true +--- + +# <%= current_page.data.title %> + +This page describes how to expose a new Virtual Network to the HMCTS VPN. +To do this you will need to setup network peering between the VPN and your Virtual Network. +Your project will need to be managed by Terraform to set this up. + +This is an example of the Terraform used to [setup peering between two Vnets](https://github.com/hmcts/aks-sds-deploy/blob/0061d448847a7a16154ae3a23772c4e9a57587f1/components/01-network/21-peering.tf#L82-L104). + +The destination Virtual Network details are: + +- vnet name: `core-infra-vnet-mgmt` +- subscription id: `ed302caf-ec27-4c64-a05e-85731c3ce90e` +- vnet resource group: `rg-mgmt` +- net subscription: `Reform-CFT-Mgmt` + +Once this is setup and deployed, you can then make a request to PlatOps in [\#platops-help](https://hmcts-reform.slack.com/app_redirect?channel=platops-help). Raise a new request using the 'Help request with plato' shortcut, which will automatically raise a ticket for you and all messages will be synced to JIRA. + +Raise a request to add the Address Space for your Vnet or Virtual Machine with what ports to whitelist, please include: + +- Network name: +- IP Range: +- Ports to allow: 443 (sometimes 80, 22 as well) diff --git a/source/new-components/guides/creating-a-new-subscription.html.md.erb b/source/new-components/guides/creating-a-new-subscription.html.md.erb new file mode 100644 index 00000000..cfd42780 --- /dev/null +++ b/source/new-components/guides/creating-a-new-subscription.html.md.erb @@ -0,0 +1,15 @@ +--- +title: Creating a new subscription +last_reviewed_on: 2022-11-03 +review_in: 6 months +weight: 10 +hide_in_navigation: true +--- + +# <%= current_page.data.title %> + +Subscriptions are managed via a GitHub repository [hmcts/azure-enterprise](https://github.com/hmcts/azure-enterprise). + +The [README](https://github.com/hmcts/azure-enterprise/blob/main/README.md) in the repository contains full details on the procedure that must be followed to add a new subscription to the Azure AD tenant. + +<%= warning_text('Make sure to seek approval from the Head of Platform Operations via email and attach evidence of this approval to the pull request you submit.')%> diff --git a/source/cloud-native-platform/guides/flyway-database-migrations.html.md.erb b/source/new-components/guides/flyway-database-migrations.html.md.erb similarity index 100% rename from source/cloud-native-platform/guides/flyway-database-migrations.html.md.erb rename to source/new-components/guides/flyway-database-migrations.html.md.erb diff --git a/source/cloud-native-platform/guides/index.html.md.erb b/source/new-components/guides/index.html.md.erb similarity index 100% rename from source/cloud-native-platform/guides/index.html.md.erb rename to source/new-components/guides/index.html.md.erb diff --git a/source/new-components/guides/managing-manual-key-vault-secrets.html.md.erb b/source/new-components/guides/managing-manual-key-vault-secrets.html.md.erb new file mode 100644 index 00000000..ec4435dd --- /dev/null +++ b/source/new-components/guides/managing-manual-key-vault-secrets.html.md.erb @@ -0,0 +1,71 @@ +--- +title: Managing Manual Key Vault Secrets +last_reviewed_on: 2022-10-10 +review_in: 6 months +weight: 9 +hide_in_navigation: true +--- + +# <%= current_page.data.title %> + +Some secrets cannot be automated via Terraform and therefore we need to add them manually to the Key Vaults. + +Please note that we would advise to always go down the Terraform route where possible. + +## Manual Secrets + +The easiest option is to add the secrets manually directly to the project Key Vault. +Using the [CNP-Module-Key-Vault](https://github.com/hmcts/cnp-module-key-vault) will grant the Developers group the correct permissions to enable you to add these. + +We would also advise to tag your secrets to easily identify between Terraform and manually created secrets. + +## Secret Store + +You can create a secret store much like how Azure DevOps has a Library for secure values. + +The concept here is a separate Key Vault for your manually created secrets and then your project Key Vault which will only contain secrets created by Terraform. + + + +### Setup Secret Store + +Start by creating your Secret Store Key Vault Repositories following Create a Github Repository. +You can then add your Key Vault using the [CNP-Module-Key-Vault](https://github.com/hmcts/cnp-module-key-vault) for example [PIP-Shared-Infrastructure-Bootstrap](https://github.com/hmcts/pip-shared-infrastructure-bootstap) + +### Import Secrets to Project Key Vault + +Now in your project Key Vault you can use the below example to import named secrets. +These can be generic secrets shared between environments, or you can add environment specific secrets. + +This can also be use for certficates as per this example [PIP tf-kv-certs-bootstrap](https://github.com/hmcts/pip-shared-infrastructures/blob/master/tf-kv-certs-bootstrap.tf) + +```terraform +locals { + secret_store_secrets = ["example-secret", "environment-specific-secret-${var.env}"] +} + +data "azurerm_key_vault" "secret_store" { + name = "${var.product}-store-kv-${var.env}" + resource_group_name = "${var.product}-store-${var.env}-rg" +} + +data "azurerm_key_vault_secret" "secret_store" { + for_each = { for secret in local.secret_store : secret => secret } + name = each.value + key_vault_id = data.azurerm_key_vault.secret_store.id +} + +resource "azurerm_key_vault_secret" "secret_store" { + for_each = data.azurerm_key_vault_secret.secret_store + key_vault_id = module.this.key_vault_id + name = each.value.name + value = each.value.value + tags = merge(var.tags, { + "source" : "secret store ${data.azurerm_key_vault.bootstrap_kv.name} secrets" + }) + content_type = "Manual Secret" +} +``` + + +You can also access these via Jenkins for Smoke and Functional Tests following the guide on [cnp-jenkins-library](https://github.com/hmcts/cnp-jenkins-library#secrets-for-functional--smoke-testing) diff --git a/source/new-components/guides/pr-url.html.md.erb b/source/new-components/guides/pr-url.html.md.erb new file mode 100644 index 00000000..af86edf7 --- /dev/null +++ b/source/new-components/guides/pr-url.html.md.erb @@ -0,0 +1,17 @@ +--- +title: How to get the test URL for a PR build +last_reviewed_on: 2022-05-17 +review_in: 6 months +weight: 3 +hide_in_navigation: true +--- + +# <%= current_page.data.title %> + +## How to get the test URL for a PR build + +> You will need to be connected to the HMCTS VPN for this to work. + +Click the 'View deployment' deployment button next to the hmcts-jenkins-cnp bot entry on the conversation tab of the PR that you would like to get the URL for. + +e.g. [hmcts/pip-frontend#79:](https://github.com/hmcts/pip-frontend/pull/79) diff --git a/source/new-components/guides/profile-java-app-in-aks.html.md.erb b/source/new-components/guides/profile-java-app-in-aks.html.md.erb new file mode 100644 index 00000000..64e071ec --- /dev/null +++ b/source/new-components/guides/profile-java-app-in-aks.html.md.erb @@ -0,0 +1,124 @@ +--- +title: Profile a Java application running in an AKS cluster +last_reviewed_on: 2022-07-13 +review_in: 6 months +weight: 10 +hide_in_navigation: true +--- + +# <%= current_page.data.title %> + +This page describes how to profile a java application which is deployed in one of our AKS clusters. + +## Prerequisites +1. Your user will need to have permissions to run kubectl exec and to ssh to a node. (Subscription contributor role works) +2. Install [kubectl node-shell](https://github.com/kvaps/kubectl-node-shell) which in turn requires [krew](https://krew.sigs.k8s.io/) +3. Something to open .jfr files with like [Java Mission Control](https://adoptopenjdk.net/jmc.html) + +## Setup +You'll need the following bits of information during this exercise: +1. namespace (hopefully you just know this or browse `kubectl get namespaces`) +2. pod name (find it in `kubectl get pods -n $namespace`) +3. container name (`kubectl get pods -n $namespace $podName -o=jsonpath='{.spec.containers[0].name}'`) +4. node name (`kubectl get pods -n $namespace $podName -o=jsonpath='{.spec.nodeName}'`) + +Example + +```shell +namespace=rpe +podName=rpe-send-letter-service-java-7fc6fffdf9-qvfd6 +containerName=$(kubectl get pods -n $namespace $podName -o=jsonpath='{.spec.containers[0].name}') +nodeName=$(kubectl get pods -n $namespace $podName -o=jsonpath='{.spec.nodeName}') +``` + +## Running Java Flight Recorder (JFR) +After completing the setup above you can run +`kubectl exec $podName -n $namespace -- jattach 1 jcmd "JFR.start name=profile1"` +The output will look something like this + +```shell +Connected to remote JVM +Response code = 0 +Started recording 1. No limit specified, using maxsize=250MB as default. +Use jcmd 1 JFR.dump name=1 filename=FILEPATH to copy recording data to file. +``` + +At this point you can run any tests you want to capture any data in the profile report. + +Once complete, you can dump the data to a file on the container: +`kubectl exec $podName -n $namespace -- jattach 1 jcmd JFR.dump` +If you don't specify a name in the command, a time-stamped filename will be generated. +The output will look something like this + +```shell +Connected to remote JVM +Response code = 0 +Dumped recording, 4.5 MB written to: +/opt/app/hotspot-pid-1-2022_01_20_14_32_49.jfr +``` +Make a note of the filename for retrieving later + +Run `kubectl exec $podName -n $namespace -- jattach 1 jcmd "JFR.stop name=profile1"` +to stop the recording. You can use `kubectl exec $podName -n $namespace -- jattach 1 jcmd JFR.check` +to list all in-flight recordings + +## Downloading the file +As our images don't have a shell installed or tar (can't use `kubectl cp`) it's a little bit more involved to get the +JFR report. + +To figure out where the file is, you need to use nsenter to get a shell onto the node and find the mount point for the +specific container you need. + +```shell +kubectl node-shell $nodeName -n admin +spawning "nsenter-x8uq64" on "aks-linux-81166528-vmss00005t" +If you don't see a command prompt, try pressing enter. +root@aks-linux-81166528-vmss00005T:/# +``` + +Make a note of the name of the spawned instance (in this case `nsenter-x8uq64`). + +At this point you'll need your variables from earlier + +```shell +namespace=rpe +containerName=rpe-send-letter-service-java +pod=rpe-send-letter-service-java-6fb55dd776-87b6f +``` + +You'll also need some additional information out of containerd using crictl. + +```shell +podId=$(crictl pods --namespace $namespace --name $pod --state Ready --quiet) +containerId=$(crictl ps -p $podId --no-trunc --label io.kubernetes.container.name=$containerName -q) +targetPid=$(crictl inspect $containerId | jq .info.pid) +``` + +Now we can use nsenter to create us a shell at the mount point for the container + +```shell +nsenter --target $targetPid --uts --net --pid unshare --mount-proc sh -c "cd /run/containerd/io.containerd.runtime.v2.task/k8s.io/$containerId/rootfs/opt/app && exec bash" +groups: cannot find name for group ID 11 +To run a command as administrator (user "root"), use "sudo ". +See "man sudo_root" for details. +``` + +When we list the files you can see the hotspot JFR file which was dumped by jattach earlier. + +```shell +root@rpe-send-letter-service-java-6fb55dd776-87b6f:/run/containerd/io.containerd.runtime.v2.task/k8s.io/9704b2c0b9b559001eb23156a06e40d25e10ceeffc4849bb51b1688025da6f61/rootfs/opt/app# ls +AI-Agent.xml hotspot-pid-1-2022_01_20_14_32_49.jfr +applicationinsights-agent-2.5.1.jar send-letter-service.jar +``` + +To download this file we need to go back to a terminal on our local machine and run `kubectl cp` to copy from the spawned instance + (`nsenter-x8uq64`) using the path we just discovered above + (`/run/containerd/io.containerd.runtime.v2.task/k8s.io/9704b2c0b9b559001eb23156a06e40d25e10ceeffc4849bb51b1688025da6f61/rootfs/opt/app`) +appended by the file name `hotspot-pid-1-2022_01_20_14_32_49.jfr` like so: + +```shell +kubectl cp admin/nsenter-x8uq64:/run/containerd/io.containerd.runtime.v2.task/k8s.io/9704b2c0b9b559001eb23156a06e40d25e10ceeffc4849bb51b1688025da6f61/rootfs/opt/app/hotspot-pid-1-2022_01_20_14_32_49.jfr ./hotspot-pid-1-2022_01_20_14_32_49.jfr +tar: Removing leading `/' from member names +``` + +You can now open this file locally using Java Mission Control and terminate your connection to the nsenter pod. diff --git a/source/cloud-native-platform/standards/index.html.md.erb b/source/new-components/index.html.md.erb similarity index 50% rename from source/cloud-native-platform/standards/index.html.md.erb rename to source/new-components/index.html.md.erb index 27374bbb..7ad4b2d1 100644 --- a/source/cloud-native-platform/standards/index.html.md.erb +++ b/source/new-components/index.html.md.erb @@ -1,8 +1,8 @@ --- -title: Concepts and standards +title: New Components last_reviewed_on: 2023-02-24 review_in: 6 months -weight: 2 +weight: 4 --- # <%= current_page.data.title %> @@ -13,8 +13,5 @@ weight: 2 - Testing - Deployment - Release - - [Feature flags](../new-component/feature-flags.html#feature-flags) -- [API design](apis.html) -- [Branching strategies](../new-component/feature-flags.html#trunk-based-development) -- [Pull requests](pull-requests.html) -- [Making a database schema change](db-schema-change.html) + - [Feature flags](./new-component/feature-flags.html#feature-flags) +- [Branching strategies](./new-component/feature-flags.html#trunk-based-development) diff --git a/source/cloud-native-platform/new-component/elasticsearch.html.md.erb b/source/new-components/new-component/elasticsearch.html.md.erb similarity index 94% rename from source/cloud-native-platform/new-component/elasticsearch.html.md.erb rename to source/new-components/new-component/elasticsearch.html.md.erb index 1f00e44d..0d8498a6 100644 --- a/source/cloud-native-platform/new-component/elasticsearch.html.md.erb +++ b/source/new-components/new-component/elasticsearch.html.md.erb @@ -17,5 +17,5 @@ The following process must be adhered to for teams wanting to consume ElasticSea 1. Contact CCD and talk through your requirements with them explaining your circumstances and why you want to use ES in prod 2. Provide detailed information consumption/ usage thresholds and obtain approval from QA/performance team usage is supportable by the system 3. Ensure the latest definition file you want to be used in prod is applied in the ccd-definition-store -4. Once approval has been obtained from CCD that they are happy for you to go live please raise a DTSPO ticket on Jira with Platform Operations and attach your latest definition file to the ticket. You must include the jurisdiction you wish elasticsearch to pull in. +4. Once approval has been obtained from CCD that they are happy for you to go live please raise a DTSPO ticket on JIRA with Platform Operations and attach your latest definition file to the ticket. You must include the jurisdiction you wish elasticsearch to pull in. 5. After Platform Operations have confirmed that the work has been done on their end validate the results. \ No newline at end of file diff --git a/source/cloud-native-platform/new-component/feature-flags.html.md.erb b/source/new-components/new-component/feature-flags.html.md.erb similarity index 100% rename from source/cloud-native-platform/new-component/feature-flags.html.md.erb rename to source/new-components/new-component/feature-flags.html.md.erb diff --git a/source/cloud-native-platform/new-component/github-repo.html.md.erb b/source/new-components/new-component/github-repo.html.md.erb similarity index 100% rename from source/cloud-native-platform/new-component/github-repo.html.md.erb rename to source/new-components/new-component/github-repo.html.md.erb diff --git a/source/cloud-native-platform/new-component/gitops-flux.html.md.erb b/source/new-components/new-component/gitops-flux.html.md.erb similarity index 100% rename from source/cloud-native-platform/new-component/gitops-flux.html.md.erb rename to source/new-components/new-component/gitops-flux.html.md.erb diff --git a/source/cloud-native-platform/new-component/helm-chart.html.md.erb b/source/new-components/new-component/helm-chart.html.md.erb similarity index 100% rename from source/cloud-native-platform/new-component/helm-chart.html.md.erb rename to source/new-components/new-component/helm-chart.html.md.erb diff --git a/source/cloud-native-platform/new-component/index.html.md.erb b/source/new-components/new-component/index.html.md.erb similarity index 98% rename from source/cloud-native-platform/new-component/index.html.md.erb rename to source/new-components/new-component/index.html.md.erb index 1ec46a53..1067129d 100644 --- a/source/cloud-native-platform/new-component/index.html.md.erb +++ b/source/new-components/new-component/index.html.md.erb @@ -2,7 +2,7 @@ title: Starting a new component last_reviewed_on: 2023-03-20 review_in: 6 months -weight: 4 +weight: 3 --- # <%= current_page.data.title %> diff --git a/source/cloud-native-platform/new-component/infrastructure-as-code.html.md.erb b/source/new-components/new-component/infrastructure-as-code.html.md.erb similarity index 100% rename from source/cloud-native-platform/new-component/infrastructure-as-code.html.md.erb rename to source/new-components/new-component/infrastructure-as-code.html.md.erb diff --git a/source/cloud-native-platform/new-component/secrets-management.html.md.erb b/source/new-components/new-component/secrets-management.html.md.erb similarity index 100% rename from source/cloud-native-platform/new-component/secrets-management.html.md.erb rename to source/new-components/new-component/secrets-management.html.md.erb diff --git a/source/cloud-native-platform/onboarding/index.html.md.erb b/source/onboarding/index.html.md.erb similarity index 91% rename from source/cloud-native-platform/onboarding/index.html.md.erb rename to source/onboarding/index.html.md.erb index 66d58440..1fb473eb 100644 --- a/source/cloud-native-platform/onboarding/index.html.md.erb +++ b/source/onboarding/index.html.md.erb @@ -13,7 +13,7 @@ The following guide assumes that people have the following accounts, memberships - `@hmcts.net` or `@justice.gov.uk` email - Member of [HMCTS Reform Slack](https://hmcts-reform.slack.com) -- Jira and Confluence (Usually provided by PMO) +- JIRA and Confluence (Usually provided by PMO) - Member of [HMCTS Github org](https://github.com/hmcts) ## Guides diff --git a/source/cloud-native-platform/onboarding/person/index.html.md.erb b/source/onboarding/person/index.html.md.erb similarity index 98% rename from source/cloud-native-platform/onboarding/person/index.html.md.erb rename to source/onboarding/person/index.html.md.erb index 3dbe0f60..78e0a094 100644 --- a/source/cloud-native-platform/onboarding/person/index.html.md.erb +++ b/source/onboarding/person/index.html.md.erb @@ -65,7 +65,7 @@ The two available tutorials will guide you through deploying your first Java or ## Office 365 -You should have your Office 365 team through the Jira onboarding process, but ask your team lead to add you to any email groups. +You should have your Office 365 team through the JIRA onboarding process, but ask your team lead to add you to any email groups. Often there's a `$team-developers@hmcts.net` and a `$team@hmcts.net` group. ## Docker Business Licence diff --git a/source/cloud-native-platform/onboarding/team/azuread.html.md.erb b/source/onboarding/team/azuread.html.md.erb similarity index 100% rename from source/cloud-native-platform/onboarding/team/azuread.html.md.erb rename to source/onboarding/team/azuread.html.md.erb diff --git a/source/cloud-native-platform/onboarding/team/github.html.md.erb b/source/onboarding/team/github.html.md.erb similarity index 100% rename from source/cloud-native-platform/onboarding/team/github.html.md.erb rename to source/onboarding/team/github.html.md.erb diff --git a/source/cloud-native-platform/onboarding/team/index.html.md.erb b/source/onboarding/team/index.html.md.erb similarity index 84% rename from source/cloud-native-platform/onboarding/team/index.html.md.erb rename to source/onboarding/team/index.html.md.erb index 9a18aa39..14543641 100644 --- a/source/cloud-native-platform/onboarding/team/index.html.md.erb +++ b/source/onboarding/team/index.html.md.erb @@ -5,7 +5,9 @@ review_in: 6 months weight: 1 --- -# Creating a <%= current_page.data.title %> +# <%= current_page.data.title %> + +## Creating a <%= current_page.data.title %> When creating a team, you may need to create some or all of the following: diff --git a/source/cloud-native-platform/onboarding/team/jenkins.html.md.erb b/source/onboarding/team/jenkins.html.md.erb similarity index 100% rename from source/cloud-native-platform/onboarding/team/jenkins.html.md.erb rename to source/onboarding/team/jenkins.html.md.erb diff --git a/source/cloud-native-platform/onboarding/team/office365.html.md.erb b/source/onboarding/team/office365.html.md.erb similarity index 100% rename from source/cloud-native-platform/onboarding/team/office365.html.md.erb rename to source/onboarding/team/office365.html.md.erb diff --git a/source/cloud-native-platform/onboarding/team/slack.html.md.erb b/source/onboarding/team/slack.html.md.erb similarity index 100% rename from source/cloud-native-platform/onboarding/team/slack.html.md.erb rename to source/onboarding/team/slack.html.md.erb diff --git a/source/cloud-native-platform/path-to-live/_front-door-config.md b/source/path-to-live/_front-door-config.md similarity index 100% rename from source/cloud-native-platform/path-to-live/_front-door-config.md rename to source/path-to-live/_front-door-config.md diff --git a/source/cloud-native-platform/path-to-live/front-door.html.md.erb b/source/path-to-live/front-door.html.md.erb similarity index 100% rename from source/cloud-native-platform/path-to-live/front-door.html.md.erb rename to source/path-to-live/front-door.html.md.erb diff --git a/source/cloud-native-platform/path-to-live/index.html.md.erb b/source/path-to-live/index.html.md.erb similarity index 98% rename from source/cloud-native-platform/path-to-live/index.html.md.erb rename to source/path-to-live/index.html.md.erb index c27f4dea..0f16e1d6 100644 --- a/source/cloud-native-platform/path-to-live/index.html.md.erb +++ b/source/path-to-live/index.html.md.erb @@ -2,7 +2,7 @@ title: Path to live last_reviewed_on: 2023-03-17 review_in: 8 months -weight: 5 +weight: 4 --- # <%= current_page.data.title %> diff --git a/source/cloud-native-platform/path-to-live/load-balancer-configuration.html.md.erb b/source/path-to-live/load-balancer-configuration.html.md.erb similarity index 100% rename from source/cloud-native-platform/path-to-live/load-balancer-configuration.html.md.erb rename to source/path-to-live/load-balancer-configuration.html.md.erb diff --git a/source/cloud-native-platform/path-to-live/oat.html.md.erb b/source/path-to-live/oat.html.md.erb similarity index 98% rename from source/cloud-native-platform/path-to-live/oat.html.md.erb rename to source/path-to-live/oat.html.md.erb index ba752a7c..061fda32 100644 --- a/source/cloud-native-platform/path-to-live/oat.html.md.erb +++ b/source/path-to-live/oat.html.md.erb @@ -29,9 +29,9 @@ This covers details on the Operational Acceptance Testing (OAT) performed by Pla ## How do I ask for an OAT? - Requests comes through Release Management. -- Release Management have a set of OAT tasks on the release Jira. +- Release Management have a set of OAT tasks on the release JIRA. - Development team need to assess and comment on each of the OAT tickets on how the changes made are compliant with the criteria on OAT tasks. Refer [OAT Tasks and recommendations](#oat-tasks-and-recommendations) for more guidance. -- Once all tickets and [Service Operations Guide](#service-operations-guide) have been updated, Release management will create a Jira ticket for Platform Operations to validate the OAT tasks. +- Once all tickets and [Service Operations Guide](#service-operations-guide) have been updated, Release management will create a JIRA ticket for Platform Operations to validate the OAT tasks. ## OAT Tasks and recommendations diff --git a/source/cloud-native-platform/path-to-live/public-dns.html.md.erb b/source/path-to-live/public-dns.html.md.erb similarity index 100% rename from source/cloud-native-platform/path-to-live/public-dns.html.md.erb rename to source/path-to-live/public-dns.html.md.erb diff --git a/source/cloud-native-platform/path-to-live/shutter.html.md.erb b/source/path-to-live/shutter.html.md.erb similarity index 100% rename from source/cloud-native-platform/path-to-live/shutter.html.md.erb rename to source/path-to-live/shutter.html.md.erb diff --git a/source/cloud-native-platform/path-to-live/tls-certificates.html.md.erb b/source/path-to-live/tls-certificates.html.md.erb similarity index 100% rename from source/cloud-native-platform/path-to-live/tls-certificates.html.md.erb rename to source/path-to-live/tls-certificates.html.md.erb diff --git a/source/standards/index.html.md.erb b/source/standards/index.html.md.erb index 388fae73..b45a1b1d 100644 --- a/source/standards/index.html.md.erb +++ b/source/standards/index.html.md.erb @@ -1,6 +1,7 @@ --- title: Standards weight: 6 +hide_in_navigation: true --- # Principles, practices and standards diff --git a/source/cloud-native-platform/guides/afd-waf.html.md.erb b/source/troubleshooting and guides/afd-waf.html.md.erb similarity index 100% rename from source/cloud-native-platform/guides/afd-waf.html.md.erb rename to source/troubleshooting and guides/afd-waf.html.md.erb diff --git a/source/cloud-native-platform/asking-for-help/index.html.md.erb b/source/troubleshooting and guides/asking-for-help/index.html.md.erb similarity index 100% rename from source/cloud-native-platform/asking-for-help/index.html.md.erb rename to source/troubleshooting and guides/asking-for-help/index.html.md.erb diff --git a/source/cloud-native-platform/guides/automated-dependency-updates.html.md.erb b/source/troubleshooting and guides/automated-dependency-updates.html.md.erb similarity index 100% rename from source/cloud-native-platform/guides/automated-dependency-updates.html.md.erb rename to source/troubleshooting and guides/automated-dependency-updates.html.md.erb diff --git a/source/cloud-native-platform/guides/connect-via-vpn.html.md.erb b/source/troubleshooting and guides/connect-via-vpn.html.md.erb similarity index 98% rename from source/cloud-native-platform/guides/connect-via-vpn.html.md.erb rename to source/troubleshooting and guides/connect-via-vpn.html.md.erb index 53c7a8a2..a408e43f 100644 --- a/source/cloud-native-platform/guides/connect-via-vpn.html.md.erb +++ b/source/troubleshooting and guides/connect-via-vpn.html.md.erb @@ -21,7 +21,7 @@ The destination Virtual Network details are: - vnet resource group: `rg-mgmt` - net subscription: `Reform-CFT-Mgmt` -Once this is setup and deployed, you can then make a request to the Platform Operations team in [\#platops-help](https://hmcts-reform.slack.com/app_redirect?channel=platops-help). Raise a new request using the 'Help request with plato' shortcut, which will automatically raise a ticket for you and all messages will be synced to Jira. +Once this is setup and deployed, you can then make a request to the Platform Operations team in [\#platops-help](https://hmcts-reform.slack.com/app_redirect?channel=platops-help). Raise a new request using the 'Help request with plato' shortcut, which will automatically raise a ticket for you and all messages will be synced to JIRA. Raise a request to add the Address Space for your Vnet or Virtual Machine with what ports to whitelist, please include: diff --git a/source/cloud-native-platform/guides/creating-a-new-subscription.html.md.erb b/source/troubleshooting and guides/creating-a-new-subscription.html.md.erb similarity index 100% rename from source/cloud-native-platform/guides/creating-a-new-subscription.html.md.erb rename to source/troubleshooting and guides/creating-a-new-subscription.html.md.erb diff --git a/source/troubleshooting and guides/flyway-database-migrations.html.md.erb b/source/troubleshooting and guides/flyway-database-migrations.html.md.erb new file mode 100644 index 00000000..f2652f50 --- /dev/null +++ b/source/troubleshooting and guides/flyway-database-migrations.html.md.erb @@ -0,0 +1,140 @@ +--- +title: Flyway database migration +last_reviewed_on: 2023-02-20 +review_in: 6 months +weight: 10 +hide_in_navigation: true +--- + +# <%= current_page.data.title %> + +To prevent application startup failure for larger migrations due to Kubernetes probes it's essential that databases are migrated outside of Kubernetes via the Continuous Delivery pipeline. +This page attempts to show the steps required to perform a safe database migration. + +For a more detailed explanation on why this is necessary see: [Making a database schema change](../standards/db-schema-change.html) + + +Example [pull request](https://github.com/hmcts/am-role-assignment-service/pull/1149) + +Alternatively see below for a step-by-step guide. + +### Config + +Enable database migration for Jenkins: + +- Add `enableDbMigration(product)` to `Jenkinsfile_CNP` file + +Add config to `build.gradle`: + +- Ensure the [latest flyway version](https://plugins.gradle.org/plugin/org.flywaydb.flyway) is set e.g. `id 'org.flywaydb.flyway' version ''` +- Add this block: + +```groovy +flyway { + url = System.getenv('FLYWAY_URL') + user = System.getenv('FLYWAY_USER') + password = System.getenv('FLYWAY_PASSWORD') + baselineOnMigrate = true + baselineVersion = '000' +} + +task migratePostgresDatabase(type: org.flywaydb.gradle.task.FlywayMigrateTask) { + baselineOnMigrate = true + if (project.hasProperty("dburl")) { + url = "jdbc:postgresql://${dburl}" + } +} +``` + +Ensure the `RUN_DB_MIGRATION_ON_STARTUP` is configured appropriately: + +- Add `RUN_DB_MIGRATION_ON_STARTUP: true` to the `values.preview.template.yaml` file - This is so that preview will migrate on startup +- Add `RUN_DB_MIGRATION_ON_STARTUP: false` to the `values.yaml` file + +Set the migration to run on app startup + +- In `application.yaml` set: + +```yaml +dbMigration: + # When true, the app will run DB migration on startup. + # Otherwise, it will just check if all migrations have been applied (and fail to start if not). + runOnStartup: ${RUN_DB_MIGRATION_ON_STARTUP:true} +``` +### Java files + + +Make sure you update the package name to match your application's in the below examples + +- Add a class to configure flyway: + +```java +package uk.gov.hmcts.reform.example; + +import org.flywaydb.core.Flyway; +import org.springframework.boot.autoconfigure.AutoConfigureAfter; +import org.springframework.boot.autoconfigure.AutoConfigureBefore; +import org.springframework.boot.autoconfigure.condition.ConditionalOnClass; +import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; +import org.springframework.boot.autoconfigure.flyway.FlywayAutoConfiguration; +import org.springframework.boot.autoconfigure.flyway.FlywayMigrationStrategy; +import org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration; +import org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaAutoConfiguration; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import uk.gov.hmcts.reform.example.data.migration.FlywayNoOpStrategy; + +@AutoConfigureAfter({ + DataSourceAutoConfiguration.class, + HibernateJpaAutoConfiguration.class +}) +@AutoConfigureBefore(FlywayAutoConfiguration.class) +@Configuration +@ConditionalOnClass(Flyway.class) +@ConditionalOnProperty(prefix = "dbMigration", name = "runOnStartup", havingValue = "false") +public class FlywayConfiguration { + + @Bean + public FlywayMigrationStrategy flywayMigrationStrategy() { + return new FlywayNoOpStrategy(); + } +} +``` + +- And then this class to do the flyway migration: + +```java +package uk.gov.hmcts.reform.example.data.migration; + +import org.flywaydb.core.Flyway; +import org.springframework.boot.autoconfigure.flyway.FlywayMigrationStrategy; +import uk.gov.hmcts.reform.example.controller.exception.PendingMigrationScriptException; + +import java.util.stream.Stream; + +public class FlywayNoOpStrategy implements FlywayMigrationStrategy { + + @Override + public void migrate(Flyway flyway) { + Stream.of(flyway.info().all()) + .filter(info -> !info.getState().isApplied()) + .findFirst() + .ifPresent(info -> { + throw new PendingMigrationScriptException(info.getScript()); + }); + } +} + ``` + + - Migration exception class: + + ```java +package uk.gov.hmcts.reform.example.data.exception; + +public class PendingMigrationScriptException extends RuntimeException { + + public PendingMigrationScriptException(String script) { + super("Found migration not yet applied: " + script); + } +} + ``` diff --git a/source/troubleshooting and guides/index.html.md.erb b/source/troubleshooting and guides/index.html.md.erb new file mode 100644 index 00000000..c0ef24e4 --- /dev/null +++ b/source/troubleshooting and guides/index.html.md.erb @@ -0,0 +1,14 @@ +--- +title: Troubleshooting and Guides +last_reviewed_on: 2022-11-03 +review_in: 12 months +weight: 5 +--- + +# <%= current_page.data.title %> + +Guides and how-to documents: + +<% current_page.children.each do |page| %> +- [<%= page.data.title %>](<%= page.url %>) +<% end %> diff --git a/source/cloud-native-platform/guides/managing-manual-key-vault-secrets.html.md.erb b/source/troubleshooting and guides/managing-manual-key-vault-secrets.html.md.erb similarity index 100% rename from source/cloud-native-platform/guides/managing-manual-key-vault-secrets.html.md.erb rename to source/troubleshooting and guides/managing-manual-key-vault-secrets.html.md.erb diff --git a/source/cloud-native-platform/guides/postgresql-singleserver-to-flexibleserver-migration-dms.html.md.erb b/source/troubleshooting and guides/postgresql-singleserver-to-flexibleserver-migration-dms.html.md.erb similarity index 100% rename from source/cloud-native-platform/guides/postgresql-singleserver-to-flexibleserver-migration-dms.html.md.erb rename to source/troubleshooting and guides/postgresql-singleserver-to-flexibleserver-migration-dms.html.md.erb diff --git a/source/cloud-native-platform/guides/postgresql-singleserver-to-flexibleserver-migration-portal.html.md.erb b/source/troubleshooting and guides/postgresql-singleserver-to-flexibleserver-migration-portal.html.md.erb similarity index 100% rename from source/cloud-native-platform/guides/postgresql-singleserver-to-flexibleserver-migration-portal.html.md.erb rename to source/troubleshooting and guides/postgresql-singleserver-to-flexibleserver-migration-portal.html.md.erb diff --git a/source/cloud-native-platform/guides/pr-url.html.md.erb b/source/troubleshooting and guides/pr-url.html.md.erb similarity index 100% rename from source/cloud-native-platform/guides/pr-url.html.md.erb rename to source/troubleshooting and guides/pr-url.html.md.erb diff --git a/source/cloud-native-platform/guides/profile-java-app-in-aks.html.md.erb b/source/troubleshooting and guides/profile-java-app-in-aks.html.md.erb similarity index 100% rename from source/cloud-native-platform/guides/profile-java-app-in-aks.html.md.erb rename to source/troubleshooting and guides/profile-java-app-in-aks.html.md.erb diff --git a/source/cloud-native-platform/troubleshooting/_clusters.md b/source/troubleshooting and guides/troubleshooting/_clusters.md similarity index 100% rename from source/cloud-native-platform/troubleshooting/_clusters.md rename to source/troubleshooting and guides/troubleshooting/_clusters.md diff --git a/source/troubleshooting and guides/troubleshooting/aks-apps.html.md.erb b/source/troubleshooting and guides/troubleshooting/aks-apps.html.md.erb new file mode 100644 index 00000000..b62ccc85 --- /dev/null +++ b/source/troubleshooting and guides/troubleshooting/aks-apps.html.md.erb @@ -0,0 +1,50 @@ +--- +title: Debug Application Startup issues in AKS +last_reviewed_on: 2023-03-08 +review_in: 6 months +weight: 99 +--- +# <%= current_page.data.title %> +- There could be many reasons why applications could fail to startup like : + - A secret referred in helm chart is missing in keyvaults + - Pod identity is not able to pull keyvault secrets due to missing permissions + - There is not enough space in the cluster to fit in a new pod. + - Pod is scheduled, but fails to pass readiness (`/health/readiness`) or liveness (`/health/liveness`) checks. + - A misconfigured environment variable, example - incorrect URL of a dependent service. + +- Below are some handy kubectl commands to debug the issues + + To check latest events on your namespace: + + ```shell + kubectl get events -n + ``` + + To check status of pods: + + ```shell + kubectl get pods -n | grep + + #Examples + # kubectl get pods -n ccd | grep ccd-data-store-api + # kubectl get pods -n ccd | grep pr-123 + ``` + + To check status of a specific pod which is not running + + ```shell + kubectl describe pod -n + ``` + + To check logs of pods which is not starting + + ```shell + kubectl logs -n + + #To follow logs + kubectl logs -n -f + + # To check previous pod logs if its restarting + kubectl logs -n -p + + ``` diff --git a/source/troubleshooting and guides/troubleshooting/connecting-to-aks.html.md.erb b/source/troubleshooting and guides/troubleshooting/connecting-to-aks.html.md.erb new file mode 100644 index 00000000..6156bc9d --- /dev/null +++ b/source/troubleshooting and guides/troubleshooting/connecting-to-aks.html.md.erb @@ -0,0 +1,18 @@ +--- +title: Connecting to AKS Clusters +last_reviewed_on: 2023-03-08 +review_in: 6 months +weight: 99 +--- +# <%= current_page.data.title %> +- By Default, all developers have read access to non-prod AKS clusters and slightly higher privileges to their namespaces. +- You can connect to AKS clusters using `az aks get-credentials`. Below are some handy commands: + +<%= partial 'clusters' %> + +Once you have logged in, you can switch between clusters using [kubectx](https://github.com/ahmetb/kubectx) or below kubectl commands: + +```shell +kubectl config use-context cft-perftest-00-aks +kubectl config use-context cft-aat-00-aks +``` diff --git a/source/troubleshooting and guides/troubleshooting/github.html.md.erb b/source/troubleshooting and guides/troubleshooting/github.html.md.erb new file mode 100644 index 00000000..6daee7a5 --- /dev/null +++ b/source/troubleshooting and guides/troubleshooting/github.html.md.erb @@ -0,0 +1,27 @@ +--- +title: GitHub +last_reviewed_on: 2023-03-08 +review_in: 6 months +weight: 99 +--- +# <%= current_page.data.title %> +The GitHub troubleshooting topic contains the sections: + +* [Adding a new user on GitHub](#adding-a-new-user-on-github) +* [Creating a new GitHub repository](#creating-a-new-github-repository) +* [Access to GitHub Repo](#access-to-github-repo) + +## Adding a new user on GitHub + + - See [GitHub Onboarding](/cloud-native-platform/onboarding/team/github.html#access) for how to get access + - Make sure the user has accepted the invite received on the email setup in Github or by visiting [github.com/hmcts](https://github.com/hmcts). + +## Creating a new GitHub repository + +See [create a GitHub repository](../new-component/github-repo.html#) + +## Access to GitHub Repo + + - All repositories should be administered by the team who owns it. + - Access should be managed through GitHub teams and not individuals. + - If no one from your team have access, follow [asking for help](/cloud-native-platform/asking-for-help/#asking-for-help) to request org admins in one of the community/ support channels. diff --git a/source/troubleshooting and guides/troubleshooting/gitops.html.md.erb b/source/troubleshooting and guides/troubleshooting/gitops.html.md.erb new file mode 100644 index 00000000..3e5b37f2 --- /dev/null +++ b/source/troubleshooting and guides/troubleshooting/gitops.html.md.erb @@ -0,0 +1,112 @@ +--- +title: Flux / GitOps +last_reviewed_on: 2023-03-08 +review_in: 6 months +weight: 99 +--- +# <%= current_page.data.title %> +> Always check __why__ your release or pod has failed in the first instance. +> Although you may have permissions to delete a helm release or pod in a non-production environment, use this privilege wisely as you could be _hiding a potential bug_ which could also _occur in production_. + +The Flux / GitOps troubleshooting topic contains the sections: + +* [Latest image is not updated in cluster](#latest-image-is-not-updated-in-cluster) +* [Flux did not commit latest image to Github](#flux-did-not-commit-latest-image-to-github) +* [Updated HelmRelease is not deployed to cluster](#updated-helmrelease-is-not-deployed-to-cluster) +* [Change in git is not applied to cluster](#change-in-git-is-not-applied-to-cluster) + +## Latest image is not updated in cluster + +- Start with checking [cnp-flux-config](https://github.com/hmcts/cnp-flux-config) to make sure flux has updated/ committed the image. +- If image hasn't been committed to Github, see [ Flux did not commit latest image to Github](#flux-did-not-commit-latest-image-to-github). +- If flux has committed the new image to Github, check if the `HelmRelease` has been updated by Flux. Run below command and check that the image tag has been updated in the output + + ```shell + kubectl get hr -n -o yaml + ``` +- If Image is not updated in above, [Change in git is not applied to cluster](#change-in-git-is-not-applied-to-cluster). +- If the image tag is updated and still application pods are not deployed, see [Updated HelmRelease is not deployed to cluster](#updated-helmrelease-is-not-deployed-to-cluster) + +## Flux did not commit latest image to Github + +- Image automation is run from management cluster (CFTPTL). Please login to cftptl cluster before further troubleshooting. +- Image reflector controller keeps polling ACR for new images, but it should generally update the new image in 10 minutes. +- Check status of `imagerepositories` and verify the last scan. + + ```shell + kubectl get imagerepositories -n flux-system + ``` +- If the last scan doesn't update, check image reflector controller logs to see if there any logs related to the helm repo. + + ```shell + kubectl logs -n flux-system -l app=image-reflector-controller --tail=-1 + # search for specific image + kubectl logs -n flux-system -l app=image-reflector-controller --tail=-1 | grep + ``` +- If the last scan is latest, check `imagepolicy` status to verify that the image returned matches the expectation. + + ```shell + kubectl get imagepolicies -n flux-system + ``` +- If it doesn't match the expected tag, verify image reflector controller logs as described above. +- If the `imagepolicy` object returned shows the expected image, but it didn't commit to Github, check image automation controller logs. + + ```shell + kubectl logs -n flux-system -l app=image-automation-controller + # search for specific image + kubectl logs -n flux-system -l app=image-automation-controller | grep + ``` + +## Updated HelmRelease is not deployed to cluster + +- Helm operator queues all the updates, so it could take up to 20 minutes sometimes to be picked up. +- Check HelmRelease status to see the status. + + ```shell + kubectl get hr -n + ``` +- Look at helm operator logs to see if there are any errors specific to your helm release + + ```shell + kubectl logs -n flux-system -l app=helm-controller --tail=1000 | grep + ``` +- If you see any errors like, `status 'pending-install' of release does not allow a safe upgrade"`. You need to delete `HelmRelease` for fixing this, request platops for help if you do not have permissions. + + ```shell + kubectl delete hr -n + ``` +- In most cases, helm release gets timed out with an error in log similar to ` failed: timed out waiting for the condition`. This usually means application pods didn't startup in time and you need to look at your pods to know more. + + Check the latest status on helm release and if it has already been rolled back to previous release. + + ```shell + kubectl describe hr -n + ``` +- If you are looking at pods after a long time, `HelmRelease` might have been rolled back and you won't have failed pods. Easiest way is to add a simple change like a dummy environment variable in flux-config to re-trigger the release and debug the issue when it occurs. + +- If your old pods are still running when you check, follow [Debug Application Startup issues in AKS](#debug-application-startup-issues-in-aks) to troubleshoot further. + +## Change in git is not applied to cluster + +- To check if latest github commit has been downloaded by checking status + + ```shell + kubectl get gitrepositories flux-config -n flux-system + ``` +- If the commit doesn't match latest id, verify source controller logs to see any related errors + + ```shell + kubectl logs -n flux-system -l app=source-controller + ``` +- If commit id is recent, verify status of flux kustomization for your namespace to get the version of git applied. + + ```shell + kubectl get kustomizations.kustomize.toolkit.fluxcd.io -n flux-system + ``` +- If the above status doesn't show latest commit/ show any error , see kustomize controller logs to find relevant errors. + +```shell +kubectl logs -n flux-system -l app=kustomize-controller +# search for specific image +kubectl logs -n flux-system -l app=kustomize-controller | grep +``` diff --git a/source/troubleshooting and guides/troubleshooting/inactive-accounts.html.md.erb b/source/troubleshooting and guides/troubleshooting/inactive-accounts.html.md.erb new file mode 100644 index 00000000..7a99e340 --- /dev/null +++ b/source/troubleshooting and guides/troubleshooting/inactive-accounts.html.md.erb @@ -0,0 +1,32 @@ +--- +title: Inactive Accounts +last_reviewed_on: 2023-03-08 +review_in: 6 months +weight: 99 +--- +# <%= current_page.data.title %> +## Receiving an email regarding inactive account + +You may have received an email from our scheduled pipeline which deletes guest user accounts if they're inactive for longer than 31 days. + +### Account close to being deleted + +Users who are within a week of being deleted get notified via email on the lead up to the deletion date, starting 7 days before. + +If you have received an email about your account being close to being deleted, because of inactivity, that is because you haven't logged into the `CJS Common Platform` Azure Tenant for a while and need to do so now to keep your access. + +Please don't ignore this email if you use the `CJS Common Platform` Tenant or GitHub as you will lose access to both if you don't log in. + +To extend your access, visit the [CJS Common Platform Azure portal](https://portal.azure.com/531ff96d-0ae9-462a-8d2d-bec7c0b42082). + +Login link: [CJS Common Platform]( https://portal.azure.com/531ff96d-0ae9-462a-8d2d-bec7c0b42082) + +*Note:* We're looking into fixing this soon so that as long as you're active in either GitHub or the `CJS Common Platform` Tenant you will not lose access. + +### Account has been deleted + +If your account has been deleted and you need it re-enabled, you need to create a ticket via slack in the [#platops-help](https://hmcts-reform.slack.com/app_redirect?channel=platops-help) channel and someone will send you another guest invite. + +Once that is done you will then be able to get your GitHub access back by asking one of your team members to create a PR in the [github-management](https://github.com/hmcts/github-management) repository, adding your user back in. + +[Example PR - adding a user](https://github.com/hmcts/github-management/pull/1079/files) (requires access to the HMCTS GitHub organisation) diff --git a/source/troubleshooting and guides/troubleshooting/index.html.md.erb b/source/troubleshooting and guides/troubleshooting/index.html.md.erb new file mode 100644 index 00000000..c801330b --- /dev/null +++ b/source/troubleshooting and guides/troubleshooting/index.html.md.erb @@ -0,0 +1,12 @@ +--- +title: Troubleshooting issues +last_reviewed_on: 2022-08-05 +review_in: 6 months +weight: 98 +--- +# <%= current_page.data.title %> +Find common troubleshooting topics below: + +<% current_page.children.each do |page| %> +- [<%= page.data.title %>](<%= page.url %>) +<% end %> diff --git a/source/troubleshooting and guides/troubleshooting/jenkins.html.md.erb b/source/troubleshooting and guides/troubleshooting/jenkins.html.md.erb new file mode 100644 index 00000000..c9621ced --- /dev/null +++ b/source/troubleshooting and guides/troubleshooting/jenkins.html.md.erb @@ -0,0 +1,127 @@ +--- +title: Jenkins +last_reviewed_on: 2023-03-08 +review_in: 6 months +weight: 99 +--- +# <%= current_page.data.title %> + +The Jenkins troubleshooting topic contains the sections: + +* [Jenkins is unavailable](#jenkins-is-unavailable) +* [Cannot login to Jenkins](#cannot-login-to-jenkins) +* [You are now logged out of Jenkins/ Infinite loop when trying to login to Jenkins](#you-are-now-logged-out-of-jenkins-infinite-loop-when-trying-to-login-to-jenkins) +* [Cannot see my new repo in Jenkins org / Dashboard](#cannot-see-my-new-repo-in-jenkins-org-dashboard) +* [Cannot find default branch](#cannot-find-default-branch) +* [Cannot see my branch/PR or This project is currently disabled in Jenkins](#cannot-see-my-branch-pr-or-this-project-is-currently-disabled-in-jenkins) +* [Build / Docker Build / Unit Test failure](#build-docker-build-unit-test-failure) +* [Sonar scan timeout](#sonar-scan-timeout) +* [Helm Upgrade Failed, Helm Release timed out waiting for condition, Helm Release Failure](#helm-upgrade-failed-helm-release-timed-out-waiting-for-condition-helm-release-failure) +* [Jenkins managed helm releases / pods are automatically deleted](#jenkins-managed-helm-releases-pods-are-automatically-deleted) +* [Smoke / Functional test failure](#smoke-functional-test-failure) +* [Terraform/ Build Infrastructure failure](#terraform-build-infrastructure-failure) +* [Using branches to troubleshoot issues](#using-branches-to-troubleshoot-issues) +* [Error message channel_not_found](#error-message-channel-not-found) + +## Jenkins is unavailable + +- Check if there is a planned outage in [#cloud-native-announce](https://hmcts-reform.slack.com/archives/CA4F2MAFR). +- Please note Jenkins could be temporarily unavailable while rolling out a change, give it a few minutes before you raise an issue. + +## Cannot login to Jenkins + +- Login to Jenkins is managed by Azure AD, please make sure user is added to right [Azure AD groups](/cloud-native-platform/onboarding/person/#azure-ad-groups) + +## You are now logged out of Jenkins/ Infinite loop when trying to login to Jenkins + +- Please try clearing browser cookies on [login.microsoft.com](https://login.microsoftonline.com/) + +## Cannot see my new repo in Jenkins org / Dashboard + +- See [Jenkins Onboarding section](/cloud-native-platform/onboarding/team/jenkins.html#jenkins) to add your app to Jenkins Org / Dashboard + +## Cannot find default branch + +You may get the error below, when the pipeline has not ran on the master/main branch first. + +When you run it on the master/main branch it will setup the default branch and then the pull request build will start working. + +``` +[2021-09-16T10:59:54.154Z] Execution failed for task ':sonarqube'. +[2021-09-16T10:59:54.154Z] > Could not find a default branch to fall back on. +``` +## Cannot see my branch/PR or This project is currently disabled in Jenkins + +- Any branches which are also filed as PRs are not listed as a branch, they will only be listed in pull requests section. +- Branch/ PR is not listed if its last commit creation date is older than 30 days. + +## Build / Docker Build / Unit Test failure + +- If your build is failing in these stages, it's most likely to fail in your local as well. Look at the first line of the Jenkins step that fails and try run the same command Jenkins is running. +- Try on a colleague's machine as you might have cached something locally. + +## Sonar scan timeout + +- Please see [sonarcloud status](https://sonarcloud.statuspage.io/) for any known issues with sonar cloud. +- Remember that Platform Operation do not maintain SonarCloud, issues are usually discussed on community forums. + +## Helm Upgrade Failed, Helm Release timed out waiting for condition, Helm Release Failure + +- See [Connecting to AKS Clusters](#connecting-to-aks-clusters) and connect to the relevant cluster. +- These errors usually mean your pods didn't start as expected in time. +- It could be that they are stuck in `Pending`, `ContainerCreating` status or might be failing to startup leading to `CrashLoopBackOff` status. +- Follow [Debug Application Startup issues in AKS](#debug-application-startup-issues-in-aks) to troubleshoot further + +## Jenkins managed helm releases / pods are automatically deleted + +- To maintain the health of the cluster, it is important to cleanup unwanted pods regularly. +- Helm release is cleared for PRs which are merged or closed. +- Helm release of PRs raised by dependency bots (based on `dependencies` label) is cleared once the functional tests pass. +- Helm release on AAT Staging is also cleared once functional tests pass. +- For optimal usage, You can also configure your pipeline to [clear Helm releases on successful build](https://github.com/hmcts/cnp-jenkins-library#clear-helm-release-on-successful-build). +- A scheduled pipeline runs every hour to clear any helm releases which are not updated in last 3 days. Teams can do more frequent cleanup by overriding in the [cleanup script](https://github.com/hmcts/cnp-aks-pipelines/blob/0fe733120f78b6dabcdd5895bb16134085631842/scripts/delete-inactive-helm-releases.sh#L5) + +## Smoke / Functional test failure + +- Jenkins only sets secrets as environment variables and runs the `gradle` / `yarn` task to run tests. +- Access the URL on VPN and try running tests manually using the `TEST_URL` printed in the logs. +- You can also run tests locally by setting the required secrets while on the VPN. +- To add additional logging, see [Example config](#example-2). + +## Terraform/ Build Infrastructure failure + +- It is important that you need to review your plan in a pull request before applying it to an environment. +- Please check if its an intermittent failure with Azure as a retry could fix it. +- Also, see if there are any open issues/discussions on community channels for the failure. +- There could be open GitHub issues on terraform/ azurerm, so googling it could help as well. + +## Using branches to troubleshoot issues + +If your pipeline is throwing an error, you may be able to more easily troubleshoot the issue by using a branch in the cnp-jenkins-library repo. + +### Example 1 +Gradle is failing because a plugin cannot be found in artifactory. You've checked the plugin exists and there are no typos in your code. +You can check if the issue lies with artifactory by temporarily bypassing it using a [branch](https://github.com/hmcts/cnp-jenkins-library/blob/073ad8587b7281d62bac705ed984e739a0911c83/resources/uk/gov/hmcts/gradle/init.gradle#L3). + +And then referencing that branch within your repo's [Jenkinsfile](https://github.com/hmcts/sds-toffee-recipes-service/pull/12/files) + +In this example, the issue is network related. This may be down to routing in Azure or traffic being blocked by a firewall. + +### Example 2 +Gradle is failing a functional test. To help get more information on why, you could update the Gradle logging level in a branch of [cnp-jenkins-library](https://github.com/hmcts/cnp-jenkins-library/blob/cfb31f3a2699b2a1dafd66fed0b525ae145d627d/src/uk/gov/hmcts/contino/GradleBuilder.groovy#L62) + +And then reference this branch within your repo's [Jenkinsfile](https://github.com/hmcts/document-management-store-app/blob/646593336377fd59112b0b6c84fd223d0cb7832c/Jenkinsfile_CNP#L12) + +## Error message `channel_not_found` + +When you want to send Jenkins pipeline notifications to a slack channel, you must make sure the Jenkins app has been added to the channel. + +If you don't do this, you will receive the `channel_not_found` error message and the pipeline will fail. + +See the instructions on the [slack](/cloud-native-platform/onboarding/team/slack.html) section of the onboarding guide to find out how to do this. + +If you are getting this message on a PR pipeline and the error also says `Failed to notify @U1234ABCD`, that means your github username has been mapped to an invalid slack id. + +Find your slack id by clicking on `View profile` within the slack app, then click on the three dots and click `Copy member ID`. + +Update your github to slack user mapping in [this file](https://github.com/hmcts/github-slack-user-mappings/blob/master/slack.json) and try running the pipeline again.