Skip to content

Commit

Permalink
merging main
Browse files Browse the repository at this point in the history
  • Loading branch information
mboudreau committed Dec 9, 2021
2 parents b26d52c + 6a7daa7 commit 747f037
Show file tree
Hide file tree
Showing 93 changed files with 2,851 additions and 1,688 deletions.
6 changes: 3 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ Once you've set up your GitHub app and cloned this repo, copy the content from `
+ `PRIVATE_KEY_PATH`: You'll also need to generate a new private key on your GitHub app page, download it, move it to the source root of this repo, and set `PRIVATE_KEY_PATH=<your-private-key-name>.pem`
+ `ATLASSIAN_URL`: The URL for the Jira instance you're testing it. If you don't have one now, please set the value of this variable after going through the step 1 of "Configuring the Jira instance" section of this document.
+ `STORAGE_SECRET`: It needs to be set to a 32 char secret (anything else fails). You can generate one by running `openssl rand -hex 32` in your terminal and paste directly to your .env file.
+ `INSTANCE_NAME`: Your Jira app name - will show as "Github (instance-name)"
+ `INSTANCE_NAME`: Your Jira app name - will show as "GitHub (instance-name)"
+ `WEBHOOK_PROXY_URL`: `https://DOMAIN/github/events`

### Running the app
Expand Down Expand Up @@ -102,7 +102,7 @@ Go to your Jira instance that you created earlier and do the following steps:

### Setting up the App

In your Jira instance, in the `Manage Apps` section, click on your App's button, then click on `Get Started`. This will bring you to the App's dashboard. Click the `Add an Organization` button and follow the steps to install the App on Github and allow it permission to view your repos.
In your Jira instance, in the `Manage Apps` section, click on your App's button, then click on `Get Started`. This will bring you to the App's dashboard. Click the `Add an Organization` button and follow the steps to install the App on GitHub and allow it permission to view your repos.

After this is done, you should see your repos starting to sync in the App's dashboard.

Expand Down Expand Up @@ -130,4 +130,4 @@ That being said, here are the steps needed to create a Pull Request for us to re
1. Commit and Push your changes - verify it passes all checks.
1. Submit your pull request with a detailed message about what's changed.
1. Wait for us to review and answer questions/make changes where requested.
1. Once merged, celebrate with your drink of choice because you've just helped thousands (if not millions) of people get a better experience in both Jira and Github! :beers:
1. Once merged, celebrate with your drink of choice because you've just helped thousands (if not millions) of people get a better experience in both Jira and GitHub! :beers:
11 changes: 10 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ For general support inquiries, [please contact the Atlassian Support team](https
- [See GitHub CI/CD data in Jira](#see-github-cicd-data-in-jira)
- [How the integration works](#how-the-integration-works)
- [Migrate from the DVCS connector](#migrate-from-the-dvcs-connector)
- [Enterprise Features](#enterprise-features)
- [IP Allow List](#ip-allow-list)
- [Need help?](#need-help)
- [Contribute](#contribute)
- [License](#license)
Expand Down Expand Up @@ -134,7 +136,7 @@ If an issue body contains a valid Jira issue key on your instance, the integrati
This makes it so Jira issues can be linked inside a comment without it interrupting the flow of the comment as a whole.

### See GitHub CI/CD data in Jira
GitHub Actions is a feature from GitHub for automation such as CI/CD. If you’re setting this up for the first time, follow [GitHub Actions Documentation - GitHub Docs](https://docs.github.com/en/actions). If you already have GitHub Actions and want to see CI/CD data from Github in Jira, include the Jira issue key in your commit message, branch name, or PR.
GitHub Actions is a feature from GitHub for automation such as CI/CD. If you’re setting this up for the first time, follow [GitHub Actions Documentation - GitHub Docs](https://docs.github.com/en/actions). If you already have GitHub Actions and want to see CI/CD data from GitHub in Jira, include the Jira issue key in your commit message, branch name, or PR.

### How the integration works
When a workflow (e.g. GitHub Action) or development event (e.g. pull request, commit, branch) runs, the app receives a webhook from GitHub. The app then extract the issue key from the respective branch/commit/PR and send this information to Jira.
Expand All @@ -146,6 +148,13 @@ Existing users of Jira's built-in DVCS connector that meet the [requirements](#r
2. From the left sidebar in Jira, select **Jira Settings > Applications > DVCS accounts**.
3. Follow the prompt to upgrade your GitHub connection.

## Enterprise Features

### IP Allow List

GitHub has the ability to limit who can communicate with your organization's GitHub API which we now fully support.
To enable this feature or to debug any issues, please refer to our [GitHub IP Allow List documentation](./docs/ip-allowlist.md).

## Need help?
Take a look through the troubleshooting steps in our [support guide](./SUPPORT.md).

Expand Down
7 changes: 6 additions & 1 deletion db/config.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
{
"development": {
"dialect": "postgres",
"use_env_variable": "DATABASE_URL"
"use_env_variable": "DATABASE_URL",
"pool": {
"max": 15,
"min": 0,
"idle": 1000
}
},
"test": {
"dialect": "postgres",
Expand Down
76 changes: 76 additions & 0 deletions docs/builds.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# GitHub Actions - Builds

GitHub for Jira supports builds via [GitHub Actions workflow syntax](https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions).
To set this up, you will need to create a .github folder at the root of a given repository and then make a child directory
called workflows. Inside of .github/workflows you will need to create a build.yml file. This is where you will specify the workflow for your builds.

Following is an example of a build.yml file:

```
# This is a basic workflow to help you get started with Actions
name: CI
# Controls when the action will run.
on:
# Triggers the workflow on push or pull request events but only for the main branch
push:
branches:
- main
pull_request:
branches:
- main
- feature/**
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
# Runs a single command using the runners shell
- name: Run a one-line script
run: echo Hello, world!
# Runs a set of commands using the runners shell
- name: Run a multi-line script
run: |
echo Add other actions to build,
echo test, and deploy your project.
sleep 60s
```

Once you have a similar file in any of your repositories that are connected to your Jira via the app, you will start to see builds data
in the development panel in Jira.

![Builds data in Jira](./images/builds-data-jira-dev-panel.png)

#### Supporting Multiple Commits with Issue Keys

One important thing to note in the above example is the branches being targeted under `pull_requests` `branches`:

```
pull_request:
branches:
- main
- feature/**
```

When you open a pull request in a connected repository, the GitHub for Jira app can compare two points in your git history is we have access to a base branch and a head branch. If the app has both of these, it can make a request to GitHub to compare your
commits. On the flipside, if you only listed `main`, for instance, there would be no way for the app to ask GitHub for a comparison.
Instead, the best it can do is use the data GitHub sends in the response when the `workflow_run` event is triggered,
which only includes the most recent commit. This means that if a developer were to make multiple commits, perhaps on multiple branches, and
reference various Jira issue keys in each commit, GitHub would only send the data to Jira about the latest commit. In turn, this
would mean that we could only extract any issue keys from that single message. Although there may be numerous Jira issues
involved, in this scenario, you would only see builds data for any issue keys from the latest commit message.

When you list branches under `pull_request` you'll need to be very specific about the branches you want the app to target. If any branch is created that isn't listed here, the app won't be able to compare your commits and send the most accurate data to Jira.
Binary file added docs/images/builds-data-jira-dev-panel.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/github-ip-allowlist.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
65 changes: 65 additions & 0 deletions docs/ip-allowlist.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# GitHub IP Allow List Configuration

If your organization is using [GitHub's Organization IP Allow List](https://docs.github.
com/en/organizations/keeping-your-organization-secure/managing-allowed-ip-addresses-for-your-organization), it needs to
be configured properly for this GitHub app to be able to communicate with your organization's GitHub API.

There are 2 methods to remedy this based on your company's security policy:

1. a simple method which will enable _all_ GitHub apps with IPs specified in them to have access to your GitHub org's
APIs (_recommended_)
2. a manual way by adding each CIDR ranges possible for this app to communicate from.

**We recommend using the first method** as you only have to set it once and never have to think about it again. If there
are any IP changes on our end in the future, we can just change the list on our end and it will automatically propagate
to your organization. But for this to happen, you must trust every GitHub app installed or else risk a potential
security breach by an app adding an attacker's IP to your allow list.

If you'd like to have complete control over your IP Allow List, then you can enter the CIDR ranges manually in your
GitHub organization. But it does come with the drawback that if the CIDR ranges ever change or a new one needs to be
added, you will have to manually update those as well. Furthermore, we don't have a way to easily send a message to all
GitHub org admins about a change like this and it could be possible that the integration might break because of the
change.

### Simple Method

As an admin go to your GitHub org page `https://github.com/<your org>`, press on the `Settings` tab, then in the sidebar
select the `Organization security` option. Scroll down to the `IP allow list` section. Both
checkboxes `Enable IP allow list` and `Enable IP allow list configuration for installed GitHub Apps` should be selected
and saved independently.

![](images/github-ip-allowlist.png)

That's it!

### Manual Method

As an admin your GitHub org page `https://github.com/<your org>`, press on the `Settings` tab, then in the sidebar
select the `Organization security` option. Scroll down to the `IP allow list` section until you can see the list of IP
addresses with a `+ Add` button. From here, you need
to [add the whole list of CIDR ranges specified in this Atlassian document](https://support.atlassian.com/organization-administration/docs/ip-addresses-and-domains-for-atlassian-cloud
-products/#AtlassiancloudIPrangesanddomains-OutgoingConnections).

For simplicity, here's the list of CIDR ranges, but it might not be up to date:

```
13.52.5.96/28
13.236.8.224/28
18.136.214.96/28
18.184.99.224/28
18.234.32.224/28
18.246.31.224/28
52.215.192.224/28
104.192.137.240/28
104.192.138.240/28
104.192.140.240/28
104.192.142.240/28
104.192.143.240/28
185.166.143.240/28
185.166.142.240/28
```

## If problems persist

Feel free to [contact Atlassian support](https://support.atlassian.com/contact/#/?
inquiry_category=technical_issues&is_cloud=true&product_key=third-party-product) for guidance and extra help.
6 changes: 3 additions & 3 deletions docs/legacy-features.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,17 @@

See
- (Atlassian docs)[https://confluence.atlassian.com/adminjiracloud/connect-jira-cloud-to-github-814188429.html]
- (Github docs)[https://help.github.com/articles/integrating-jira-with-your-organization-s-projects/]
- (GitHub docs)[https://help.github.com/articles/integrating-jira-with-your-organization-s-projects/]

## Installation

From within Jira Software, customers would need to

1. go to the DVCS Accounts page within settings
2. set up the Github connection via oAuth client ID and Code Secret
2. set up the GitHub connection via oAuth client ID and Code Secret
3. Sync all repos that were added.

Note: The same method is used for Github & Github Enterprise, but an additional Host URL is required on installation. When using Atlassian Connect to host the Github application, it will need to deal with Github + Github EE itself.
Note: The same method is used for GitHub & GitHub Enterprise, but an additional Host URL is required on installation. When using Atlassian Connect to host the Github application, it will need to deal with Github + Github EE itself.

![](https://user-images.githubusercontent.com/173/32561336-abd01cb6-c471-11e7-8719-13e165cd3dcd.png)

Expand Down
9 changes: 9 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@atlassian/github-for-jira",
"version": "0.0.0",
"description": "Github integration for Atlassian Jira",
"description": "GitHub integration for Atlassian Jira",
"repository": "https://github.com/atlassian/github-for-jira.git",
"engines": {
"node": ">= 14.16 <15",
Expand Down Expand Up @@ -73,6 +73,7 @@
"bull": "^3.12.1",
"bunyan": "^1.8.15",
"bunyan-format": "^0.2.1",
"cookie-parser": "^1.4.6",
"cookie-session": "^2.0.0-rc.1",
"csurf": "^1.11.0",
"date-fns": "^1.29.0",
Expand Down
50 changes: 50 additions & 0 deletions src/backfill-queue-supplier.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
import Queue from "bull";
import {SqsQueue} from "./sqs";
import {BackfillMessagePayload} from "./sqs/backfill";
import {BackfillQueue} from "./sync/installation";
import {LoggerWithTarget} from "probot/lib/wrap-logger";
import {getLogger} from "./config/logger";
import {booleanFlag, BooleanFlags} from "./config/feature-flags";

const fallbackLogger = getLogger("queue-supplier-default");

/**
* A temp class to support switching between Redis and SQS. Will be gone with the feature flag and the
* Redis queue.
*/
class BackfillQueueSupplier {
private redisQueue: Queue.Queue;
private sqsQueue: SqsQueue<BackfillMessagePayload>;

setRedisQueue(redisQueue: Queue.Queue) {
this.redisQueue = redisQueue;
}

setSQSQueue(sqsQueue: SqsQueue<BackfillMessagePayload>) {
this.sqsQueue = sqsQueue;
}

async supply(): Promise<BackfillQueue> {
if (!this.redisQueue) {
return Promise.reject(new Error("Redis queue wasn't provided"));
}
if (!this.sqsQueue) {
return Promise.reject(new Error("SQS queue wasn't provided"));
}
return {
schedule: async (payload, delayMsec?: number, log?: LoggerWithTarget) => {
if (await booleanFlag(BooleanFlags.USE_SQS_FOR_BACKFILL, false, payload.jiraHost)) {
await this.sqsQueue.sendMessage(payload, (delayMsec || 0) / 1000, (log || fallbackLogger));
} else {
if (delayMsec) {
await this.redisQueue.add(payload, {delay: delayMsec});
} else {
await this.redisQueue.add(payload);
}
}
}
};
}
}

export default new BackfillQueueSupplier();
1 change: 1 addition & 0 deletions src/config/errors.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
export enum Errors {
MISSING_JIRA_HOST = "Jira Host url is missing",
MISSING_GITHUB_TOKEN = "Github Auth token is missing",
IP_ALLOWLIST_MISCONFIGURED = "IP Allowlist Misconfigured",
}

25 changes: 19 additions & 6 deletions src/config/feature-flags.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import LaunchDarkly, { LDUser } from "launchdarkly-node-server-sdk";
import { getLogger } from "./logger";
import envVars from "./env";
import crypto from "crypto";
import {LoggerWithTarget} from "probot/lib/wrap-logger";

const logger = getLogger("feature-flags");

Expand All @@ -16,21 +17,22 @@ export enum BooleanFlags {
EXPOSE_QUEUE_METRICS = "expose-queue-metrics",
PROCESS_PUSHES_IMMEDIATELY = "process-pushes-immediately",
SIMPLER_PROCESSOR = "simpler-processor",
NEW_GITHUB_CONFIG_PAGE = "new-github-config-page",
NEW_CONNECT_AN_ORG_PAGE = "new-connect-an-org-page",
NEW_GITHUB_ERROR_PAGE = "new-git-hub-error-page",
NEW_SETUP_PAGE = "new-setup-page",
NEW_BACKFILL_PROCESS_ENABLED = "new-backfill-process-enabled",
USE_DEDUPLICATOR_FOR_BACKFILLING = "use-deduplicator-for-backfilling",
// When cleaning up the SEND_PUSH_TO_SQS feature flag, please also clean up the PRIORITIZE_PUSHES
// feature flag, because it doesn't make sense with SQS any more.
SEND_PUSH_TO_SQS = "send-push-events-to-sqs",
PRIORITIZE_PUSHES = "prioritize-pushes",
USE_NEW_GITHUB_CLIENT__FOR_PR = "git-hub-client-for-pullrequests",
NEW_REPO_SYNC_STATE = "new-repo-sync-state",
PAYLOAD_SIZE_METRIC = "payload-size-metrics",
USE_BACKFILL_QUEUE_SUPPLIER = "use-backfill-queue-supplier",
SUPPORT_BRANCH_AND_MERGE_WORKFLOWS_FOR_DEPLOYMENTS = "support-branch-and-merge-workflows-for-deployments",
TRACE_LOGGING = "trace-logging",
USE_SQS_FOR_BACKFILL = "use-sqs-for-backfill",
SUPPORT_BRANCH_AND_MERGE_WORKFLOWS_FOR_BUILDS = "support-branch-and-merge-workflows-for-builds",
USE_NEW_GITHUB_CLIENT_FOR_PUSH = "use-new-github-client-for-push",
USE_NEW_GITHUB_CLIENT_TO_COUNT_REPOS = "use-new-github-client-to-count-repos",
REPO_SYNC_STATE_AS_SOURCE = "repo-sync-state-as-source"

}

export enum StringFlags {
Expand Down Expand Up @@ -69,3 +71,14 @@ export const booleanFlag = async (flag: BooleanFlags, defaultValue: boolean, jir

export const stringFlag = async (flag: StringFlags, defaultValue: string, jiraHost?: string): Promise<string> =>
String(await getLaunchDarklyValue(flag, defaultValue, jiraHost));

export const isBlocked = async (installationId: number, logger: LoggerWithTarget): Promise<boolean> => {
try {
const blockedInstallationsString = await stringFlag(StringFlags.BLOCKED_INSTALLATIONS, "[]");
const blockedInstallations: number[] = JSON.parse(blockedInstallationsString);
return blockedInstallations.includes(installationId);
} catch (e) {
logger.error({ err: e, installationId }, "Cannot define if isBlocked")
return false;
}
};
Loading

0 comments on commit 747f037

Please sign in to comment.