Skip to content

Commit

Permalink
feat(release): publish JSII Python Artifacts to AWS CodeArtifact (#3475)
Browse files Browse the repository at this point in the history 
Makes pypi publisher job support targeting AWS CodeArtifact. Re-uses the `CodeArtifactOptions` already added to support publishing of NPM packages to CodeArtifact.

---
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
  • Loading branch information
@timothymathison
timothymathison committed Mar 25, 2024
1 parent 4024908 commit cf67e86
Show file tree
Hide file tree
Showing 5 changed files with 308 additions and 13 deletions.
14 changes: 14 additions & 0 deletions docs/api/cdk.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9302,6 +9302,7 @@ const jsiiPythonTarget: cdk.JsiiPythonTarget = { ... }
| <code><a href="#projen.cdk.JsiiPythonTarget.property.postPublishSteps">postPublishSteps</a></code> | <code>projen.github.workflows.JobStep[]</code> | Steps to execute after executing the publishing command. |
| <code><a href="#projen.cdk.JsiiPythonTarget.property.prePublishSteps">prePublishSteps</a></code> | <code>projen.github.workflows.JobStep[]</code> | Steps to execute before executing the publishing command. These can be used to prepare the artifact for publishing if neede. |
| <code><a href="#projen.cdk.JsiiPythonTarget.property.publishTools">publishTools</a></code> | <code>projen.github.workflows.Tools</code> | Additional tools to install in the publishing job. |
| <code><a href="#projen.cdk.JsiiPythonTarget.property.codeArtifactOptions">codeArtifactOptions</a></code> | <code>projen.release.CodeArtifactOptions</code> | Options for publishing to AWS CodeArtifact. |
| <code><a href="#projen.cdk.JsiiPythonTarget.property.twinePasswordSecret">twinePasswordSecret</a></code> | <code>string</code> | The GitHub secret which contains PyPI password. |
| <code><a href="#projen.cdk.JsiiPythonTarget.property.twineRegistryUrl">twineRegistryUrl</a></code> | <code>string</code> | The registry url to use when releasing packages. |
| <code><a href="#projen.cdk.JsiiPythonTarget.property.twineUsernameSecret">twineUsernameSecret</a></code> | <code>string</code> | The GitHub secret which contains PyPI user name. |
Expand Down Expand Up @@ -9358,6 +9359,19 @@ Additional tools to install in the publishing job.

---

##### `codeArtifactOptions`<sup>Optional</sup> <a name="codeArtifactOptions" id="projen.cdk.JsiiPythonTarget.property.codeArtifactOptions"></a>

```typescript
public readonly codeArtifactOptions: CodeArtifactOptions;
```

- *Type:* projen.release.CodeArtifactOptions
- *Default:* undefined

Options for publishing to AWS CodeArtifact.

---

##### `twinePasswordSecret`<sup>Optional</sup> <a name="twinePasswordSecret" id="projen.cdk.JsiiPythonTarget.property.twinePasswordSecret"></a>

```typescript
Expand Down
30 changes: 30 additions & 0 deletions docs/api/release.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2131,6 +2131,7 @@ const jsiiReleasePyPi: release.JsiiReleasePyPi = { ... }
| <code><a href="#projen.release.JsiiReleasePyPi.property.postPublishSteps">postPublishSteps</a></code> | <code>projen.github.workflows.JobStep[]</code> | Steps to execute after executing the publishing command. |
| <code><a href="#projen.release.JsiiReleasePyPi.property.prePublishSteps">prePublishSteps</a></code> | <code>projen.github.workflows.JobStep[]</code> | Steps to execute before executing the publishing command. These can be used to prepare the artifact for publishing if neede. |
| <code><a href="#projen.release.JsiiReleasePyPi.property.publishTools">publishTools</a></code> | <code>projen.github.workflows.Tools</code> | Additional tools to install in the publishing job. |
| <code><a href="#projen.release.JsiiReleasePyPi.property.codeArtifactOptions">codeArtifactOptions</a></code> | <code><a href="#projen.release.CodeArtifactOptions">CodeArtifactOptions</a></code> | Options for publishing to AWS CodeArtifact. |
| <code><a href="#projen.release.JsiiReleasePyPi.property.twinePasswordSecret">twinePasswordSecret</a></code> | <code>string</code> | The GitHub secret which contains PyPI password. |
| <code><a href="#projen.release.JsiiReleasePyPi.property.twineRegistryUrl">twineRegistryUrl</a></code> | <code>string</code> | The registry url to use when releasing packages. |
| <code><a href="#projen.release.JsiiReleasePyPi.property.twineUsernameSecret">twineUsernameSecret</a></code> | <code>string</code> | The GitHub secret which contains PyPI user name. |
Expand Down Expand Up @@ -2191,6 +2192,21 @@ Additional tools to install in the publishing job.

---

##### ~~`codeArtifactOptions`~~<sup>Optional</sup> <a name="codeArtifactOptions" id="projen.release.JsiiReleasePyPi.property.codeArtifactOptions"></a>

- *Deprecated:* Use `PyPiPublishOptions` instead.

```typescript
public readonly codeArtifactOptions: CodeArtifactOptions;
```

- *Type:* <a href="#projen.release.CodeArtifactOptions">CodeArtifactOptions</a>
- *Default:* undefined

Options for publishing to AWS CodeArtifact.

---

##### ~~`twinePasswordSecret`~~<sup>Optional</sup> <a name="twinePasswordSecret" id="projen.release.JsiiReleasePyPi.property.twinePasswordSecret"></a>

- *Deprecated:* Use `PyPiPublishOptions` instead.
Expand Down Expand Up @@ -3002,6 +3018,7 @@ const pyPiPublishOptions: release.PyPiPublishOptions = { ... }
| <code><a href="#projen.release.PyPiPublishOptions.property.postPublishSteps">postPublishSteps</a></code> | <code>projen.github.workflows.JobStep[]</code> | Steps to execute after executing the publishing command. |
| <code><a href="#projen.release.PyPiPublishOptions.property.prePublishSteps">prePublishSteps</a></code> | <code>projen.github.workflows.JobStep[]</code> | Steps to execute before executing the publishing command. These can be used to prepare the artifact for publishing if neede. |
| <code><a href="#projen.release.PyPiPublishOptions.property.publishTools">publishTools</a></code> | <code>projen.github.workflows.Tools</code> | Additional tools to install in the publishing job. |
| <code><a href="#projen.release.PyPiPublishOptions.property.codeArtifactOptions">codeArtifactOptions</a></code> | <code><a href="#projen.release.CodeArtifactOptions">CodeArtifactOptions</a></code> | Options for publishing to AWS CodeArtifact. |
| <code><a href="#projen.release.PyPiPublishOptions.property.twinePasswordSecret">twinePasswordSecret</a></code> | <code>string</code> | The GitHub secret which contains PyPI password. |
| <code><a href="#projen.release.PyPiPublishOptions.property.twineRegistryUrl">twineRegistryUrl</a></code> | <code>string</code> | The registry url to use when releasing packages. |
| <code><a href="#projen.release.PyPiPublishOptions.property.twineUsernameSecret">twineUsernameSecret</a></code> | <code>string</code> | The GitHub secret which contains PyPI user name. |
Expand Down Expand Up @@ -3056,6 +3073,19 @@ Additional tools to install in the publishing job.

---

##### `codeArtifactOptions`<sup>Optional</sup> <a name="codeArtifactOptions" id="projen.release.PyPiPublishOptions.property.codeArtifactOptions"></a>

```typescript
public readonly codeArtifactOptions: CodeArtifactOptions;
```

- *Type:* <a href="#projen.release.CodeArtifactOptions">CodeArtifactOptions</a>
- *Default:* undefined

Options for publishing to AWS CodeArtifact.

---

##### `twinePasswordSecret`<sup>Optional</sup> <a name="twinePasswordSecret" id="projen.release.PyPiPublishOptions.property.twinePasswordSecret"></a>

```typescript
Expand Down
96 changes: 87 additions & 9 deletions src/release/publisher.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -501,25 +501,74 @@ export class Publisher extends Component {
* @param options Options
*/
public publishToPyPi(options: PyPiPublishOptions = {}) {
let permissions: JobPermissions = { contents: JobPermission.READ };
const prePublishSteps = options.prePublishSteps ?? [];
let workflowEnv: Record<string, string | undefined> = {};
const isAwsCodeArtifact = isAwsCodeArtifactRegistry(
options.twineRegistryUrl
);
if (isAwsCodeArtifact) {
const { domain, account, region } = awsCodeArtifactInfoFromUrl(
options.twineRegistryUrl
);
const {
authProvider,
roleToAssume,
accessKeyIdSecret,
secretAccessKeySecret,
} = options.codeArtifactOptions ?? {};
const useOidcAuth = authProvider === CodeArtifactAuthProvider.GITHUB_OIDC;
if (useOidcAuth) {
if (!roleToAssume) {
throw new Error(
'"roleToAssume" property is required when using GITHUB_OIDC for AWS CodeArtifact options'
);
}
permissions = { ...permissions, idToken: JobPermission.WRITE };
prePublishSteps.push({
name: "Configure AWS Credentials via GitHub OIDC Provider",
uses: "aws-actions/configure-aws-credentials@v4",
with: {
"role-to-assume": roleToAssume,
"aws-region": region,
},
});
}
prePublishSteps.push({
name: "Generate CodeArtifact Token",
run: `echo "TWINE_PASSWORD=$(aws codeartifact get-authorization-token --domain ${domain} --domain-owner ${account} --region ${region} --query authorizationToken --output text)" >> $GITHUB_ENV`,
env: useOidcAuth
? undefined
: {
AWS_ACCESS_KEY_ID: secret(
accessKeyIdSecret ?? "AWS_ACCESS_KEY_ID"
),
AWS_SECRET_ACCESS_KEY: secret(
secretAccessKeySecret ?? "AWS_SECRET_ACCESS_KEY"
),
},
});
workflowEnv = { TWINE_USERNAME: "aws" };
} else {
workflowEnv = {
TWINE_USERNAME: secret(options.twineUsernameSecret ?? "TWINE_USERNAME"),
TWINE_PASSWORD: secret(options.twinePasswordSecret ?? "TWINE_PASSWORD"),
};
}

this.addPublishJob(
(_branch, _branchOptions): PublishJobOptions => ({
name: "pypi",
registryName: "PyPI",
publishTools: PUBLIB_TOOLCHAIN.python,
prePublishSteps: options.prePublishSteps ?? [],
permissions,
prePublishSteps,
postPublishSteps: options.postPublishSteps ?? [],
run: this.publibCommand("publib-pypi"),
env: {
TWINE_REPOSITORY_URL: options.twineRegistryUrl,
},
workflowEnv: {
TWINE_USERNAME: secret(
options.twineUsernameSecret ?? "TWINE_USERNAME"
),
TWINE_PASSWORD: secret(
options.twinePasswordSecret ?? "TWINE_PASSWORD"
),
},
workflowEnv,
})
);
}
Expand Down Expand Up @@ -977,6 +1026,13 @@ export interface PyPiPublishOptions extends CommonPublishOptions {
* @default "TWINE_PASSWORD"
*/
readonly twinePasswordSecret?: string;

/**
* Options for publishing to AWS CodeArtifact.
*
* @default - undefined
*/
readonly codeArtifactOptions?: CodeArtifactOptions;
}

/**
Expand Down Expand Up @@ -1190,6 +1246,28 @@ export function isAwsCodeArtifactRegistry(registryUrl: string | undefined) {
return registryUrl && AWS_CODEARTIFACT_REGISTRY_REGEX.test(registryUrl);
}

/**
* Info extracted from AWS CodeArtifact URL
*/
interface AwsCodeArtifactInfo {
readonly domain?: string;
readonly account?: string;
readonly region?: string;
}

/**
* Parses info about code artifact domain from given AWS code artifact url
* @param url Of code artifact domain
* @returns domain, account, and region of code artifact domain
*/
function awsCodeArtifactInfoFromUrl(url?: string): AwsCodeArtifactInfo {
const captureRegex =
/([a-z0-9-]+)-(.+)\.d\.codeartifact\.(.+)\.amazonaws\.com/;
const matches = url?.match(captureRegex) ?? [];
const [_, domain, account, region] = matches;
return { domain, account, region };
}

/**
* Publishing options for GitHub releases.
*/
Expand Down

0 comments on commit cf67e86

Please sign in to comment.