feat(pacmak): build all java targets at once #849

rix0rrr · 2019-10-03T16:47:35Z

Improve build speed by building all jsii targets at the same time,
and using a special aggregate POM build for Java.

Improves speed by not unnecessarily re-reading jsii assemblies
for each individual build.
Improves speed by using an aggregate POM to build all Java
targets (can reuse JVM instances and compilation metadata by Maven).
Improves wall-clock time by building all languages in parallel.

On my machine, reduces the time to pack the entire CDK repository
from 25min user/15min wall clock to 15min user/5 min wall clock.

Fixes #

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Improve build speed by building all jsii targets at the same time, and using a special aggregate POM build for Java. - Improves speed by not unnecessarily re-reading jsii assemblies for each individual build. - Improves speed by using an aggregate POM to build all Java targets (can reuse JVM instances and compilation metadata by Maven). - Improves wall-clock time by building all languages in parallel. On my machine, reduces the time to pack the entire CDK repository from 25min user/15min wall clock to 15min user/5 min wall clock.

Improve build times by using the new "build-all-at-once" feature of jsii-pacmak. Goes together with aws/jsii#849.

aws-cdk-automation · 2019-10-03T16:51:13Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2019-10-04T10:12:52Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2019-10-04T11:15:39Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2019-10-04T11:41:37Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2019-10-04T11:45:08Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2019-10-07T09:43:08Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2019-10-07T12:31:00Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2019-10-07T13:27:23Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2019-10-08T09:53:04Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2019-10-08T11:36:04Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2019-10-08T11:49:01Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2019-10-08T12:41:06Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2019-10-08T14:22:27Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

RomainMuller

Pushing on a partial review so you can start working through the comments I have thus far (didn't have enough time to go through it all, and I don't want this PR to sit in review for longer than it must)

RomainMuller · 2019-10-08T15:03:12Z

packages/jsii-pacmak/bin/jsii-pacmak.ts

+import { JsiiModule } from '../lib/packaging';
+import { Thunk, ConcurrencyLimiter } from '../lib/util';
+import { ALL_BUILDERS, TargetName } from '../lib/targets';
+import pLimit from 'p-limit';


Why do we need this? Can't we run unbounded?

100 jobs in parallel? I don't think so.

packages/jsii-pacmak/bin/jsii-pacmak.ts

RomainMuller · 2019-10-08T15:06:49Z

packages/jsii-pacmak/bin/jsii-pacmak.ts

@@ -54,6 +58,11 @@ import { VERSION_DESC } from '../lib/version';
      desc: 'force generation into a target-named subdirectory, even in single-target mode',
      default: true,
    })
+    .option('force-target', {
+      type: 'boolean',
+      desc: 'force generation of the given targets, even if the source package.json doesnt declare it',


This can typically not be expected to work (missing inputs for many targets). I'd rather not even offer the option.

Hah. Well the Ruby runtime depended on it (wanted to generate ruby code for jsii-calc and friends, even though it wasn't declared and is otherwise untested). My first instinct was to get rid of the ruby runtime, but since I already overstepped my bounds a bit by removing the Sphinx target (ooohhh) I didn't want to make the scope of this PR even bigger and risk incurring more discussion.

I also did not want to enshrine the current Ruby generation into jsii-pacmak's expected tests, so this seemed like the smallest delta to keep the repository building without drastically changing things.

Oh I see. I wouldn't have opposed dropping Ruby (having side concerns about he lost effort, but that's another discussion).

Regardless - maybe just mention in the description that --force-target might cause jsii-pacmak to exit in error.

RomainMuller · 2019-10-08T15:07:08Z

packages/jsii-pacmak/bin/jsii-pacmak.ts

+    .option('concurrency', {
+      alias: 'C',
+      type: 'boolean',
+      desc: 'whether to work in parallel when possible',


What's the reason not to?

Your CPU might not like it.

Just, control to the user.

packages/jsii-pacmak/bin/jsii-pacmak.ts

RomainMuller · 2019-10-08T15:21:46Z

packages/jsii-pacmak/bin/jsii-pacmak.ts

      force: argv.force,
-      arguments: argv
+      fingerprint: argv.fingerprint,
+      arguments: argv,


Looks like the keys were sorted alphanumerically, except for this one?

RomainMuller · 2019-10-08T15:24:19Z

packages/jsii-pacmak/bin/jsii-pacmak.ts

-
-      const ts = new reflect.TypeSystem();
-      const assembly = await ts.loadModule(packageDir);
+    const perLanguageDirectory = targetSets.length > 1 || argv['force-subdirectory'];


Can we just settle on always having per-language directory?

Maybe. Some runtime tests depend on that not being the case, and I don't want to dive into the labyrinth of bash build scripts for all the runtimes. Already had to do too much of that and it scared/annoyed me.

RomainMuller · 2019-10-08T17:08:46Z

packages/jsii-pacmak/bin/jsii-pacmak.ts

+  await timers.recordAsync('npm pack', () => {
+    logging.info('Packaging NPM bundles');
+    return awaitAll(modulesToPackage
+      .map(m => () => m.npmPack()));
+  });


Would we not extract the logging bit & make this a little less syntactically heavy?

RomainMuller · 2019-10-08T17:09:58Z

packages/jsii-pacmak/bin/jsii-pacmak.ts

-  let modified = false;
-  if (await fs.pathExists(npmIgnorePath)) {
-    lines = (await fs.readFile(npmIgnorePath)).toString().split('\n');
+function sliceTargets(modules: JsiiModule[], requestedTargets: string[] | undefined, force: boolean) {


I think this should make sure all available modules support all requested targets (or only return targets supported by all). If we brought in additional modules for dependencies, and some dependencies don't support a target, then the build is doomed to fail anyway...

Could be, although to do that properly we would have to record the dependencies (not all packages are necessarily dependencies of each other).

So unrelated packages A and B could look like:

A: java+python B: python

And I might want to build all targets. Though unlikely, I did not see a reason to prohibit that kind of build. Since the goal of this is speed, independent builds would require analyzing A twice, which would take away some of the purpose.

RomainMuller · 2019-10-08T17:10:21Z

packages/jsii-pacmak/bin/jsii-pacmak.ts

+function describePackages(target: TargetSet) {
+  if (target.modules.length > 0 && target.modules.length < 5) {
+    return target.modules.map(m => m.name).join(', ');
  }
-}
+  return `${target.modules.length} modules`;
+}


RomainMuller · 2019-10-09T11:28:53Z

packages/jsii-pacmak/lib/builder.ts

+}
+
+/**
+ * Return the output directory if all modules have the same directory


That comment isn't an accurate description of what the method does.

RomainMuller · 2019-10-09T11:32:32Z

packages/jsii-pacmak/lib/npm-modules.ts

+  const ret: JsiiModule[] = [];
+  const visited = new Set<string>();
+  for (const dir of directories.length > 0 ? directories : ['.']) {
+    await visitPackage(dir, true);


Would be mighty sweet if we could avoid await in for loops.

Why? I know eslint complains about it, but I don't really see a good reason why it does.

RomainMuller · 2019-10-09T11:33:00Z

packages/jsii-pacmak/lib/npm-modules.ts

+    if (visited.has(realPath)) { return; } // Already visited
+    visited.add(realPath);
+
+    const pkg = await fs.readJson(path.join(realPath, 'package.json'));


This could be require'd, which could then not require an `await.

RomainMuller · 2019-10-09T11:34:42Z

packages/jsii-pacmak/lib/npm-modules.ts

+  }
+}
+
+async function updateNpmIgnore(packageDir: string, excludeOutdir: string | undefined) {


Seriously, this function is very long for the value it produces. I would be inclined to just stopping doing that.

RomainMuller · 2019-10-09T11:35:05Z

packages/jsii-pacmak/lib/packaging.ts

+  */
+  defaultOutputDirectory: string;
+}
+export class JsiiModule {


Missing blank line

RomainMuller · 2019-10-09T11:36:26Z

packages/jsii-pacmak/lib/packaging.ts

+
+  public get assembly(): reflect.Assembly {
+    if (!this._assembly) {
+      throw new Error('Assembly not available yet, call load() first');


Couldn't this call load() instead of barfing? This class has so much temporal coupling in it's features 😕

No because load() is async and this getter isn't.

RomainMuller · 2019-10-09T11:37:39Z

packages/jsii-pacmak/lib/targets/index.ts

+
+export type TargetName = 'dotnet' | 'java' | 'js' | 'python' | 'ruby';
+
+export const ALL_BUILDERS: {[key in TargetName]: TargetBuilder} = {


Spaces within the curly braces, please?

Noooooooooooo :)

RomainMuller · 2019-10-09T11:38:21Z

packages/jsii-pacmak/lib/targets/index.ts

+
+export type TargetName = 'dotnet' | 'java' | 'js' | 'python' | 'ruby';
+
+export const ALL_BUILDERS: {[key in TargetName]: TargetBuilder} = {


How about { readonly [key in TargetName]: TargetBuilder }?

RomainMuller · 2019-10-09T11:40:51Z

packages/jsii-pacmak/lib/targets/java.ts

+  private async generateAggregatePom(sourceDirectories: Array<Scratch<void>>) {
+    const parentDir = this.findSharedParentDirectory(sourceDirectories.map(s => s.directory));
+
+    const aggregatePom = xmlbuilder.create({


Would be interesting if this mode could also produce a Maven BOM file!

What's that?

aws-cdk-automation · 2019-10-10T14:34:56Z

AWS CodeBuild CI Report

Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

…uilds

aws-cdk-automation · 2019-10-11T11:21:07Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

rix0rrr · 2019-10-14T15:38:55Z

I removed the parallelism restriction. Can you re-review?

RomainMuller · 2019-10-16T07:55:33Z

packages/jsii-pacmak/bin/jsii-pacmak.ts

@@ -54,6 +58,11 @@ import { VERSION_DESC } from '../lib/version';
      desc: 'force generation into a target-named subdirectory, even in single-target mode',
      default: true,
    })
+    .option('force-target', {
+      type: 'boolean',
+      desc: 'force generation of the given targets, even if the source package.json doesnt declare it',


Oh I see. I wouldn't have opposed dropping Ruby (having side concerns about he lost effort, but that's another discussion).

Regardless - maybe just mention in the description that --force-target might cause jsii-pacmak to exit in error.

…uilds

aws-cdk-automation · 2019-10-17T12:38:41Z

AWS CodeBuild CI Report

Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Improve build times by using the new "build-all-at-once" feature of jsii-pacmak (for the Java build). Goes together with aws/jsii#849. Also parallelize the "does this package exist on NPM" checks.

…/packages/jsii-pacmak/lib/targets/python (#3367) Updates the requirements on [twine](https://github.com/pypa/twine) to permit the latest version. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/pypa/twine/releases">twine's releases</a>.</em></p> <blockquote> <h2>3.8.0</h2> <p><a href="https://pypi.org/project/twine/3.8.0/">https://pypi.org/project/twine/3.8.0/</a></p> <p><a href="https://twine.readthedocs.io/en/stable/changelog.html#twine-3-8-0-2022-02-02">Changelog</a></p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pypa/twine/blob/main/docs/changelog.rst">twine's changelog</a>.</em></p> <blockquote> <h2>Twine 3.8.0 (2022-02-02)</h2> <p>Features ^^^^^^^^</p> <ul> <li>Add <code>--verbose</code> logging for querying keyring credentials. (<code>[#849](pypa/twine#849) <https://github.com/pypa/twine/issues/849></code>_)</li> <li>Log all upload responses with <code>--verbose</code>. (<code>[#859](pypa/twine#859) <https://github.com/pypa/twine/issues/859></code>_)</li> <li>Show more helpful error message for invalid metadata. (<code>[#861](pypa/twine#861) <https://github.com/pypa/twine/issues/861></code>_)</li> </ul> <p>Bugfixes ^^^^^^^^</p> <ul> <li>Require a recent version of urllib3. (<code>[#858](pypa/twine#858) <https://github.com/pypa/twine/issues/858></code>_)</li> </ul> <h2>Twine 3.7.1 (2021-12-07)</h2> <p>Improved Documentation ^^^^^^^^^^^^^^^^^^^^^^</p> <ul> <li>Fix broken link to packaging tutorial. (<code>[#844](pypa/twine#844) <https://github.com/pypa/twine/issues/844></code>_)</li> </ul> <h2>Twine 3.7.0 (2021-12-01)</h2> <p>Features ^^^^^^^^</p> <ul> <li>Add support for core metadata version 2.2, defined in PEP 643. (<code>[#833](pypa/twine#833) <https://github.com/pypa/twine/issues/833></code>_)</li> </ul> <h2>Twine 3.6.0 (2021-11-10)</h2> <p>Features ^^^^^^^^</p> <ul> <li>Add support for Python 3.10. (<code>[#827](pypa/twine#827) <https://github.com/pypa/twine/issues/827></code>_)</li> </ul> <h2>Twine 3.5.0 (2021-11-02)</h2> <p>Features ^^^^^^^^</p> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pypa/twine/commit/c5769e0fe27064c77cd5b09512d686913d420f95"><code>c5769e0</code></a> Update 3.8.0 release date (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/863">#863</a>)</li> <li><a href="https://github.com/pypa/twine/commit/a8333644f21c8f2cf2c086375753fa04aa5b4424"><code>a833364</code></a> Update changelog for 3.8.0 (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/862">#862</a>)</li> <li><a href="https://github.com/pypa/twine/commit/cf9295f6af8c396a88968b3bef55af981fb62457"><code>cf9295f</code></a> Log all upload responses with <code>--verbose</code> (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/859">#859</a>)</li> <li><a href="https://github.com/pypa/twine/commit/a60c565dd4560f4afd4780f87ceba84973ffb9fa"><code>a60c565</code></a> Display a more helpful error message for invalid metadata (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/861">#861</a>)</li> <li><a href="https://github.com/pypa/twine/commit/fd8607d2524356f7e5a17ffa051bf4d2335b74bd"><code>fd8607d</code></a> Fix mypy error from <code>urllib.Retry</code> kwargs (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/858">#858</a>)</li> <li><a href="https://github.com/pypa/twine/commit/ce87465e0917ed1a9154beb911146e3fe5b022f9"><code>ce87465</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/twine/issues/850">#850</a> from bhrutledge/847-log-keyring</li> <li><a href="https://github.com/pypa/twine/commit/7d608424a77dfe10d63910a18db17723cb2ff213"><code>7d60842</code></a> Add changelog entry</li> <li><a href="https://github.com/pypa/twine/commit/5f1a5a478ca1f61f2ff99a951e021aed6863401c"><code>5f1a5a4</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/pypa/twine/issues/849">#849</a> from bhrutledge/847-log-keyring</li> <li><a href="https://github.com/pypa/twine/commit/86723c468b8cba1afc61419d1df6363beccb34a3"><code>86723c4</code></a> Add verbose logging for getting keyring credentials</li> <li>See full diff in <a href="https://github.com/pypa/twine/compare/3.7.1...3.8.0">compare view</a></li> </ul> </details> <br /> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details>

…/packages/jsii-pacmak/lib/targets/python (#3469) Updates the requirements on [twine](https://github.com/pypa/twine) to permit the latest version. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pypa/twine/blob/main/docs/changelog.rst">twine's changelog</a>.</em></p> <blockquote> <h2>Twine 4.0.0 (2022-03-31)</h2> <p>Features ^^^^^^^^</p> <ul> <li>Drop support for Python 3.6. (<code>[#869](pypa/twine#869) <https://github.com/pypa/twine/issues/869></code>_)</li> <li>Use Rich to add color to <code>upload</code> output. (<code>[#851](pypa/twine#851) <https://github.com/pypa/twine/issues/851></code>_)</li> <li>Use Rich to add color to <code>check</code> output. (<code>[#874](pypa/twine#874) <https://github.com/pypa/twine/issues/874></code>_)</li> <li>Use Rich instead of tqdm for upload progress bar. (<code>[#877](pypa/twine#877) <https://github.com/pypa/twine/issues/877></code>_)</li> </ul> <p>Bugfixes ^^^^^^^^</p> <ul> <li>Remove Twine's dependencies from the <code>User-Agent</code> header when uploading. (<code>[#871](pypa/twine#871) <https://github.com/pypa/twine/issues/871></code>_)</li> <li>Improve detection of disabled BLAKE2 hashing due to FIPS mode. (<code>[#879](pypa/twine#879) <https://github.com/pypa/twine/issues/879></code>_)</li> <li>Restore warning for missing <code>long_description</code>. (<code>[#887](pypa/twine#887) <https://github.com/pypa/twine/issues/887></code>_)</li> </ul> <h2>Twine 3.8.0 (2022-02-02)</h2> <p>Features ^^^^^^^^</p> <ul> <li>Add <code>--verbose</code> logging for querying keyring credentials. (<code>[#849](pypa/twine#849) <https://github.com/pypa/twine/issues/849></code>_)</li> <li>Log all upload responses with <code>--verbose</code>. (<code>[#859](pypa/twine#859) <https://github.com/pypa/twine/issues/859></code>_)</li> <li>Show more helpful error message for invalid metadata. (<code>[#861](pypa/twine#861) <https://github.com/pypa/twine/issues/861></code>_)</li> </ul> <p>Bugfixes ^^^^^^^^</p> <ul> <li>Require a recent version of urllib3. (<code>[#858](pypa/twine#858) <https://github.com/pypa/twine/issues/858></code>_)</li> </ul> <h2>Twine 3.7.1 (2021-12-07)</h2> <p>Improved Documentation ^^^^^^^^^^^^^^^^^^^^^^</p> <ul> <li>Fix broken link to packaging tutorial. (<code>[#844](pypa/twine#844) <https://github.com/pypa/twine/issues/844></code>_)</li> </ul> <h2>Twine 3.7.0 (2021-12-01)</h2> <p>Features</p> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pypa/twine/commit/36695abf8837aba72d87304d99b789c3f2872c99"><code>36695ab</code></a> Update changelog for 4.0.0 (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/888">#888</a>)</li> <li><a href="https://github.com/pypa/twine/commit/4931a2a413229a77d1848bebd02b15f6d106ab69"><code>4931a2a</code></a> Make missing <code>long_description</code> check more flexible (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/887">#887</a>)</li> <li><a href="https://github.com/pypa/twine/commit/7cd0b236bf1b7531bc57ee7647fdb7054890486d"><code>7cd0b23</code></a> Subclass StringIO for _WarningStream. (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/886">#886</a>)</li> <li><a href="https://github.com/pypa/twine/commit/aa7c0473f0fc12b0b92376dc9d437929bde7713f"><code>aa7c047</code></a> Update sampleproject fixture (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/885">#885</a>)</li> <li><a href="https://github.com/pypa/twine/commit/a6dd69c79f7b5abfb79022092a5d3776a499e31b"><code>a6dd69c</code></a> Adopt Python 3.7+ syntax (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/882">#882</a>)</li> <li><a href="https://github.com/pypa/twine/commit/a0ba32dcd0ea5af5de7b88d6645b7743bf003760"><code>a0ba32d</code></a> Drop support for Python 3.6 (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/869">#869</a>)</li> <li><a href="https://github.com/pypa/twine/commit/55652f0a65d433860024650d0d66f62b690fe26c"><code>55652f0</code></a> Replace tqdm with Rich for progress bar (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/877">#877</a>)</li> <li><a href="https://github.com/pypa/twine/commit/c506b225b3ab57a40571001825dea0058a55807a"><code>c506b22</code></a> Filter unnecessary deps from User-Agent string (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/871">#871</a>)</li> <li><a href="https://github.com/pypa/twine/commit/a9e9cd6ecec3c4d4504704ccaba08be487d67ebf"><code>a9e9cd6</code></a> Fix detection of FIPS mode for blake2b (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/879">#879</a>)</li> <li><a href="https://github.com/pypa/twine/commit/f69d4b7e41b8d40342133d5c8e43df7316e73993"><code>f69d4b7</code></a> Use Rich for <code>print()</code> output (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/878">#878</a>)</li> <li>Additional commits viewable in <a href="https://github.com/pypa/twine/compare/3.8.0...4.0.0">compare view</a></li> </ul> </details> <br /> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details>

…/packages/jsii-pacmak/lib/targets/python (#3568) Updates the requirements on [twine](https://github.com/pypa/twine) to permit the latest version. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/pypa/twine/releases">twine's releases</a>.</em></p> <blockquote> <h2>4.0.1</h2> <p><a href="https://pypi.org/project/twine/4.0.1/">https://pypi.org/project/twine/4.0.1/</a></p> <p><a href="https://twine.readthedocs.io/en/stable/changelog.html#twine-4-0-1-2022-06-01">Changelog</a></p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pypa/twine/blob/main/docs/changelog.rst">twine's changelog</a>.</em></p> <blockquote> <h2>Twine 4.0.1 (2022-06-01)</h2> <p>Bugfixes ^^^^^^^^</p> <ul> <li>Improve logging when keyring fails. (<code>[#890](pypa/twine#890) <https://github.com/pypa/twine/issues/890></code>_)</li> <li>Reconfgure root logger to show all log messages. (<code>[#896](pypa/twine#896) <https://github.com/pypa/twine/issues/896></code>_)</li> </ul> <h2>Twine 4.0.0 (2022-03-31)</h2> <p>Features ^^^^^^^^</p> <ul> <li>Drop support for Python 3.6. (<code>[#869](pypa/twine#869) <https://github.com/pypa/twine/issues/869></code>_)</li> <li>Use Rich to add color to <code>upload</code> output. (<code>[#851](pypa/twine#851) <https://github.com/pypa/twine/issues/851></code>_)</li> <li>Use Rich to add color to <code>check</code> output. (<code>[#874](pypa/twine#874) <https://github.com/pypa/twine/issues/874></code>_)</li> <li>Use Rich instead of tqdm for upload progress bar. (<code>[#877](pypa/twine#877) <https://github.com/pypa/twine/issues/877></code>_)</li> </ul> <p>Bugfixes ^^^^^^^^</p> <ul> <li>Remove Twine's dependencies from the <code>User-Agent</code> header when uploading. (<code>[#871](pypa/twine#871) <https://github.com/pypa/twine/issues/871></code>_)</li> <li>Improve detection of disabled BLAKE2 hashing due to FIPS mode. (<code>[#879](pypa/twine#879) <https://github.com/pypa/twine/issues/879></code>_)</li> <li>Restore warning for missing <code>long_description</code>. (<code>[#887](pypa/twine#887) <https://github.com/pypa/twine/issues/887></code>_)</li> </ul> <h2>Twine 3.8.0 (2022-02-02)</h2> <p>Features ^^^^^^^^</p> <ul> <li>Add <code>--verbose</code> logging for querying keyring credentials. (<code>[#849](pypa/twine#849) <https://github.com/pypa/twine/issues/849></code>_)</li> <li>Log all upload responses with <code>--verbose</code>. (<code>[#859](pypa/twine#859) <https://github.com/pypa/twine/issues/859></code>_)</li> <li>Show more helpful error message for invalid metadata. (<code>[#861](pypa/twine#861) <https://github.com/pypa/twine/issues/861></code>_)</li> </ul> <p>Bugfixes ^^^^^^^^</p> <ul> <li>Require a recent version of urllib3. (<code>[#858](pypa/twine#858) <https://github.com/pypa/twine/issues/858></code>_)</li> </ul> <h2>Twine 3.7.1 (2021-12-07)</h2> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pypa/twine/commit/8f5e5d6d42d582ef3ea6ef07da277e0cabd22fd2"><code>8f5e5d6</code></a> Update changelog for 4.0.1 (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/904">#904</a>)</li> <li><a href="https://github.com/pypa/twine/commit/62f3c67fa2f74cde433d6003b7ebf4256f129a7d"><code>62f3c67</code></a> Log keyring tracebacks (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/890">#890</a>)</li> <li><a href="https://github.com/pypa/twine/commit/d30df7038fd3545e30a2c9bd3728aa787659aa38"><code>d30df70</code></a> Update links to requests docs (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/899">#899</a>)</li> <li><a href="https://github.com/pypa/twine/commit/5525a2a628317eecb891859e395b0a54f2c57043"><code>5525a2a</code></a> Restore missing <code>__main__</code> logs (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/896">#896</a>)</li> <li><a href="https://github.com/pypa/twine/commit/b0b932f2da604e90f8a7a5a5c7e2841f519a8fb7"><code>b0b932f</code></a> Fix typos in tests (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/898">#898</a>)</li> <li><a href="https://github.com/pypa/twine/commit/4223ee154f1c962a0c33e2a3a95ed4c42bc62d41"><code>4223ee1</code></a> Require latest version of readme_renderer (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/892">#892</a>)</li> <li>See full diff in <a href="https://github.com/pypa/twine/compare/4.0.0...4.0.1">compare view</a></li> </ul> </details> <br /> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details>

…/packages/jsii-pacmak/lib/targets/python (#3868) Updates the requirements on [twine](https://github.com/pypa/twine) to permit the latest version. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/pypa/twine/releases">twine's releases</a>.</em></p> <blockquote> <h2>4.0.2</h2> <p><a href="https://pypi.org/project/twine/4.0.2/">https://pypi.org/project/twine/4.0.2/</a></p> <p><a href="https://twine.readthedocs.io/en/stable/changelog.html#twine-4-0-2-2022-11-30">Changelog</a></p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pypa/twine/blob/main/docs/changelog.rst">twine's changelog</a>.</em></p> <blockquote> <h2>Twine 4.0.2 (2022-11-30)</h2> <p>Bugfixes ^^^^^^^^</p> <ul> <li>Remove deprecated function to fix <code>twine check</code> with pkginfo 1.9.0. (<code>[#941](pypa/twine#941) <https://github.com/pypa/twine/issues/941></code>_)</li> </ul> <h2>Twine 4.0.1 (2022-06-01)</h2> <p>Bugfixes ^^^^^^^^</p> <ul> <li>Improve logging when keyring fails. (<code>[#890](pypa/twine#890) <https://github.com/pypa/twine/issues/890></code>_)</li> <li>Reconfgure root logger to show all log messages. (<code>[#896](pypa/twine#896) <https://github.com/pypa/twine/issues/896></code>_)</li> </ul> <h2>Twine 4.0.0 (2022-03-31)</h2> <p>Features ^^^^^^^^</p> <ul> <li>Drop support for Python 3.6. (<code>[#869](pypa/twine#869) <https://github.com/pypa/twine/issues/869></code>_)</li> <li>Use Rich to add color to <code>upload</code> output. (<code>[#851](pypa/twine#851) <https://github.com/pypa/twine/issues/851></code>_)</li> <li>Use Rich to add color to <code>check</code> output. (<code>[#874](pypa/twine#874) <https://github.com/pypa/twine/issues/874></code>_)</li> <li>Use Rich instead of tqdm for upload progress bar. (<code>[#877](pypa/twine#877) <https://github.com/pypa/twine/issues/877></code>_)</li> </ul> <p>Bugfixes ^^^^^^^^</p> <ul> <li>Remove Twine's dependencies from the <code>User-Agent</code> header when uploading. (<code>[#871](pypa/twine#871) <https://github.com/pypa/twine/issues/871></code>_)</li> <li>Improve detection of disabled BLAKE2 hashing due to FIPS mode. (<code>[#879](pypa/twine#879) <https://github.com/pypa/twine/issues/879></code>_)</li> <li>Restore warning for missing <code>long_description</code>. (<code>[#887](pypa/twine#887) <https://github.com/pypa/twine/issues/887></code>_)</li> </ul> <h2>Twine 3.8.0 (2022-02-02)</h2> <p>Features ^^^^^^^^</p> <ul> <li>Add <code>--verbose</code> logging for querying keyring credentials. (<code>[#849](pypa/twine#849) <https://github.com/pypa/twine/issues/849></code>_)</li> <li>Log all upload responses with <code>--verbose</code>. (<code>[#859](pypa/twine#859) <https://github.com/pypa/twine/issues/859></code>_)</li> <li>Show more helpful error message for invalid metadata. (<code>[#861](pypa/twine#861) <https://github.com/pypa/twine/issues/861></code>_)</li> </ul> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pypa/twine/commit/75c3d8623c0847d8ce5a59c1d14a9fcc71e2a4a2"><code>75c3d86</code></a> Release 4.0.2 (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/946">#946</a>)</li> <li><a href="https://github.com/pypa/twine/commit/5b5d081bd520ec0cf49f10ebaa52dfc582e5214f"><code>5b5d081</code></a> Fix twine( check) with the newly released pkginfo 1.9. (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/941">#941</a>)</li> <li><a href="https://github.com/pypa/twine/commit/717ae3d55b1b4a3e294319102ca70b33a4c6315c"><code>717ae3d</code></a> Fix failing CI (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/943">#943</a>)</li> <li><a href="https://github.com/pypa/twine/commit/bb51e46092c99de18e9309a3bee8fb996eb154e7"><code>bb51e46</code></a> Remove unused mypy ignores (<a href="https://github-redirect.dependabot.com/pypa/twine/issues/927">#927</a>)</li> <li>See full diff in <a href="https://github.com/pypa/twine/compare/4.0.1...4.0.2">compare view</a></li> </ul> </details> <br /> Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details>

rix0rrr requested review from RomainMuller and eladb October 3, 2019 16:47

rix0rrr requested a review from a team as a code owner October 3, 2019 16:47

rix0rrr added a commit to aws/aws-cdk that referenced this pull request Oct 3, 2019

chore: use new parallel build feature of jsii

ec56ad9

Improve build times by using the new "build-all-at-once" feature of jsii-pacmak. Goes together with aws/jsii#849.

rix0rrr mentioned this pull request Oct 3, 2019

chore: use new parallel build feature of jsii aws/aws-cdk#4367

Merged

Satisfy eslint

daacb88

dotnet build needs to emit absolute paths into NuGet config

f0c4c74

rix0rrr and others added 2 commits October 4, 2019 13:36

Remove 'shpinx' from all packages

e33180c

Update bootstrap.sh

0f0a47f

Fix assembly fingerprint

07953f7

Update more fingerprints

c1e06d0

Remove sphinx from kernel

57412b2

More build fixes

bbaddaa

Force building of Ruby targets

093bf6c

Satisfy eslint

6436234

Make all output dirs absolute

1fe325e

Merge branch 'master' into huijbers/parallel-builds

0deced4

RomainMuller reviewed Oct 8, 2019

View reviewed changes

RomainMuller suggested changes Oct 9, 2019

View reviewed changes

Remove parallelism restrictions

5eb81b1

Rico Huijbers added 2 commits October 11, 2019 13:04

Merge remote-tracking branch 'origin/master' into huijbers/parallel-b…

8582457

…uilds

Un-thunk all promises properly

d32ddc4

rix0rrr requested a review from RomainMuller October 14, 2019 15:39

RomainMuller approved these changes Oct 16, 2019

View reviewed changes

RomainMuller added 2 commits October 17, 2019 14:30

Merge remote-tracking branch 'origin/master' into huijbers/parallel-b…

d0d03d7

…uilds

Merge remote-tracking branch 'origin/master' into huijbers/parallel-b…

41f04d3

…uilds

RomainMuller removed the request for review from eladb October 17, 2019 13:05

RomainMuller merged commit 5d7824d into master Oct 17, 2019

RomainMuller deleted the huijbers/parallel-builds branch October 17, 2019 13:06


		export type TargetName = 'dotnet' \| 'java' \| 'js' \| 'python' \| 'ruby';

		export const ALL_BUILDERS: {[key in TargetName]: TargetBuilder} = {

feat(pacmak): build all java targets at once #849

feat(pacmak): build all java targets at once #849

Conversation

rix0rrr commented Oct 3, 2019

aws-cdk-automation commented Oct 3, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Oct 4, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Oct 4, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Oct 4, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Oct 4, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Oct 7, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Oct 7, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Oct 7, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Oct 8, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Oct 8, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Oct 8, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Oct 8, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Oct 8, 2019

AWS CodeBuild CI Report

RomainMuller left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rix0rrr Oct 9, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rix0rrr Oct 9, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aws-cdk-automation commented Oct 10, 2019

AWS CodeBuild CI Report

aws-cdk-automation commented Oct 11, 2019

AWS CodeBuild CI Report

rix0rrr commented Oct 14, 2019

Choose a reason for hiding this comment

aws-cdk-automation commented Oct 17, 2019

AWS CodeBuild CI Report

rix0rrr Oct 9, 2019 •

edited

Loading

rix0rrr Oct 9, 2019 •

edited

Loading