Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(rosetta): extract and compile samples into "tablets" #925

Merged
merged 24 commits into from
Nov 13, 2019
Merged

feat(rosetta): extract and compile samples into "tablets" #925

merged 24 commits into from
Nov 13, 2019

Conversation

rix0rrr
Copy link
Contributor

@rix0rrr rix0rrr commented Oct 31, 2019 •
edited
Loading

Version 2 of the "sampiler" is called "Rosetta". It now provides
more control over the compilation of samples found in the source code.

It integrates into an existing build by running jsii-rosetta extract,
which will extract sample code from a jsii assembly, compile it, convert
it to all supported languages, and storing the result in a a "tablet
file" (effectively, a sample dictionary).

Tablet files can then be used by jsii-pacmak to look up translations
for the code samples it encounters as it is generating language-specific
sources.

In case the build does not contain a Rosetta step, Pacmak will try to
convert samples that are not found in the tablet on the fly. However,
the samples will not benefit from compilation, type checking and fixture
support.

This change also fixes the jsii-pacmak all-at-once builder, and will
properly copy out binaries for the .NET and Java builds back to the
declared output directories, and properly use output directories of
packages not included in the megabuild as local package repositories.

Fixes #

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@rix0rrr
feat(rosetta): extract and compile samples into "tablets"
d990d68 
Version 2 of the "sampiler" is called "Rosetta". It now provides
more control over the compilation of samples found in the source code.

It integrates into an existing build by running `jsii-rosetta extract`,
which will extract sample code from a jsii assembly, compile it, convert
it to all supported languages, and storing the result in a a "tablet
file" (effectively, a sample dictionary).

Tablet files can then be used by `jsii-pacmak` to look up translations
for the code samples it encounters as it is generating language-specific
sources.

In case the build does not contain a Rosetta step, Pacmak will try to
convert samples that are not found in the tablet on the fly. However,
the samples will not benefit from compilation, type checking and fixture
support.
@rix0rrr rix0rrr requested a review from a team as a code owner October 31, 2019 13:42
@rix0rrr rix0rrr self-assigned this Oct 31, 2019
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

eladb
eladb approved these changes Nov 5, 2019
View reviewed changes
Copy link
Contributor

@eladb eladb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Masterpiece!

I think the tooling ergonomics can be improved and eventually we should merge all of them into jsii so it's a one stop shop.

packages/jsii-calc-base-of-base/package.json Outdated Show resolved Hide resolved
packages/jsii-calc-lib/package.json Outdated Show resolved Hide resolved
packages/jsii-calc/package.json Outdated Show resolved Hide resolved
packages/jsii-pacmak/bin/jsii-pacmak.ts Show resolved Hide resolved
packages/jsii-pacmak/bin/jsii-pacmak.ts Outdated Show resolved Hide resolved
packages/jsii-rosetta/lib/o-tree.ts Show resolved Hide resolved
packages/jsii-rosetta/bin/jsii-rosetta.ts Show resolved Hide resolved
packages/jsii-rosetta/bin/jsii-rosetta.ts Outdated Show resolved Hide resolved
packages/jsii-rosetta/bin/jsii-rosetta.ts Show resolved Hide resolved
packages/jsii-rosetta/bin/jsii-rosetta.ts Outdated Show resolved Hide resolved
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Rico Huijbers added 2 commits November 11, 2019 15:54
Merge remote-tracking branch 'origin/master' into huijbers/extract-sa…
8a0d470 
…mples
Fix pacmak unified build sanity
5380991 
For both .NET and Java, build in a temporary directory and copy
artifacts out to final destination.

Fix .NET projects to build all in one solution, and use project
references to automatically fix build order.
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

rix0rrr added a commit to aws/aws-cdk that referenced this pull request Nov 13, 2019
@rix0rrr
chore: examples in aws-ec2 package are compilable
2b160ad 
Using the new "rosetta" sample compiler from aws/jsii#925, introduce
fixtures and fix up sample code to make all examples in the `aws-ec2`
package compile.

This serves as a demonstration of how to set up fixtures and how
to write the examples.
@rix0rrr rix0rrr mentioned this pull request Nov 13, 2019
Merged
@rix0rrr rix0rrr merged commit eec44e1 into master Nov 13, 2019
@rix0rrr rix0rrr deleted the huijbers/extract-samples branch November 13, 2019 14:47
mergify bot pushed a commit to aws/aws-cdk that referenced this pull request Jan 6, 2020
@rix0rrr @mergify
chore: make examples in aws-ec2 package compilable
7885025 
Using the new "rosetta" sample compiler from aws/jsii#925, introduce
fixtures and fix up sample code to make all examples in the `aws-ec2`
package compile.

This serves as a demonstration of how to set up fixtures and how
to write the examples.

Need to reverse the order between `@example` and `@attribute`; because 
of a bug in the TypeScript compiler, it will think a new tag starts if
a `@` comes up in the example. This means that we cannot type
`import s3 = require('@aws-cdk/aws-s3');` in examples (it would parse `@aws = -cdk/aws-s3');`.

To fix that, we absorb non-recognized tags that follow an `@example` 
back into the example body, but since `@attribute` is non-recognized, 
we'd absorb that as well. The solution is to have `@example` as the last tag.

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

<!-- 
Please read the contribution guidelines and follow the pull-request checklist:
https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md
 -->
mergify bot pushed a commit that referenced this pull request Aug 1, 2022
@dependabot
chore(deps): Update attrs requirement from ~=21.2 to >=21.2,<23.0 in …
5275da6 
…/packages/@jsii/python-runtime (#3692)

Updates the requirements on [attrs](https://github.com/python-attrs/attrs) to permit the latest version.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/python-attrs/attrs/releases">attrs's releases</a>.</em></p>
<blockquote>
<h2>22.1.0</h2>
<h1>Highlights</h1>
<p>The main features of this release are:</p>
<ul>
<li>The departure of Python 2.7 (enjoy your retirement!),</li>
<li>and the arrival of Python 3.11.</li>
</ul>
<p>We had loftier goals feature-wise, but didn't want to block others embracing Python 3.11.</p>
<p>❤️ <strong>Huge</strong> thanks to my <a href="https://github.com/sponsors/hynek">GitHub sponsors</a>, <a href="https://tidelift.com/subscription/pkg/pypi-attrs">Tidelift subscribers</a>, and <a href="https://ko-fi.com/the_hynek">Ko-fi buyers</a>! ❤️</p>
<p>None of my projects would exist in their current form without you!</p>
<h1>Full Changelog</h1>
<h2>Backwards-incompatible Changes</h2>
<ul>
<li>
<p>Python 2.7 is not supported anymore.</p>
<p>Dealing with Python 2.7 tooling has become too difficult for a volunteer-run project.</p>
<p>We have supported Python 2 more than 2 years after it was officially discontinued and feel that we have paid our dues. All version up to 21.4.0 from December 2021 remain fully functional, of course. <a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/936">#936</a></p>
</li>
<li>
<p>The deprecated <code>cmp</code> attribute of <code>attrs.Attribute</code> has been removed. This does not affect the <em>cmp</em> argument to <code>attr.s</code> that can be used as a shortcut to set <em>eq</em> and <em>order</em> at the same time. <a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/939">#939</a></p>
</li>
</ul>
<h2>Changes</h2>
<ul>
<li>Instantiation of frozen slotted classes is now faster. <a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/898">#898</a></li>
<li>If an <code>eq</code> key is defined, it is also used before hashing the attribute. <a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/909">#909</a></li>
<li>Added <code>attrs.validators.min_len()</code>. <a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/916">#916</a></li>
<li><code>attrs.validators.deep_iterable()</code>'s <em>member_validator</em> argument now also accepts a list of validators and wraps them in an <code>attrs.validators.and_()</code>. <a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/925">#925</a></li>
<li>Added missing type stub re-imports for <code>attrs.converters</code> and <code>attrs.filters</code>. <a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/931">#931</a></li>
<li>Added missing stub for <code>attr(s).cmp_using()</code>. <a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/949">#949</a></li>
<li><code>attrs.validators._in()</code>'s <code>ValueError</code> is not missing the attribute, expected options, and the value it got anymore. <a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/951">#951</a></li>
<li>Python 3.11 is now officially supported. <a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/969">#969</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/python-attrs/attrs/blob/main/CHANGELOG.rst">attrs's changelog</a>.</em></p>
<blockquote>
<h2>22.1.0 (2022-07-28)</h2>
<p>Backwards-incompatible Changes
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^</p>
<ul>
<li>
<p>Python 2.7 is not supported anymore.</p>
<p>Dealing with Python 2.7 tooling has become too difficult for a volunteer-run project.</p>
<p>We have supported Python 2 more than 2 years after it was officially discontinued and feel that we have paid our dues.
All version up to 21.4.0 from December 2021 remain fully functional, of course.
<code>[#936](python-attrs/attrs#936) &lt;https://github.com/python-attrs/attrs/issues/936&gt;</code>_</p>
</li>
<li>
<p>The deprecated <code>cmp</code> attribute of <code>attrs.Attribute</code> has been removed.
This does not affect the <em>cmp</em> argument to <code>attr.s</code> that can be used as a shortcut to set <em>eq</em> and <em>order</em> at the same time.
<code>[#939](python-attrs/attrs#939) &lt;https://github.com/python-attrs/attrs/issues/939&gt;</code>_</p>
</li>
</ul>
<p>Changes
^^^^^^^</p>
<ul>
<li>Instantiation of frozen slotted classes is now faster.
<code>[#898](python-attrs/attrs#898) &lt;https://github.com/python-attrs/attrs/issues/898&gt;</code>_</li>
<li>If an <code>eq</code> key is defined, it is also used before hashing the attribute.
<code>[#909](python-attrs/attrs#909) &lt;https://github.com/python-attrs/attrs/issues/909&gt;</code>_</li>
<li>Added <code>attrs.validators.min_len()</code>.
<code>[#916](python-attrs/attrs#916) &lt;https://github.com/python-attrs/attrs/issues/916&gt;</code>_</li>
<li><code>attrs.validators.deep_iterable()</code>'s <em>member_validator</em> argument now also accepts a list of validators and wraps them in an <code>attrs.validators.and_()</code>.
<code>[#925](python-attrs/attrs#925) &lt;https://github.com/python-attrs/attrs/issues/925&gt;</code>_</li>
<li>Added missing type stub re-imports for <code>attrs.converters</code> and <code>attrs.filters</code>.
<code>[#931](python-attrs/attrs#931) &lt;https://github.com/python-attrs/attrs/issues/931&gt;</code>_</li>
<li>Added missing stub for <code>attr(s).cmp_using()</code>.
<code>[#949](python-attrs/attrs#949) &lt;https://github.com/python-attrs/attrs/issues/949&gt;</code>_</li>
<li><code>attrs.validators._in()</code>'s <code>ValueError</code> is not missing the attribute, expected options, and the value it got anymore.
<code>[#951](python-attrs/attrs#951) &lt;https://github.com/python-attrs/attrs/issues/951&gt;</code>_</li>
<li>Python 3.11 is now officially supported.
<code>[#969](python-attrs/attrs#969) &lt;https://github.com/python-attrs/attrs/issues/969&gt;</code>_</li>
</ul>
<hr />
<h2>21.4.0 (2021-12-29)</h2>
<p>Changes
^^^^^^^</p>
<ul>
<li>Fixed the test suite on PyPy3.8 where <code>cloudpickle</code> does not work.
<code>[#892](python-attrs/attrs#892) &lt;https://github.com/python-attrs/attrs/issues/892&gt;</code>_</li>
</ul>

</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/python-attrs/attrs/commit/b3dfebe2e10b44437c4f97d788fb5220d790efd0"><code>b3dfebe</code></a> Prepare 22.1.0</li>
<li><a href="https://github.com/python-attrs/attrs/commit/c89abcd6e6a826b00898f93e851c96b78c80891f"><code>c89abcd</code></a> It totally is correct</li>
<li><a href="https://github.com/python-attrs/attrs/commit/9f118b7dd5328ee5abd7f3880971ab9554047c3a"><code>9f118b7</code></a> Tune first steps</li>
<li><a href="https://github.com/python-attrs/attrs/commit/a3d7f20e54ba29d5ec73563c167d877351228cf2"><code>a3d7f20</code></a> Polish contributing guide</li>
<li><a href="https://github.com/python-attrs/attrs/commit/899497f165b3c8291b66c9c3a01b64881b69cf39"><code>899497f</code></a> Clarify (c) ownership</li>
<li><a href="https://github.com/python-attrs/attrs/commit/696fd786901a6b82b828dd71e3fd64bfae1014ca"><code>696fd78</code></a> Hyphenate compound adjectives</li>
<li><a href="https://github.com/python-attrs/attrs/commit/65c06831c1eee4d9895511c7db5caffba9d93c0e"><code>65c0683</code></a> Use NG APIs in glossary</li>
<li><a href="https://github.com/python-attrs/attrs/commit/5d84d9a5686e8210b1616447ac05021b6b211157"><code>5d84d9a</code></a> Move mypy config to pyproject.toml</li>
<li><a href="https://github.com/python-attrs/attrs/commit/1590917bf7c49e35bb429d117650f660aa45e0e4"><code>1590917</code></a> Update readme.rst (<a href="https://github-redirect.dependabot.com/python-attrs/attrs/issues/986">#986</a>)</li>
<li><a href="https://github.com/python-attrs/attrs/commit/a67c84f51e4e3df875961d287583abaef673eb48"><code>a67c84f</code></a> Add more prominent callout about slots to API docs</li>
<li>Additional commits viewable in <a href="https://github.com/python-attrs/attrs/compare/21.2.0...22.1.0">compare view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@eladb eladb eladb approved these changes

@RomainMuller RomainMuller Awaiting requested review from RomainMuller

@skinny85 skinny85 Awaiting requested review from skinny85

@bmaizels bmaizels Awaiting requested review from bmaizels

Assignees

@rix0rrr rix0rrr

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@rix0rrr @aws-cdk-automation @eladb