Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial commit for generating an integrated sample #137

Merged
merged 12 commits into from
Jul 11, 2019

Conversation

software-dov
Copy link
Contributor

Adds basic functionality and tests for end-to-end sample generation.

Includes tests for comprehensive sample generation

Conduct translation from special variable '$resp' to 'response' in
template rendering

Includes macros to set up method invocation, adds code to include
license and copyright information,

Includes minor tweaks to various names, tests, and macros.

Note: sample generation is still ignorant of method and message
signatures, types, attributes, and so forth.
TODOs mark these areas in the code.
More full-featured semantic analysis will be conducted
in subsequent PRs.

Adds basic functionality and tests for end-to-end sample generation.

Includes tests for comprehensive sample generation

Conduct translation from special variable '$resp' to 'response' in
template rendering

Includes macros to set up method invocation, adds code to include
license and copyright information,

Note: sample generation is still ignorant of method and message
      signatures, types, attributes, and so forth.
      More full-featured semantic analysis will be conducted
      in subsequent PRs.

Includes minor tweaks to various names, tests, and macros.
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Jul 3, 2019
@codecov
Copy link

codecov bot commented Jul 3, 2019

Codecov Report

Merging #137 into master will not change coverage.
The diff coverage is 100%.

Impacted file tree graph

@@          Coverage Diff          @@
##           master   #137   +/-   ##
=====================================
  Coverage     100%   100%           
=====================================
  Files          21     21           
  Lines         928    975   +47     
  Branches      198    206    +8     
=====================================
+ Hits          928    975   +47
Impacted Files Coverage Δ
gapic/schema/api.py 100% <ø> (ø) ⬆️
gapic/generator/generator.py 100% <100%> (ø) ⬆️
gapic/utils/__init__.py 100% <100%> (ø) ⬆️
gapic/samplegen/utils.py 100% <100%> (ø) ⬆️
gapic/utils/code.py 100% <100%> (ø) ⬆️
gapic/samplegen/samplegen.py 100% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6cb41c7...291848b. Read the comment docs.

Update type annotations for utils.code.partition
Flesh out docstrings
gapic/generator/generator.py Outdated Show resolved Hide resolved
tests/unit/samplegen/test_template.py Outdated Show resolved Hide resolved
tests/unit/samplegen/test_template.py Show resolved Hide resolved
@yihanzhen
Copy link

There are some changes which seem to be code formatting only. Is it possible to have a separate PR dedicated to fix formatting issues?

@software-dov
Copy link
Contributor Author

software-dov commented Jul 8, 2019

I'll open an issue for formatting tweaks and make a separate PR to address the issue.
Tracked by #141
Will yank the formatting changes on best effort, but the underlying issue issue is emacs auto-formatting hooks.

Copy link
Contributor

@vchudnov-g vchudnov-g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Most comments are minor.

gapic/utils/code.py Outdated Show resolved Hide resolved
gapic/generator/generator.py Outdated Show resolved Hide resolved
gapic/samplegen/samplegen.py Outdated Show resolved Hide resolved
gapic/samplegen/samplegen.py Outdated Show resolved Hide resolved
gapic/samplegen/samplegen.py Outdated Show resolved Hide resolved
gapic/templates/examples/feature_fragments.j2 Outdated Show resolved Hide resolved
tests/unit/samplegen/test_integration.py Show resolved Hide resolved
tests/unit/samplegen/test_samplegen.py Show resolved Hide resolved
tests/unit/samplegen/test_template.py Outdated Show resolved Hide resolved
tests/unit/samplegen/test_template.py Outdated Show resolved Hide resolved
Replace more manually newlined strings with triple-quoted strings
Copy link
Contributor

@lukesneeringer lukesneeringer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks pretty good! I am on vacation still, but figured you might want a drive-by review.

gapic/generator/generator.py Outdated Show resolved Hide resolved
gapic/samplegen/samplegen.py Outdated Show resolved Hide resolved
gapic/samplegen/samplegen.py Outdated Show resolved Hide resolved
naming=naming,
prior_protos=prior_protos or {},
).proto
file_to_generate=file_to_generate,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What auto-formatter are you using? Perhaps we should settle on one and reify it in CI.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

autopep8, part of my python-mode-hook.
Opened issue #142 as a feature task for CI. Do you have strong feelings on autopep8 vs black or something else? My vote is black because there are no choices to make.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not willing to use black because it generates non-PEP-8 compliant code. autopep8 is fine with me.

Copy link
Contributor Author

@software-dov software-dov Jul 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of what I've read about black indicates that its output is PEP-8 compliant. What parts does it violate? Just curious.

gapic/templates/examples/feature_fragments.j2 Outdated Show resolved Hide resolved
gapic/templates/examples/feature_fragments.j2 Outdated Show resolved Hide resolved
gapic/templates/examples/feature_fragments.j2 Outdated Show resolved Hide resolved
gapic/utils/code.py Outdated Show resolved Hide resolved
Change the order of the parameters in 'partition'
Remove license and copyright information from the sample template
s/resp_to_response/coerce_variable_name/g
Add the python package name in the pip install comment in the sample header
Copy link
Contributor

@vchudnov-g vchudnov-g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly minor comments, but some highlights:

  • License: I suggest a placeholder so we can verify things work until we implement reading the license from a file (and have a test for that new functionality)
  • Don't need the *_core region tags for now (yes, that's a recent change)

I agree that making wholesale formatting changes, especially if they touch code not already under review, is better left to a subsequent PR.

gapic/samplegen/samplegen.py Outdated Show resolved Hide resolved
@@ -92,6 +103,20 @@ class InvalidRequestSetup(SampleError):
pass


def coerce_variable_name(s):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the intent of this is to remain specific to replacing $resp (ie, it only takes one argument), I suggest renaming to coerce_response_name

Also, is the intent for the signature above and logic below to grow to accommodate the extra cases where the target is not "response"? (eg for paged methods, etc.)

Copy link
Contributor Author

@software-dov software-dov Jul 10, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand your question. The ideal solution to the problem this functionlet/filter is trying to solve would be

sed -i "s/\$resp/response/g" sample_file.py

Handling it in the validation code is fiddly, actually using sed in the pipeline won't work and doing a big string replace in python code that invokes rendering feels like a hack, so the substitution is punted to the template.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The question was referring to the fact that sometimes we may choose to have $resp have a slightly different name, eg here and here

I think it's fine for each µgen to decide independently whether or not to always $respresponse in these cases or use an alternate name, though the set of alternate names should be small across languages to minimize forbidden user-provided identifiers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The consistency that I aimed for in the calling-form handling template code is that response is always the most interesting top level proto response, i.e. it's the repeated element in paginated or streamed return values, it's the result of waiting on a promise, and so forth.

gapic/samplegen/samplegen.py Show resolved Hide resolved
gapic/templates/examples/sample.py.j2 Outdated Show resolved Hide resolved
gapic/templates/examples/feature_fragments.j2 Show resolved Hide resolved
gapic/templates/examples/feature_fragments.j2 Outdated Show resolved Hide resolved
gapic/utils/code.py Outdated Show resolved Hide resolved
Integrate minor review comments:

Remove _core comment in rendered output
s/coerce_variable_name/coerce_response_name
Add back the license and copyright headers with dummy values
gapic/samplegen/samplegen.py Show resolved Hide resolved
tests/unit/samplegen/test_template.py Show resolved Hide resolved
@@ -92,6 +103,20 @@ class InvalidRequestSetup(SampleError):
pass


def coerce_variable_name(s):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The question was referring to the fact that sometimes we may choose to have $resp have a slightly different name, eg here and here

I think it's fine for each µgen to decide independently whether or not to always $respresponse in these cases or use an alternate name, though the set of alternate names should be small across languages to minimize forbidden user-provided identifiers.

@software-dov software-dov merged commit 6a28387 into googleapis:master Jul 11, 2019
@software-dov software-dov deleted the samplegen-partII branch July 11, 2019 18:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants