Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Autogen: fix coverage landmines. #8174

Closed
tseaver opened this issue May 25, 2019 · 2 comments
Closed

Autogen: fix coverage landmines. #8174

tseaver opened this issue May 25, 2019 · 2 comments
Labels
api: automl Issues related to the AutoML API. api: bigquerydatatransfer Issues related to the BigQuery Data Transfer Service API. api: bigtable Issues related to the Bigtable API. api: cloudasset Issues related to the Cloud Asset Inventory API. api: clouderrorreporting Issues related to the Error Reporting API. api: cloudiot Issues related to the IoT Core API. api: cloudkms Issues related to the Cloud Key Management Service API. api: cloudscheduler Issues related to the Cloud Scheduler API. api: cloudtasks Issues related to the Cloud Tasks API. api: cloudtrace Issues related to the Cloud Trace API. api: container Issues related to the Kubernetes Engine API API. api: datacatalog Issues related to the Data Catalog API. api: datalabeling Issues related to the AI Platform Data Labeling Service API. api: dataproc Issues related to the Dataproc API. api: datastore Issues related to the Datastore API. api: dlp Issues related to the Sensitive Data Protection API. api: firestore Issues related to the Firestore API. api: iam Issues related to the Identity and Access Management API. api: irm api: language Issues related to the Cloud Natural Language API API. api: logging Issues related to the Cloud Logging API. api: monitoring Issues related to the Cloud Monitoring API. api: oslogin Issues related to the Cloud OS Login API API. api: phishingprotection Issues related to the Phishing Protection API. api: pubsub Issues related to the Pub/Sub API. api: redis Issues related to the Memorystore for Redis API. api: securitycenter Issues related to the Security Command Center API. api: spanner Issues related to the Spanner API. api: speech Issues related to the Speech-to-Text API. api: talent api: texttospeech Issues related to the Text-to-Speech API. api: translation Issues related to the Cloud Translation API API. api: videointelligence Issues related to the Video Intelligence API API. api: vision Issues related to the Cloud Vision API. api: webrisk Issues related to the Web Risk API. api: websecurityscanner Issues related to the Web Security Scanner API. type: process A process-related concern. May include testing, release, or the like.

Comments

@tseaver
Copy link
Contributor

tseaver commented May 25, 2019

The unit_cov_level numbers in the various noxfile.py files are landmines (e.g., PR #8147 adds a # pragma: NO COVER and actually trips the check, because the total number of lines checked in BQDT drops!) The numbers don't represent any real goals (why 79% in one library or 95% in another?), but are merely "high water marks." The real goal is 100% coverage across all unit test runs, which we don't hit for many of the autogen-only libraries.

To address the first issue, I would argue that we should just drop the coverage checks inside the unit / default sessions.

To address the second issue, we need to apply a change like the one in #7413 which actually tests the "shim" modules for the autogen-only libraries (and excludes the namespace package files).

Both of these changes need to happen in the generator / templates (dropping the --cov-fail-under bit in the unit / default session, tweaking the .coveragerc, adding a testcase module for each generated shim module).

@tseaver tseaver added api: datastore Issues related to the Datastore API. api: pubsub Issues related to the Pub/Sub API. api: bigtable Issues related to the Bigtable API. api: logging Issues related to the Cloud Logging API. api: vision Issues related to the Cloud Vision API. api: monitoring Issues related to the Cloud Monitoring API. api: translation Issues related to the Cloud Translation API API. api: speech Issues related to the Speech-to-Text API. api: spanner Issues related to the Spanner API. api: clouderrorreporting Issues related to the Error Reporting API. api: language Issues related to the Cloud Natural Language API API. api: cloudtrace Issues related to the Cloud Trace API. api: videointelligence Issues related to the Video Intelligence API API. api: firestore Issues related to the Firestore API. type: process A process-related concern. May include testing, release, or the like. api: dataproc Issues related to the Dataproc API. api: container Issues related to the Kubernetes Engine API API. api: bigquerydatatransfer Issues related to the BigQuery Data Transfer Service API. api: dlp Issues related to the Sensitive Data Protection API. api: texttospeech Issues related to the Text-to-Speech API. api: redis Issues related to the Memorystore for Redis API. api: cloudiot Issues related to the IoT Core API. api: cloudtasks Issues related to the Cloud Tasks API. api: automl Issues related to the AutoML API. api: cloudkms Issues related to the Cloud Key Management Service API. api: cloudasset Issues related to the Cloud Asset Inventory API. api: oslogin Issues related to the Cloud OS Login API API. api: websecurityscanner Issues related to the Web Security Scanner API. api:securitycenter labels May 25, 2019
@tseaver tseaver added api: iam Issues related to the Identity and Access Management API. api: irm api: talent api: webrisk Issues related to the Web Risk API. api: datacatalog Issues related to the Data Catalog API. api: phishingprotection Issues related to the Phishing Protection API. api: datalabeling Issues related to the AI Platform Data Labeling Service API. labels May 25, 2019
@tseaver
Copy link
Contributor Author

tseaver commented May 28, 2019

@lukesneeringer asserts that the new gapic-generator-python will produce fully-tested code, including the shim modules.

@busunkim96
Copy link
Contributor

busunkim96 commented Nov 3, 2021

The generator produces tests which cover 100% of the autogen code. If coverage falls below 100% an issue should be opened: https://github.com/googleapis/gapic-generator-python/issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: automl Issues related to the AutoML API. api: bigquerydatatransfer Issues related to the BigQuery Data Transfer Service API. api: bigtable Issues related to the Bigtable API. api: cloudasset Issues related to the Cloud Asset Inventory API. api: clouderrorreporting Issues related to the Error Reporting API. api: cloudiot Issues related to the IoT Core API. api: cloudkms Issues related to the Cloud Key Management Service API. api: cloudscheduler Issues related to the Cloud Scheduler API. api: cloudtasks Issues related to the Cloud Tasks API. api: cloudtrace Issues related to the Cloud Trace API. api: container Issues related to the Kubernetes Engine API API. api: datacatalog Issues related to the Data Catalog API. api: datalabeling Issues related to the AI Platform Data Labeling Service API. api: dataproc Issues related to the Dataproc API. api: datastore Issues related to the Datastore API. api: dlp Issues related to the Sensitive Data Protection API. api: firestore Issues related to the Firestore API. api: iam Issues related to the Identity and Access Management API. api: irm api: language Issues related to the Cloud Natural Language API API. api: logging Issues related to the Cloud Logging API. api: monitoring Issues related to the Cloud Monitoring API. api: oslogin Issues related to the Cloud OS Login API API. api: phishingprotection Issues related to the Phishing Protection API. api: pubsub Issues related to the Pub/Sub API. api: redis Issues related to the Memorystore for Redis API. api: securitycenter Issues related to the Security Command Center API. api: spanner Issues related to the Spanner API. api: speech Issues related to the Speech-to-Text API. api: talent api: texttospeech Issues related to the Text-to-Speech API. api: translation Issues related to the Cloud Translation API API. api: videointelligence Issues related to the Video Intelligence API API. api: vision Issues related to the Cloud Vision API. api: webrisk Issues related to the Web Risk API. api: websecurityscanner Issues related to the Web Security Scanner API. type: process A process-related concern. May include testing, release, or the like.
Projects
None yet
Development

No branches or pull requests

2 participants