Skip to content

feat: lazy imports gapic v1#17673

Open
hebaalazzeh wants to merge 7 commits into
mainfrom
feature/lazy-imports-gapic-v1
Open

feat: lazy imports gapic v1#17673
hebaalazzeh wants to merge 7 commits into
mainfrom
feature/lazy-imports-gapic-v1

Conversation

@hebaalazzeh

@hebaalazzeh hebaalazzeh commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Description

This PR initiates the rollout of our PEP 0810 lazy-loading architecture to google-api-core, starting with the gapic_v1 module.

By applying Python 3.15's native explicit lazy imports, we defer the eager execution of inner modules (like method.py and config.py) which recursively pull in massive third-party C-extensions like grpcio. This acts as the first step in addressing the high initialization latency and peak memory footprints we're seeing on Serverless cold-starts.

Older Python runtimes (3.14 and below) safely ignore the __lazy_modules__ set and fall back to standard eager execution, meaning this introduces zero backwards compatibility risk.

(Note: We use fully-qualified, hardcoded module strings rather than dynamic __name__ injection, and provide explicit type annotations (Set[str]), to ensure static analysis tools and type checkers can safely infer the proxies without complaints).

Related Documents

Note: This PR was kept intentionally small for review speed. We will be executing an iterative rollout, extending this pattern to other heavy modules like exceptions and operations_v1 in fast follow-up PRs once this structural pattern is approved.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds an import_profile test target to the CI script and attempts to implement lazy loading for gapic_v1 submodules using Python 3.15's __lazy_modules__ feature. However, a critical issue was identified: placing the imports inside a conditional block (sys.version_info < (3, 15)) prevents them from being executed at all on Python 3.15+, which will lead to AttributeErrors at runtime. To fix this, the imports should remain at the top level so that Python 3.15+ can lazily load them while older versions load them eagerly.

Comment thread packages/google-api-core/google/api_core/gapic_v1/__init__.py Outdated
@hebaalazzeh hebaalazzeh force-pushed the feature/lazy-imports-gapic-v1 branch from 2439765 to 603236b Compare July 8, 2026 19:49
@hebaalazzeh hebaalazzeh marked this pull request as ready for review July 8, 2026 19:49
@hebaalazzeh hebaalazzeh requested a review from a team as a code owner July 8, 2026 19:49

@parthea parthea left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the feedback from #17679, please can you also add tests?

@parthea

parthea commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

I'm going to switch this to draft until tests are added. Please mark this as ready for review once #17673 (review) is addressed

@parthea parthea marked this pull request as draft July 13, 2026 17:52
@hebaalazzeh

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements PEP 0810 explicit lazy imports for Python 3.15+ in the gapic_v1 package and adds unit tests to verify both lazy and fallback eager import behaviors. Feedback points out a critical inconsistency when grpc is not installed: on Python 3.15+, the deferred imports prevent ImportError from being caught eagerly, resulting in missing dependencies being added to __all__ and causing runtime failures during wildcard imports. The reviewer suggests using importlib.util.find_spec to conditionally define lazy modules and imports based on the presence of grpc.

Comment thread packages/google-api-core/google/api_core/gapic_v1/__init__.py Outdated
@hebaalazzeh

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request attempts to implement lazy imports in gapic_v1 using a hypothetical Python 3.15 feature (__lazy_modules__ / PEP 810), falling back to eager imports on older versions. However, because this feature and PEP do not exist, the implementation currently results in eager imports on all Python versions. The reviewer recommends implementing standard lazy loading using PEP 562 (__getattr__ and __dir__), which is compatible with all supported Python versions (3.7+). This approach would also allow the accompanying unit tests to run and verify lazy loading behavior across all environments instead of being skipped on versions below 3.15.

Comment thread packages/google-api-core/google/api_core/gapic_v1/__init__.py
Comment thread packages/google-api-core/tests/unit/gapic/test_lazy_imports.py
@hebaalazzeh hebaalazzeh marked this pull request as ready for review July 13, 2026 18:59
Comment on lines +41 to +48
from google.api_core.gapic_v1 import client_info # noqa: E402
from google.api_core.gapic_v1 import routing_header # noqa: E402

if _has_grpc:
from google.api_core.gapic_v1 import config # noqa: F401
from google.api_core.gapic_v1 import config_async # noqa: F401
from google.api_core.gapic_v1 import method # noqa: F401
from google.api_core.gapic_v1 import method_async # noqa: F401

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need these # noqa: E402 and # noqa: F401?


import pytest

SCRIPT_PYTHON_315 = """

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a better way to test this instead of using this script within a test file? Do we even want to test this?

Comment on lines +26 to +62
__lazy_modules__: Set[str] = {
"google.api_core.gapic_v1.client_info",
"google.api_core.gapic_v1.routing_header",
}

if _has_grpc:
__lazy_modules__.update(
{
"google.api_core.gapic_v1.config",
"google.api_core.gapic_v1.config_async",
"google.api_core.gapic_v1.method",
"google.api_core.gapic_v1.method_async",
}
)

from google.api_core.gapic_v1 import client_info # noqa: E402
from google.api_core.gapic_v1 import routing_header # noqa: E402

if _has_grpc:
from google.api_core.gapic_v1 import config # noqa: F401
from google.api_core.gapic_v1 import config_async # noqa: F401
from google.api_core.gapic_v1 import method # noqa: F401
from google.api_core.gapic_v1 import method_async # noqa: F401

__all__ = [
"client_info",
"routing_header",
"config",
"config_async",
"method",
"method_async",
]
else:
__all__ = [
"client_info",
"routing_header",
]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
__lazy_modules__: Set[str] = {
"google.api_core.gapic_v1.client_info",
"google.api_core.gapic_v1.routing_header",
}
if _has_grpc:
__lazy_modules__.update(
{
"google.api_core.gapic_v1.config",
"google.api_core.gapic_v1.config_async",
"google.api_core.gapic_v1.method",
"google.api_core.gapic_v1.method_async",
}
)
from google.api_core.gapic_v1 import client_info # noqa: E402
from google.api_core.gapic_v1 import routing_header # noqa: E402
if _has_grpc:
from google.api_core.gapic_v1 import config # noqa: F401
from google.api_core.gapic_v1 import config_async # noqa: F401
from google.api_core.gapic_v1 import method # noqa: F401
from google.api_core.gapic_v1 import method_async # noqa: F401
__all__ = [
"client_info",
"routing_header",
"config",
"config_async",
"method",
"method_async",
]
else:
__all__ = [
"client_info",
"routing_header",
]
__lazy_modules__: Set[str] = {
"google.api_core.gapic_v1.client_info",
"google.api_core.gapic_v1.routing_header",
}
__all__ = ["client_info", "routing_header"]
if _has_grpc:
__lazy_modules__.update({
"google.api_core.gapic_v1.config",
"google.api_core.gapic_v1.config_async",
"google.api_core.gapic_v1.method",
"google.api_core.gapic_v1.method_async",
})
from google.api_core.gapic_v1 import client_info # noqa: E402
from google.api_core.gapic_v1 import routing_header # noqa: E402
if _has_grpc:
from google.api_core.gapic_v1 import config # noqa: F401
from google.api_core.gapic_v1 import config_async # noqa: F401
from google.api_core.gapic_v1 import method # noqa: F401
from google.api_core.gapic_v1 import method_async # noqa: F401
__all__.extend(["config", "config_async", "method", "method_async"])

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants