Dramatically improve performance by lazily loading most imports

[xavdid-stripe]

Why?

In #1427, users flagged that importing stripe is really slow. We dug into some causes in this comment. This PR dramatically improves import speeds and memory usage, ensuring users only "pay" for what they actually use.

As a result, the initial import is very fast. Subsequent operations will incur slight performance hits since we're doing imports as-needed, but the overall effect is still a dramatic increase in performance.

Stats

tldr: roughly a 4x speed improvement across the board

First, a pure import of the module is much faster:

# before

python -c "print('ok'); import stripe"  0.32s user 0.07s system 65% cpu 0.595 total

# after

python -c "print('ok'); import stripe"  0.06s user 0.02s system 88% cpu 0.087 total

Everything else also got faster as a result. For example, this script (which includes a round trip to the server):

import json
import stripe
from stripe import Account
# from stripe import *

print("should be a class:", Account)
print("should be a method:", Account.list)

from stripe import radar

print("module:", radar)
print("hard module:", stripe.apps)

from stripe import StripeClient

print("class:", StripeClient)
from stripe.params.radar import ValueListRetrieveParams

print("class:", ValueListRetrieveParams)
from stripe.params import radar as RadarParams

print("module:", RadarParams)
from stripe.params import InvoiceVoidInvoiceParams

print("class:", InvoiceVoidInvoiceParams)
from stripe.v2 import ListObject

print("class:", ListObject)
from stripe.v2 import billing

print("module:", billing)
from stripe.apps import SecretService

print("class:", SecretService)

from stripe.params.v2.billing import MeterEventStreamCreateParams

print("class:", MeterEventStreamCreateParams)
from stripe.params.v2 import billing as BillingParams

print("module:", BillingParams)

c = StripeClient("sk_test_123")
print("method:", c.v1.accounts.list)
print("method:", c.v1.accounts.list)

print("method:", c.invoices.list)

print("method:", c.v2.core.event_destinations.list)
print("method:", c.v2.billing.meter_events.create)

print("class:", billing.MeterEvent)

j = stripe.v2.core.EventNotification.from_json(
    json.dumps(
        {
            "type": "v1.billing.meter.error_report_triggered",
            "id": "evt_123",
            "created": "",
            "related_object": {
                "url": "/v1/billing/meter/mer_123",
                "id": "mer_123",
                "type": "meter",
            },
        }
    ),
    c,
)

from stripe.events import (
    V1BillingMeterErrorReportTriggeredEvent,
    V1BillingMeterNoMeterFoundEventNotification,
)

customers = c.v1.customers.list()
print(customers.data[0])

Has improved runtime:

# before

python stripe/main.py  0.98s user 0.54s system 47% cpu 3.186 total

# after

python stripe/main.py  0.24s user 0.07s system 41% cpu 0.734 total

Early user feedback has been very good and we've got good test coverage here, so I'm feeling pretty good.

What?

Fixes fall into a few big categories:

1. move imports that we're just using for types into if TYPE_CHECKING blocks and quote them when used
2. for _service, __init__, & param files, move all imports into a typechecking block and use a module-level __getattr__ (docs) to dynamically resolve imports. This means we're never recursively importing everything anymore
3. In StripeClient, lean heavily on type-only imports
4. in _v1_services.py and _v2_services.py, move all types to annotations and lazily instantiate subservices as needed, instead of all at once
5. Hardcode object type and event type lookups instead of getting the strings from the imported classes themselves.

In many of these cases, we generate a dict that has everything we need to import a module, then write the __getattr__ to try and do the import when prompted. We raise AttributeErrors when the import fails, which Python interprets on our behalf.

We use importlib.import_module, which is a high-level API for programmatically importing things. It caches modules just like static imports do, so subsequent dynamic imports are much faster.

See Also

• Stripe import is very slow #1427
• DEVSDK-2709

Changelog

• move many type imports behind an if TYPE_CHECKING block
• lazily initialize subservices
• add module-level __getattr__ functions to most __init__.py files