Error when creating SubscriptionItems for metered plans

[jameschristopher]

Python==3.7.1
Django==2.1.7
djstripe==2.0.0

The issue seems to arise when processing the customer.subscription.created webhook callback. dj-stripe attempts to create the SubscriptionItems listed in the Stripe response. The Stripe response for Subscription items that are on metered plans does not contain the quantity attribute, thus when dj-stripe ingests that data it fails to create a SubscriptionItem as the quantity field does not accept null values nor does it have a default. Here is a snippet copied from my Stripe logs of what was POSTed to my webhook endpoint:

"items": {
      "object": "list",
      "data": [
        {
          "id": "si_*************",
          "object": "subscription_item",
          "billing_thresholds": null,
          "created": 1552418209,
          "metadata": {
          },
          "plan": {
            "id": "plan_*************",
            "object": "plan",
            "active": true,
            "aggregate_usage": "sum",
            "amount": 100,
            "billing_scheme": "per_unit",
            "created": 1552375552,
            "currency": "usd",
            "interval": "day",
            "interval_count": 1,
            "livemode": false,
            "metadata": {
            },
            "nickname": "Per completing record",
            "product": "prod_*************",
            "tiers": null,
            "tiers_mode": null,
            "transform_usage": null,
            "trial_period_days": null,
            "usage_type": "metered"
          },
          "subscription": "sub_*************"
        },
        {
          "id": "si_*************",
          "object": "subscription_item",
          "billing_thresholds": null,
          "created": 1552418209,
          "metadata": {
          },
          "plan": {
            "id": "plan_*************",
            "object": "plan",
            "active": true,
            "aggregate_usage": null,
            "amount": 200,
            "billing_scheme": "per_unit",
            "created": 1552369666,
            "currency": "usd",
            "interval": "day",
            "interval_count": 1,
            "livemode": false,
            "metadata": {
            },
            "nickname": "Per worker",
            "product": "prod_*************",
            "tiers": null,
            "tiers_mode": null,
            "transform_usage": null,
            "trial_period_days": null,
            "usage_type": "licensed"
          },
          "quantity": 0,
          "subscription": "sub_*************"
        }
      ],
      "has_more": false,
      "total_count": 2,
      "url": "/v1/subscription_items?subscription=sub_*************"
    },

You'll see that for the first (metered plan) SubscriptionItem the quantity attribute is missing. I believe this is causing the error I'm seeing:

psycopg2.IntegrityError: null value in column "quantity" violates not-null constraint

Not entirely sure what the remedy is here, Stripe documentation for SubscriptionItem suggests that this attribute should be a positive integer or 0, but doesn't specify that the attribute is always present. dj-stripe may need to be modified to handle this case?

Everything works fine if I do not include the metered plan in the subscription.

Has anyone else dealt with this issue before?

[blackboxoperations]

Note: the code below uses the Stripe API alongside dj-stripe, but there is likely a way to do this using just the dj-stripe code/objects as of v2. @therefromhere may be able to tell us, I don't know off the top off my head.

Also, I'm just plucking off the first plan from the set Stripe Plans in the example below, be sure to set the subscription to the plan the user actually selected in the subscription form on your end before attempting to use this code. I'm also setting the trial_end parameter, in the event you've a trialing period and just haven't mentioned it yet - this is how you've ensure the trial begins - remove trial_end if the subscription is meant to be paid for, and thus the charge created, immediately upon successful instantiation of the subscription itself.

token = request.POST['stripe_token']
stripe_plans = stripe.Plan.list()
trial_start = datetime.datetime.fromtimestamp(int(time.time())) 
trial_end = int(trial_start.timestamp()) + 2629743 #datetime.timedelta(days=30)

customer = stripe.Customer.create(
    email='john@doe.com',
    source=token
)

subscription = stripe.Subscription.create(
    customer=customer.id,
    items=[{
        'plan': stripe_plans[0]['id'], 
        'quantity': 1
    }],
    trial_end=int(trial_end)
)

# may be unnecessary or redundant as of dj-stripe v2, waiting on @therefromhere to advise/confirm
djstripe_customer = Customer.sync_from_stripe_data(customer)
djstripe_customer.subscriber = user
djstripe_customer.save()

So the question becomes: You're setting a quantity when creating the subscription, correct?

[jameschristopher]

Yes, I am setting the quantity for the first subscription item to 0 when I create the subscription. It should be noted that I created the subscription from the Stripe dashboard. So this error is purely happening during the webhooks event callback.

[therefromhere]

I've not looked into this deeply but it sounds like this could be a similar case to #773 - fudge the data on the stable 2.0 branch (which we won't allow migrations) using _manipulate_stripe_object_hook and add a migration on master that makes SubscriptionItem.quantity nullable (which would be part of a future 2.1 release).

I've a feeling there's a different PR which was more similar but I can't find it right now.

[blackboxoperations]

@jameschristopher - check your setup. I'm not sure why you want to instantiate a tiered subscription with a quantity of 0, but for gits and shiggles I went in and subscribed a customer to a tiered subscription with quantity=0, and the events/webhooks and updates to the application all went through and look fine on my end.

Stripe API Version: 2018-11-08
customer.subscription.created webhook
customer.subscription.created event object contained in the webhook

Webhook is valid, processed. Event logged. If there's something wrong here, I can't recreate it on my end. Going to look at #773 and see if that sheds some light on this matter.

[jameschristopher]

Well the issue isn’t with tiered plans it’s with metered plans.

[jameschristopher]

@therefromhere I’ll give that a shot and update this thread

[therefromhere]

Cool. Note that master and stable/2.0 are currently in sync since we didn't add any post-2.0 release breaking changes yet (migrations etc), so this would be a point where they'll diverge.

[jameschristopher]

It seems that _manipulate_stripe_object_hook is not getting called during this issue. I was able to resolve the issue by fudging the _stripe_object_to_subscription_items method using .setdefault right before it calls the _get_or_create_from_stripe_object class method.

@classmethod
def _stripe_object_to_subscription_items(cls, target_cls, data, subscription):
	"""
	Retrieves SubscriptionItems for a subscription.

	If the subscription item doesn't exist already then it is created.

	:param target_cls: The target class to instantiate per invoice item.
	:type target_cls: ``SubscriptionItem``
	:param data: The data dictionary received from the Stripe API.
	:type data: dict
	:param invoice: The invoice object that should hold the invoice items.
	:type invoice: ``djstripe.models.Subscription``
	"""

	items = data.get("items")
	if not items:
	    return []

	subscriptionitems = []
	for item_data in items.get("data", []):
	    item_data.setdefault('quantity', 0)
	    item, _ = target_cls._get_or_create_from_stripe_object(item_data, refetch=False)
	    subscriptionitems.append(item)

	return subscriptionitems

I also added a similar line to _manipulate_stripe_object_hook as suggested.

[therefromhere]

@jameschristopher I'm not sure why _manipulate_stripe_object_hook wasn't firing for you, I've gone ahead and implemented the fix as discussed as #866.