ProgrammingError: column djstripe_product.default_price_id does not exist

[agusmakmun]

Describe the bug
I'm having this issue while trying to migrate from 2.4.3 to 2.7.1 or higher.

File /usr/local/lib/python3.8/site-packages/django/db/backends/utils.py:84, in CursorWrapper._execute(self, sql, params, *ignored_wrapper_args)
     82     return self.cursor.execute(sql)
     83 else:
---> 84     return self.cursor.execute(sql, params)

ProgrammingError: column djstripe_product.default_price_id does not exist
LINE 1: ...roduct"."description", "djstripe_product"."name", "djstripe_...

To Reproduce

1. Already had the dj-stripe data in the database
2. Upgrade the packages from 2.4.3 to 2.7.1 or higher.
3. Drop the django_migrations table to fix the migrations issue:

   psql -U <username> <database_name>
   DROP TABLE django_migrations;
   

4. Run the fake migrate command to re-initialize the django_migrationsdjango_migrations: python manage.py migrate --fake
5. Create a management command to sync the product prices & execute it:

"""
sync packages from stripe command.

python manage.py djstripe_sync_packages_from_stripe
"""
from django.core.management.base import BaseCommand

from djstripe.models import Product, Plan, Price


class Command(BaseCommand):
    """Sync packages for product, prices (and plans) from stripe."""

    help = "Sync packages for product, prices (and plans) from stripe."

    def handle(self, *args, **options):
        for product_data in Product.api_list():
            product = Product.sync_from_stripe_data(product_data)
            self.stdout.write(f"Synchronized product {product.id}")

        for price_data in Price.api_list():
            price = Price.sync_from_stripe_data(price_data)
            self.stdout.write(f"Synchronized price {price.id}")

        for plan_data in Plan.api_list():
            plan = Plan.sync_from_stripe_data(plan_data)
            self.stdout.write(f"Synchronized plan {plan.id}")

Software versions

• Dj-Stripe version: 2.7.1 or higher.
• Python version: 3.9
• Django version: 3.2.18
• Stripe API version: v1.19.4
• Database type and version: Postgres

[agusmakmun]

Seems it's need to be harcode to alter the table;

my_project_db=# ALTER TABLE djstripe_product ADD COLUMN default_price_id VARCHAR(255);
ALTER TABLE
my_project_db=# \d djstripe_product;
                                                   Table "public.djstripe_product"
          Column           |           Type           | Collation | Nullable |                        Default
---------------------------+--------------------------+-----------+----------+-------------------------------------------------------
 djstripe_id               | bigint                   |           | not null | nextval('djstripe_product_djstripe_id_seq'::regclass)
 id                        | character varying(255)   |           | not null |
 livemode                  | boolean                  |           |          |
 created                   | timestamp with time zone |           |          |
 metadata                  | jsonb                    |           |          |
 description               | text                     |           |          |
 djstripe_created          | timestamp with time zone |           | not null |
 djstripe_updated          | timestamp with time zone |           | not null |
 name                      | text                     |           | not null |
 type                      | character varying(7)     |           | not null |
 active                    | boolean                  |           |          |
 attributes                | jsonb                    |           |          |
 caption                   | text                     |           | not null |
 deactivate_on             | jsonb                    |           |          |
 images                    | jsonb                    |           |          |
 package_dimensions        | jsonb                    |           |          |
 shippable                 | boolean                  |           |          |
 url                       | character varying(799)   |           |          |
 statement_descriptor      | character varying(22)    |           | not null |
 unit_label                | character varying(12)    |           | not null |
 djstripe_owner_account_id | character varying(255)   |           |          |
 default_price_id          | character varying(255)   |           |          |
Indexes:
    "djstripe_product_pkey" PRIMARY KEY, btree (djstripe_id)
    "djstripe_product_id_key" UNIQUE CONSTRAINT, btree (id)
    "djstripe_product_djstripe_owner_account_id_6c90c2b6" btree (djstripe_owner_account_id)
    "djstripe_product_djstripe_owner_account_id_6c90c2b6_like" btree (djstripe_owner_account_id varchar_pattern_ops)
    "djstripe_product_id_5b4bdbcb_like" btree (id varchar_pattern_ops)
Foreign-key constraints:
    "djstripe_product_djstripe_owner_accou_6c90c2b6_fk_djstripe_" FOREIGN KEY (djstripe_owner_account_id) REFERENCES djstripe_account(id) DEFERRABLE INITIALLY DEFERRED
Referenced by:
    TABLE "djstripe_plan" CONSTRAINT "djstripe_plan_product_id_5773384d_fk_djstripe_product_id" FOREIGN KEY (product_id) REFERENCES djstripe_product(id) DEFERRABLE INITIALLY DEFERRED
    TABLE "djstripe_price" CONSTRAINT "djstripe_price_product_id_9f3d37eb_fk_djstripe_product_id" FOREIGN KEY (product_id) REFERENCES djstripe_product(id) DEFERRABLE INITIALLY DEFERRED

Another issue are come where column djstripe_account.djstripe_owner_account_id does not exist.

my_project_db=# ALTER TABLE djstripe_account ADD COLUMN djstripe_owner_account_id VARCHAR(255);

After that, other issue are comes when doing the sync product prices;

INFO 2024-04-18 14:09:52,391 util 32 281473460199440 [util.py:63] error_code=resource_missing error_message="No such file upload: 'file_1OJ5j9BLJxZz4SVKTd1AVjqs'; a similar object exists in live mode, but a test mode key was used to make this request." error_param=file error_type=invalid_request_error message='Stripe API error received'
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/djstripe/models/base.py", line 670, in _create_from_stripe_object
    instance = cls.stripe_objects.get(id=id_)
  File "/usr/local/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 435, in get
    raise self.model.DoesNotExist(
djstripe.models.core.DoesNotExist: Price matching query does not exist.

According to the new version, the Product must have a default price relations;

dj-stripe/djstripe/models/core.py

Lines 566 to 573 in 5f4734f

	default_price = StripeForeignKey(
	"Price",
	on_delete=models.SET_NULL,
	null=True,
	blank=True,
	related_name="products",
	help_text="The default price this product is associated with.",
	)

So, I need to switch my management command to sync the prices first before product;

"""
sync packages from stripe command.

python manage.py djstripe_sync_packages_from_stripe
"""

from django.core.management.base import BaseCommand

from djstripe.models import Product, Plan, Price


class Command(BaseCommand):
    """Sync packages for product, prices (and plans) from stripe."""

    help = "Sync packages for product, prices (and plans) from stripe."

    def handle(self, *args, **options):
        for price_data in Price.api_list():
            price = Price.sync_from_stripe_data(price_data)
            self.stdout.write(f"Synchronized price {price.id}")

        for product_data in Product.api_list():
            product = Product.sync_from_stripe_data(product_data)
            self.stdout.write(f"Synchronized product {product.id}")

        for plan_data in Plan.api_list():
            plan = Plan.sync_from_stripe_data(plan_data)
            self.stdout.write(f"Synchronized plan {plan.id}")

[agusmakmun]

seems another tables are affecting, and found ProgrammingError issues;

ALTER TABLE djstripe_product ADD COLUMN default_price_id VARCHAR(255);
ALTER TABLE djstripe_account ADD COLUMN djstripe_owner_account_id VARCHAR(255);
ALTER TABLE djstripe_customer ADD COLUMN deleted BOOLEAN DEFAULT FALSE;
ALTER TABLE djstripe_customer ADD COLUMN discount JSONB;

and many more.. hmmm

[agusmakmun]

I just realize that upgrading the dj-stripe version to higher version that too far is not possible.
So, we need to upgrade it slowly.

I made this article how to do it:
https://gist.github.com/agusmakmun/1c7245b3039bdd02e3471e83077cc4b3

[jleclanche]

@agusmakmun thank you for the gist!
Indeed the upgrade path is best taken version-by-version; each dj-stripe version only supports a limited range it can upgrade from.

[agusmakmun]

@jleclanche You're welcome. Maybe we can add it into somewhere in the docs.
So, users who don't have strong understanding of the migrations things can easily follow it.

I'm facing these difficulties since ~3 weeks just for exploring how to upgrade the dj-stripe in correct way without having an issue with old data / migrations. I believe it will save a lot of time for users who facing the same issue. 🙏

[jleclanche]

Yes feel free to PR it into the docs! :)

[jleclanche]

Docs landed