Charge objects from stripe events don't deserialize due to missing "refunds" field (on api version 2022-11-15)

[eric-seppanen]

Describe the bug

I am testing webhook events, and find that charge.succeeded events don't deserialize into the Charge struct.

For example, here's a charge.succeeded event sent by the stripe trigger cli. If I try to deserialize the data object, I get an error missing field refunds ``. But the stripe documentation for the charge object says that the "refunds" field is not included by default, so I'm worried that this may be a bug?

{
  "id": "evt_3MgvHHFf8peiXbpD0hOJKLK4",
  "object": "event",
  "api_version": "2022-11-15",
  "created": 1677698728,
  "data": {
    "object": {
      "id": "ch_3MgvHHFf8peiXbpD0g9W6JSk",
      "object": "charge",
      "amount": 3000,
      "amount_captured": 3000,
      "amount_refunded": 0,
      "application": null,
      "application_fee": null,
      "application_fee_amount": null,
      "balance_transaction": "txn_3MgvHHFf8peiXbpD0sJFcWWw",
      "billing_details": {
        "address": {
          "city": null,
          "country": null,
          "line1": null,
          "line2": null,
          "postal_code": null,
          "state": null
        },
        "email": "stripe@example.com",
        "name": null,
        "phone": null
      },
      "calculated_statement_descriptor": "Stripe",
      "captured": true,
      "created": 1677698728,
      "currency": "usd",
      "customer": null,
      "description": null,
      "destination": null,
      "dispute": null,
      "disputed": false,
      "failure_balance_transaction": null,
      "failure_code": null,
      "failure_message": null,
      "fraud_details": {
      },
      "invoice": null,
      "livemode": false,
      "metadata": {
      },
      "on_behalf_of": null,
      "order": null,
      "outcome": {
        "network_status": "approved_by_network",
        "reason": null,
        "risk_level": "normal",
        "risk_score": 48,
        "seller_message": "Payment complete.",
        "type": "authorized"
      },
      "paid": true,
      "payment_intent": "pi_3MgvHHFf8peiXbpD0k3ib0Id",
      "payment_method": "pm_1MgvHHFf8peiXbpDSSoXdZlo",
      "payment_method_details": {
        "card": {
          "brand": "visa",
          "checks": {
            "address_line1_check": null,
            "address_postal_code_check": null,
            "cvc_check": null
          },
          "country": "US",
          "exp_month": 3,
          "exp_year": 2024,
          "fingerprint": "lhLy4CfJB9W5Ytp5",
          "funding": "credit",
          "installments": null,
          "last4": "4242",
          "mandate": null,
          "network": "visa",
          "three_d_secure": null,
          "wallet": null
        },
        "type": "card"
      },
      "receipt_email": null,
      "receipt_number": null,
      "receipt_url": "https://pay.stripe.com/receipts/payment...truncated",
      "refunded": false,
      "review": null,
      "shipping": {
        "address": {
          "city": "San Francisco",
          "country": "US",
          "line1": "510 Townsend St",
          "line2": null,
          "postal_code": "94103",
          "state": "CA"
        },
        "carrier": null,
        "name": "Jenny Rosen",
        "phone": null,
        "tracking_number": null
      },
      "source": null,
      "source_transfer": null,
      "statement_descriptor": null,
      "statement_descriptor_suffix": null,
      "status": "succeeded",
      "transfer_data": null,
      "transfer_group": null
    }
  },
  "livemode": false,
  "pending_webhooks": 2,
  "request": {
    "id": "req_K7HIsdYUpt3d7i",
    "idempotency_key": "271c87da-268a-423d-8a61-d77740de61dc"
  },
  "type": "charge.succeeded"
}

To Reproduce

I encountered this while trying to get events decoding (#344) working. I have a modified version of the Event struct which seems to be almost working, and I am generating test events with the stripe cli, using stripe trigger checkout.session.completed.

That's a lot of tedious setup, so maybe it would be easier to just reproduce with this test (I've extracted the "charge" field from the event shown above):

#[test]
fn test_charge_decoding() {
    let test_charge = r#"{
            "id": "ch_3MgvHHFf8peiXbpD0g9W6JSk",
            "object": "charge",
            "amount": 3000,
            "amount_captured": 3000,
            "amount_refunded": 0,
            "application": null,
            "application_fee": null,
            "application_fee_amount": null,
            "balance_transaction": "txn_3MgvHHFf8peiXbpD0sJFcWWw",
            "billing_details": {
              "address": {
                "city": null,
                "country": null,
                "line1": null,
                "line2": null,
                "postal_code": null,
                "state": null
              },
              "email": "stripe@example.com",
              "name": null,
              "phone": null
            },
            "calculated_statement_descriptor": "Stripe",
            "captured": true,
            "created": 1677698728,
            "currency": "usd",
            "customer": null,
            "description": null,
            "destination": null,
            "dispute": null,
            "disputed": false,
            "failure_balance_transaction": null,
            "failure_code": null,
            "failure_message": null,
            "fraud_details": {
            },
            "invoice": null,
            "livemode": false,
            "metadata": {
            },
            "on_behalf_of": null,
            "order": null,
            "outcome": {
              "network_status": "approved_by_network",
              "reason": null,
              "risk_level": "normal",
              "risk_score": 48,
              "seller_message": "Payment complete.",
              "type": "authorized"
            },
            "paid": true,
            "payment_intent": "pi_3MgvHHFf8peiXbpD0k3ib0Id",
            "payment_method": "pm_1MgvHHFf8peiXbpDSSoXdZlo",
            "payment_method_details": {
              "card": {
                "brand": "visa",
                "checks": {
                  "address_line1_check": null,
                  "address_postal_code_check": null,
                  "cvc_check": null
                },
                "country": "US",
                "exp_month": 3,
                "exp_year": 2024,
                "fingerprint": "lhLy4CfJB9W5Ytp5",
                "funding": "credit",
                "installments": null,
                "last4": "4242",
                "mandate": null,
                "network": "visa",
                "three_d_secure": null,
                "wallet": null
              },
              "type": "card"
            },
            "receipt_email": null,
            "receipt_number": null,
            "receipt_url": "https://pay.stripe.com/receipts/payment...truncated",
            "refunded": false,
            "review": null,
            "shipping": {
              "address": {
                "city": "San Francisco",
                "country": "US",
                "line1": "510 Townsend St",
                "line2": null,
                "postal_code": "94103",
                "state": "CA"
              },
              "carrier": null,
              "name": "Jenny Rosen",
              "phone": null,
              "tracking_number": null
            },
            "source": null,
            "source_transfer": null,
            "statement_descriptor": null,
            "statement_descriptor_suffix": null,
            "status": "succeeded",
            "transfer_data": null,
            "transfer_group": null
          }"#;

    serde_json::from_str::<stripe::Charge>(test_charge).unwrap();
}

Expected behavior

If the refunds field is optional, then a serialized charge sent by stripe should deserialize successfully.

Code snippets

No response

OS

Linux

Rust version

1.66.1

Library version

async-stripe v0.18.2

API version

2022-11-15

Additional context

No response

[arlyon]

Hello! Thanks for reporting this. We recently moved from using the regular openapi spec to the sdk version, which is intended for library codegen usage, and for which the openapi differs slightly. I believe the docs are generated from the regular spec, which is why the fields is listed as optional on their docs.

To verify consider these commands:

> curl https://raw.githubusercontent.com/stripe/openapi/master/openapi/spec3.json | jq '.components.schemas.charge.required'
[
  "amount",
  "amount_captured",
  "amount_refunded",
  "billing_details",
  "captured",
  "created",
  "currency",
  "disputed",
  "id",
  "livemode",
  "metadata",
  "object",
  "paid",
  "refunded",
  "status"
]
> curl https://raw.githubusercontent.com/stripe/openapi/master/openapi/spec3.sdk.json | jq '.components.schemas.charge.required'
[
  "amount",
  "amount_captured",
  "amount_refunded",
  "application",
  "application_fee",
  "application_fee_amount",
  "balance_transaction",
  "billing_details",
  "calculated_statement_descriptor",
  "captured",
  "created",
  "currency",
  "customer",
  "description",
  "destination",
  "dispute",
  "disputed",
  "failure_balance_transaction",
  "failure_code",
  "failure_message",
  "fraud_details",
  "id",
  "invoice",
  "livemode",
  "metadata",
  "object",
  "on_behalf_of",
  "outcome",
  "paid",
  "payment_intent",
  "payment_method",
  "payment_method_details",
  "receipt_email",
  "receipt_number",
  "receipt_url",
  "refunded",
  "refunds",
  "review",
  "shipping",
  "source",
  "source_transfer",
  "statement_descriptor",
  "statement_descriptor_suffix",
  "status",
  "transfer_data",
  "transfer_group"
]

So, what is the solution? I'm not sure in this case to be honest. I think the library is wrong, but does that mean we want to abandon the sdk definition entirely? Or only use it to infer the required fields? I am not sure about that.

[eric-seppanen]

Part of my problem may be that the stripe listen CLI doesn't allow me to set the API version, so I can't force a downgrade to 2020-08-27 while testing.

I will try to test this with a production webhook to see if pinning the version changes the webhook payload.

[eric-seppanen]

I confirmed that when I set up a "real" webhook using params.api_version = Some(stripe::ApiVersion::V2020_08_27) then I get events with Charge objects that can be deserialized (they do contain a refunds field).

So this must be something that changed between api version 2020-08-27 and 2022-11-15.

This is likely to bite other users in the future, because Stripe's documented way of testing webhooks doesn't permit you to set an arbitrary API version; you can only use your account default or the latest. (Some stripe CLI commands do support it, but not stripe listen, which is the one that matters here.) See also: stripe/stripe-cli#213

[eric-seppanen]

Since my problem is due to an api version mismatch (compounded by a limitation in stripe-cli), should this be closed?

I hope that this library will pick up the newer Stripe API version(s) at some point, but this bug (refunds field missing) is probably invalid as long as the crate policy is "we only support 2020-08-27".

[arlyon]

Hi! I would like to leave this open for now. I think if you are encountering this, then others might and we should address it properly. Thanks for taking the time to explore (and post) workarounds.

[seanpianka]

I suppose I'll bump this as I'm using 2020-08-27 and am seeing the JSON deserialization error when I query/retrieve a specific charge by its ID. Perhaps I'm reading this thread wrong, but should 2020-08-27 work in this case?

[eric-seppanen]

@seanpianka are you explicitly setting the API version when setting up the webhook? This is what I'm doing, for example:

let mut params = CreateWebhookEndpoint::new(events, webhook_url.as_str());
// The async-stripe crate requires an older api version; otherwise some events
// don't deserialize correctly.
params.api_version = Some(ApiVersion::V2020_08_27);
params.description = Some("name of my server");

let endpoint = WebhookEndpoint::create(&client, params).await?;

edit: sorry, failed to read the message correctly. If you're using e.g. Charge::retrieve, then the API version would have been set automatically and the data should deserialize cleanly.

The issue I described here is caused by an API mismatch when using webhooks, so perhaps we should move this new problem discussion to another issue.

[arlyon]

I am going to close this. Thanks for taking the time to publicize your findings.