[in_app_purchase] Removed maintaining own cache of transactions

[kinex]

Description

Removed maintaining own cache of transactions as it is not working reliably. It is better to use SKPaymentQueue.transactions where needed. The old logic trying to maintain list of transactions did not work properly causing severe issues when trying to make a purchase and when trying to finish a transaction.

Related changes:

• Removed unnecessary payment validation from addPayment (AppStore won't accept purchasing same product twice, no need to check it here). Changed also the return value of addPayment from bool to void.
• Refactored finishTransaction to finish transactions properly.

This pull request fixes also an issue in the restoreTransactions. It did not call result(nil) so the call never completed.

Related Issues

flutter/flutter#53534
flutter/flutter#57903
flutter/flutter#57356

It is possible this fixes also other reported issues in this plugin as the behavior in the old version is very unstable.

Checklist

Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

• I read the Contributor Guide and followed the process outlined there for submitting PRs.
• My PR includes unit or integration tests for all changed/updated/fixed behaviors (See Contributor Guide).
• All existing and new tests are passing.
• I updated/added relevant documentation (doc comments with ///).
• The analyzer (flutter analyze) does not report any problems on my PR.
• I read and followed the Flutter Style Guide.
• The title of the PR starts with the name of the plugin surrounded by square brackets, e.g. [shared_preferences]
• I updated pubspec.yaml with an appropriate new version according to the pub versioning philosophy.
• I updated CHANGELOG.md to add a description of the change.
• I signed the CLA.
• I am willing to follow-up on review comments in a timely manner.

Breaking Change

Does your PR require plugin users to manually update their apps to accommodate your change?

• Yes, this is a breaking change (please indicate a breaking change in CHANGELOG.md and increment major revision).
• No, this is not a breaking change.

[cyanglaz]

cc @LHLL

[d9media]

Just wondering whether this PR is expected to ship with next release or you recommend applying it manually for now?

[LHLL]

Thanks for the fix, looks like this removed all restrictions for checking transactions. Here are some questions:

1. What happens when two payments for the same auto-renewal product being added to the queue?
2. If user Foo added one transaction for product A and switched to user Bar before the transaction finishes, will this product be delivered to Foo or Bar?

[kinex]

1. According to Apple documentation you will get error "You've already purchased this In-App Purchase but it hasn't been downloaded". Doc says:
   You are getting the "You've already purchased this In-App Purchase but it hasn't been downloaded." message because you did not call SKPaymentQueue's finishTransaction: in your application.
   If you really wanted to validate something yourself, it would be better to use SKPaymentQueue.transactions for the validation and not your own cached list of transactions. But from what I know your own validation is not necessary, StoreKit takes care of it.

2. Interesting question, but whatever happens I don't think that is a problem of this plugin. If we cannot trust that SKPaymentQueue.transactions contains the pending transactions for the current Apple ID, then nothing works properly and we can only blame Apple for it. In this PR I call finishTransaction only for transactions available in the SKPaymentQueue.transactions. The old (current) implementation allows calling finishTransaction for any transactions stored in your cached transactions list so doesn't that sound more like a source of problems (is your cached transaction list reseted if user switches Apple ID?).
   But answering to your question this is my best guess and assumption (I cannot test this): If user switches to a different Apple ID, the SKPaymentQueue.transactions is updated accordingly so the added transaction does not exist in SKPaymentQueue.transactions anymore. Therefore finishTransaction call will not finish the transaction meaning that none of the users will get the product.

[LHLL]

1. I think that's not the right doc for auto-renewal subscription, that only applies to SKDownload which can be attached to consumable, non-consumable products or subscriptions.

2. You point stands well for Apple ID switching due to the fact that Apple will not expose what Apple ID the user is using at the runtime. However, a client app can has its own account system(for example, login with Google, login with Facebook, etc.). Assuming Google Account A purchases SKProduct A, kills the app before it finishes. Google Account B logs in, purchased SKProduct A again and now both transactions become purchased, then Google Account B will get product A twice? Or let me ask in a different way, under what scenario, you want two transactions of the same product in the SKPaymentQueue? Correct me if I am wrong, my understanding is app should have some sort of UI blocking the user when the transaction is still pending right? And if the user didn't finish it before killing the app, then app should asking which Google Account gets this transaction next time when the user launches app instead of letting the user create a new transaction of the same product?

[kinex]

1. You may be right about the doc link, but unfortunately I cannot test this as my Sandbox account is somehow broken. I cannot trust any errors or other behavior happening with my Sandbox account. That's why I had to test also this PR with my end users in production environment.
   Not sure could it be then this other error "Cannot connect to iTunes Store", which is the most common iap related error message I have seen in Crashlytics. This was also the error message users were reporting me when I was using the current release version of this plugin. Anyway, I am 99% sure purchasing the same subscription twice is not possible even if you try it on purpose with invalid code. If it was possible, I would have already seen such a case with my app due to issues in this plugin. Or do you have any evidence such an issue is possible?

2. Sorry I misunderstood your question first. In this question you assume that purchasing the same product twice is possible in iOS? Therefore my comments in 1. answer also this question I guess. So the situation you described is not possible in my opinion.

[LHLL]

"Cannot connect to iTunes Store" normally happens when a user is using the same test account in both sandbox environment and prod environment. But other scenarios such as Apple service is unavailable could also lead to this error.

Having multiple transactions of the same SKProduct is allowed by the StoreKit and that's why we need to prevent it from happening since it's super hard to debug due to fact that Apple will not expose the Apple ID to the third-party apps. The cached transaction array is used to ensure there will be only one transaction of a given product ID at any given time.

[kinex]

What was actually the reason trying to maintain your own cached transaction array instead of using SKPaymentQueue.transactions?

My end users do not use Sandbox environment and it is very hard to believe that Apple service is unavailable every day. So it is some other issue which happens occasionally.

My experience from StoreKit behavior is very different. If you have already added a SKPayment to the queue i.e. the Apple purchase dialog has been shown and user has confirmed the purchase, but you have failed to call finishTransaction for any reason, another attempt to add a SKPayment to the queue will fail and the purchase dialog is NOT shown.

Therefore since the purchase dialog is not shown and user has to always confirm a purchase using that dialog, user cannot make a double purchase. What I don't know for sure is that will you get some related error for the second purchase attempt via observer (e.g. "Cannot connect to iTunes Store") or will it just fail silently. And after you have called finishTransaction as you should, another attempt to add a SKPayment to the queue will result an error dialog "You already own this" and not a new purchase is done. That's why having your own checks for double payments and cached transactions is unnecessary.

[LHLL]

My experience from StoreKit behavior is very different. If you have already added a SKPayment to the queue i.e. the Apple purchase dialog has been shown and user has confirmed the purchase, but you have failed to call finishTransaction for any reason, another attempt to add a SKPayment to the queue will fail and the purchase dialog is NOT shown.
That's depends on the type of SKProduct you are offering. It's not the same across all type of products.

What was actually the reason trying to maintain your own cached transaction array instead of using SKPaymentQueue.transactions?
To prevent delivering the product to the wrong account. A SKPaymentTransaction's identifier is nil until it reaches state SKPaymentTransactionStatePurchased or SKPaymentTransactionStateRestored, therefore, if a transaction is not finished in the current session, there is no guarantee next time the user will login with the same account.

[kinex]

That's depends on the type of SKProduct you are offering. It's not the same across all type of products.

Based on my tests (I use Non-Consumables and auto-renewing Subscriptions in my app) and also what is reported in several GitHub issues and StackOverflow cases the behavior seems to be the same for all product types, even for Consumables.

What was actually the reason trying to maintain your own cached transaction array instead of using SKPaymentQueue.transactions?
To prevent delivering the product to the wrong account. A SKPaymentTransaction's identifier is nil until it reaches state SKPaymentTransactionStatePurchased or SKPaymentTransactionStateRestored, therefore, if a transaction is not finished in the current session, there is no guarantee next time the user will login with the same account.

So SKPaymentQueue.transactions contains the transactions that have not been finished yet. It should be the only source of truth.

I think the account related stuff you are talking about is completely out of the scope of this plugin. That's something the app developer must take care of, not this plugin. I don't even understand what this plugin could do for it. In my app I am already associating a subscription to a single account. I do this by uploading the subscription receipt to my backend where it is saved to a database with the UID of the authenticated user. Then if another account tries to upload the same receipt, I can reject it and user gets a response "Not subscribed". But user is allowed also to change his account by first deleting the current account and then signing in with a different account in which case the subscription gets transferred to the new account.

I think I have provided enough information related to this PR. Feel free to cancel and close this if you think it is too much against the design idea of your implementation, I understand that too. This is not a problem to me as my app is now working and I can hang in my fork as long as I see it is necessary. It may be very challenging to get that cached transactions array to work properly. But of course, as always, everything is possible :)

[d9media]

I totally agree with kinex research. It makes complete sense to remove the cache. I believe the cache has been the source of much the trouble we have had with this plug-in. I also believe it doesn't even work in some case. I spent countless hours on Browserstack testing with real-life devices and the behavior of this integration wasn't clear to me. For instance, you may be signed-in to your Apple ID on your iPhone. The plug-in then prompts another user login and you can use another Apple ID. In some cases, I have used an Apple ID different from the iPhone one (sandbox user) but still it returned the subscription of the main user rather than the new account.

Kindly would you consider my suggestion to make the transaction cache optional? We can leave it on by default. We can then get real-life feedback and see which implementation would be less troublesome. But my theory is that indeed the cache is not required, given that some devs moved to another package and seem very happy.

Another suggestion I could make is to use my Apple developer code-level ticket to have Apple take a look at this although I don't know if that will be very fruitful.

[LHLL]

I think the account related stuff you are talking about is completely out of the scope of this plugin. That's something the app developer must take care of, not this plugin. I don't even understand what this plugin could do for it. In my app I am already associating a subscription to a single account. I do this by uploading the subscription receipt to my backend where it is saved to a database with the UID of the authenticated user. Then if another account tries to upload the same receipt, I can reject it and user gets a response "Not subscribed". But user is allowed also to change his account by first deleting the current

A receipt that contains enough information or a transaction identifier is only available when a transaction enters purchased/restored state. What can cause issue transactions from the previous app running session which there is guarantee the same account will be logged in. If your only need is to be able to clear failed transactions, this can be a valid feature that some apps might not handle it properly in the previous version and the only way to handle that is to have a new API.

[LHLL]

For instance, you may be signed-in to your Apple ID on your iPhone. The plug-in then prompts another user login and you can use another Apple ID. In some cases, I have used an Apple ID different from the iPhone one (sandbox user) but still it returned the subscription of the main user rather than the new account.

All above examples are Apple's sandbox issues, you can easily fix them by resetting your test device. You are not supposed to test with different Apple IDs on the same device in the TestFlight environment.

What I think would be useful is to expose a new API that kills all failed transactions from previous sessions.

[d9media]

For instance, you may be signed-in to your Apple ID on your iPhone. The plug-in then prompts another user login and you can use another Apple ID. In some cases, I have used an Apple ID different from the iPhone one (sandbox user) but still it returned the subscription of the main user rather than the new account.

All above examples are Apple's sandbox issues, you can easily fix them by resetting your test device. You are not supposed to test with different Apple IDs on the same device in the TestFlight environment.

What I think would be useful is to expose a new API that kills all failed transactions from previous sessions.

I get that but what happens in production? Couldn't they be signed-in to one Apple ID on their phone and use another log-in for IAP? What would happen in such case?

[LHLL]

For instance, you may be signed-in to your Apple ID on your iPhone. The plug-in then prompts another user login and you can use another Apple ID. In some cases, I have used an Apple ID different from the iPhone one (sandbox user) but still it returned the subscription of the main user rather than the new account.

All above examples are Apple's sandbox issues, you can easily fix them by resetting your test device. You are not supposed to test with different Apple IDs on the same device in the TestFlight environment.
What I think would be useful is to expose a new API that kills all failed transactions from previous sessions.

I get that but what happens in production? Couldn't they be signed-in to one Apple ID on their phone and use another log-in for IAP? What would happen in such case?

iOS does allow you to use one for iCloud and one for App Store. But thing could become tricky when a user has two Apple ID for the App Store(Imaging one for US App Store, and the other for the Mexico App Store). This leads to a lot of in-app purchase bugs based on my experience and cannot be debugged purely on the app side since Apple does not expose which Apple ID the user is using. I personally use storefront to mitigate some of those bugs, but this API will not be called if two Apple IDs are used in the same region. An example is, let's say your app provides a subscription with multiple tiers. User purchased tier one with Apple ID A, and later upgraded to tier two with Apple ID B. This transaction will go through and the user will be charged on both Apple ID A and Apple ID B.

But regardless, I don't think this is related to this plugin and PR-467870043 should fix the issue discussed in this PR. And do let me know if you think a new API to remove all failed transactions is what you need.

[kinex]

It is still totally unclear to me why do you need the cached transactions. Could you please clarify these questions again:

1. How have you come to a conclusion that an user can be charged twice for the same product if it is not explicitly prevented by this plugin? Can you point me a related GitHub issue, StackOverflow question, code sample, documentation reference, anything? So why did you implement the own transactions cache in the first place?

2. Can you present a sample case, when your own cache contains more reliable status compared to the SKPaymentQueue.transactions? How is your cached transactions different from SKPaymentQueue.transactions in that case?

Here is also a related blog post, which lists caching transactions as one of the mistakes you should avoid.

[LHLL]

Let's assume you support login with Google, then you easily verify this your first question by adding a transaction of product A for Google Account A, wait for it to become success. Don't finish the transaction, switch to Google Account B, purchase the same product again. Now your backend can also validate the receipt with Apple and deliver the product to Google Account B. At this time, we should block Google Account B from purchasing until the transaction being finished.

I am not the original author of the cached transaction, but the issue you are facing is not related to the cache itself but zombie transactions in the SKPaymentQueue. And based on my testing, the only edge case that cache transaction will become an issue is actually restoring transactions and this can be fixed by PR-467870043. I am open to remove the cache if you can prevent the above example from happening.

[kinex]

Don't finish the transaction, switch to Google Account B, purchase the same product again

I am repeating myself again, but based on all my current knowledge and tests this (trying to purchase the same product again before finishing the transaction) is simply not possible, Apple (StoreKit) already prevents it. That's why I asked if you can point any related GitHub issue, StackOverflow case or anything which a kind of "proves" this could happen if you don't explicitly prevent it.

Considering previous "fact", this is not important, but I want to also mention that nowhere is said that purchases should be connected to a single app account? App can allow using any number of accounts if it decides so. Also, verifying a receipt does not require signing in to your app account so you don't need to wait for user to sign in to be able to finish a transaction. My point here is that all app account related stuff is out of the context of this plugin. This plugin is for making purchases. It is app's responsibility to decide how it uses those purchased products.

I am confident removing the transactions cache improves the stability of this plugin, but I am not saying it fixes all the issues. For some reason I still have to clear the transactions from SKPaymentQueue.transactions before adding a new transaction to the queue (or the purchase dialog may not be shown in some devices). I assume this is either some StoreKit issue or this happens only with users who have used TestFlight builds earlier (making test purchases).

[LHLL]

Don't finish the transaction, switch to Google Account B, purchase the same product again

I am repeating myself again, but based on all my current knowledge and tests this (trying to purchase the same product again before finishing the transaction) is simply not possible, Apple (StoreKit) already prevents it. That's why I asked if you can point any related GitHub issue, StackOverflow case or anything which a kind of "proves" this could happen if you don't explicitly prevent it.

Considering previous "fact", this is not important, but I want to also mention that nowhere is said that purchases should be connected to a single app account? App can allow using any number of accounts if it decides so. Also, verifying a receipt does not require signing in to your app account so you don't need to wait for user to sign in to be able to finish a transaction. My point here is that all app account related stuff is out of the context of this plugin. This plugin is for making purchases. It is app's responsibility to decide how it uses those purchased products.

I am confident removing the transactions cache improves the stability of this plugin, but I am not saying it fixes all the issues. For some reason I still have to clear the transactions from SKPaymentQueue.transactions before adding a new transaction to the queue (or the purchase dialog may not be shown in some devices). I assume this is either some StoreKit issue or this happens only with users who have used TestFlight builds earlier (making test purchases).

You can easily reproduce it by:

1. Remove all finishTransaction: calls from your current app.
2. Generate a new build to TestFlight.
3. Launch product page with Google Account A, and purchase a subscription.
4. Verify you see Apple Pay UI and your backend delivers the subscription to Google Account A.
5. Log out and launch the product page with Google Account B, and purchase the same subscription.
6. Verify flow ends with no Apple Pay UI and your backend delivers the subscription to Google Account B.

As for nowhere is said that purchases should be connected to a single app account, this is an example of demoing prevent adding a transaction of the same product can help eliminate hard-to-debug bugs. The root cause of the problem is not the cache itself even if the design is not the best approach. Simply remove the cache can bring new issues for clients that are relying on the cache to prevent the above edge case.

The rule of thumb should be avoid creating duplicate transactions of the same product in the transaction queue except for restored transactions. I am ok to accept or make any new changes, but the change should at least cover all known edge cases instead of part of them.

[kinex]

You really have spent time on thinking edge cases :) What is the expected end result? Not sure if anyone can have a correct answer to that.

But if you want to keep the check in addPayment, would this work (not using cached transactions):

- (BOOL)addPayment:(SKPayment *)payment {
  for (SKPaymentTransaction *transaction in self.queue.transactions) {
    if ([transaction.payment.productIdentifier isEqualToString:payment.productIdentifier]) {
      return NO;
    }
  }
  [self.queue addPayment:payment];
  return YES;
}

This LGTM. Feel free the add this change to this PR.

[kinex]

Suggested fixes done!

[LHLL]

LGTM, thanks.

[cyanglaz]

Looks like some tests failed on CI. @kinex Do you know what's going on?

[kinex]

@cyanglaz lint_darwin_plugins PLUGIN_SHARDING / testAddPaymentWithSameProductIDWillFail: it seems this test works only if the old transaction cache is there. Not sure if we can just remove the whole test. Or could someone who knows this this test better fix it.

test CHANNEL:master/stable: I have difficulties to see from the log why this fails. Could this be related to the failing lint_darwin_plugins PLUGIN_SHARDING test?

Btw is there some doc on how to run those tests locally?

[cyanglaz]

@LHLL would know more about the testAddPaymentWithSameProductIDWillFail test.

To run the failed tests locally, just run flutter test under the package/in_app_purchase directory.

[LHLL]

@kinex You can just remove that test since you removed the cache.

[kinex]

I guess flutter test does not run tests under ios/Tests.

Anyway, now all tests pass. I fixed one test and removed two tests. I had to remove also testDuplicateTransactionsWillTriggerAnError, please check if that's ok.

[ziggycrane]

This PR made things even worse, since the same problem now occurs because of restored transactions...

[kinex]

@ziggycrane I doubt that. What do you mean by "same problem"? Caching the transactions was the source of all problems. My app is in AppStore, with very detailed error reporting to Crashlytics and I don't see any related issues. I guess iOS 14 caused some new issues again, but those have already been resolved in the latest version of this plugin. So make sure you are using the latest version of the plugin.

I suggest you add a new issue where you describe the issue you are facing with source code. "Things are worse" comments do not help a lot here.

[ziggycrane]

Well, it's as simple as - with a new user purchase subscription. Cancel it later and then attempt to resubscribe and you will fail with the same error as before, because you now check the queue transactions and restored transactions are in queue so it will fail every time.

[kinex]

You can see from the comments above why the check is in addPayment.

Do you receive those restored purchases from purchaseUpdatedStream on app startup and handle them there as required?

[kinex]

I checked code in app_store_connection.dart and noticed that restored purchases are not passed to purchaseUpdatedStream. That was surpising at least to me. Not sure if that is a bug or a planned functionality.

From my understanding finishTransaction needs to be called also for restored purchases (SKPaymentTransactionState.restored). If restored purchases are not passed to purchaseUpdatedStream, doesn't that mean that you have to call queryPastPurchases at least on every app startup so that you can finish the restored purchases.

I don't face this issue in my app as I am clearing the whole transaction queue before I call buyNonConsumable (a temporary workaround due to earlier issues which I have been planning to remove later).

[ziggycrane]

Yes, you are right I have gotten that far too. For me, the problem is that I for the life of me can't finish these transactions even after queryPastPurchases. They appear every time as "restored" later even when I complete them.

Could you please show me the example of your workaround? I am desperate to make it work.

[kinex]

Calling queryPastPurchases has also at least one unwanted side effect: serverVerificationData is updated on every call to queryPastPurchases forcing me to upload it to backend again unnecessarily although nothing has actually changed. So I call queryPastPurchases on iOS only when really needed.

In my opinion this should be fixed so that also restored purchases are passed to purchaseUpdatedStream.

Other possible fix would be to call finishTransaction for all restored purchases automatically in this plugin.

This is my helper function:

  Future<void> clearTransactionsIos() async {
    if (Platform.isIOS) {
      final transactions = await SKPaymentQueueWrapper().transactions();
      for (final transaction in transactions) {
        try {
          await SKPaymentQueueWrapper().finishTransaction(transaction);
        } catch (e) {
          print(e);
        }
      }
      // magic delay, not sure if needed
      await Future.delayed(const Duration(seconds: 1));
    }
  }

I call it just before before calling InAppPurchaseConnection.buyNonConsumable.