Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

invoices: rm old corrupted non-bip70 invoices #6350

Merged

Conversation

SomberNight
Copy link
Member

fixes #6345

The root cause is that kivy on e.g. 3.3.8 allowed saving (outgoing) invoices even if they were not bip70,
and this is not really handled in the storage upgrades. (I think Qt only allowed saving bip70 invoices)

I added a unit test that includes a real testnet wallet I created in 3.3.8 and used with kivy to reproduce this.
See the invoices dict:

"invoices": {
    "c58dbd42b883d60433d9fb626b772406": {
      "hex": "0801120b783530392b7368613235361a9f160ac50c3082064130820529a003020102020900e1222c5cacc7e4c2300d06092a864886f70d01010b05003081b4310b30090603550406130255533110300e060355040813074172697a6f6e61311330110603550407130a53636f74747364616c65311a3018060355040a1311476f44616464792e636f6d2c20496e632e312d302b060355040b1324687474703a2f2f63657274732e676f64616464792e636f6d2f7265706f7369746f72792f313330310603550403132a476f2044616464792053656375726520436572746966696361746520417574686f72697479202d204732301e170d3138303830323138343432305a170d3139313030313231313333385a303d3121301f060355040b1318446f6d61696e20436f6e74726f6c2056616c696461746564311830160603550403130f746573742e6269747061792e636f6d30820122300d06092a864886f70d01010105000382010f003082010a0282010100d14377b6f2b50aa73427c248915563d2c8804dd824b1d8f015dd1ae2a390c8022fd963355849b217309208c3450a016ff4001ee336a3189edadce12bc41e4999e45f6fcf0b00daa7e9a4b12e72dce19c51ae8e55b8bfec02b80a58ccb2c1b3e5bde81f679b9993bf52c871a1fadef6bb5f7d8d9208889400ba2be1d2baf82ec303470852570c3bbb6b89334d8974e61b8867bf299fb802c57e9b00d9b9f70572ac6d81fecd304c83aaf21f4f3b529e9898ea9b868f8f07b4189668e71854ae776bacd0d9706a8be03f528c68ad023e3b45bfa55e9b42e535aafc7eb8672645dcdeaf7204a468d2b84f27ed12072a411627647108e421abe7308e3bac305896f10203010001a38202ca308202c6300c0603551d130101ff04023000301d0603551d250416301406082b0601050507030106082b06010505070302300e0603551d0f0101ff0404030205a030370603551d1f0430302e302ca02aa0288626687474703a2f2f63726c2e676f64616464792e636f6d2f676469673273312d3835342e63726c305d0603551d20045630543048060b6086480186fd6d010717013039303706082b06010505070201162b687474703a2f2f6365727469666963617465732e676f64616464792e636f6d2f7265706f7369746f72792f3008060667810c010201307606082b06010505070101046a3068302406082b060105050730018618687474703a2f2f6f6373702e676f64616464792e636f6d2f304006082b060105050730028634687474703a2f2f6365727469666963617465732e676f64616464792e636f6d2f7265706f7369746f72792f67646967322e637274301f0603551d2304183016801440c2bd278ecc348330a233d7fb6cb3f0b42c80ce302f0603551d1104283026820f746573742e6269747061792e636f6d82137777772e746573742e6269747061792e636f6d301d0603551d0e041604142cd8def7d64c620cce78bdec365fd961cfd035e130820104060a2b06010401d6790204020481f50481f200f0007700a4b90990b418581487bb13a2cc67700a3c359804f91bdfb8e377cd0ec80ddc1000000164fbf55c500000040300483046022100d6a82312b4a16c7ed7911a2e559168ea77d01e3dcf9cfe63a6662e2661546405022100a43fe7cbe2fd9ae72a349ba1b19de71e7804cae2c3755774c4b8de4d432cbcd5007500747eda8331ad331091219cce254f4270c2bffd5e422008c6373579e6107bcc5600000164fbf55ed7000004030046304402201727366c115d0fb8f940cce989730727cc59d15c94120146f1c239989deeecaa0220602b946de08661ed7ff3ed64988a3d872151c89f20a9c0dbbc6edcc04870380a300d06092a864886f70d01010b05000382010100060cf534f7adea711fabcd9338fa89f4bc244da0a6af232f7d337f2d8ea79394343b5f5dfb07c3f2e71195755e0a8f3f51b0444e7e17b4b926cb0f9dc49253dc66f1ed8fd52297be8b515a4240ac59fc3f7f1fa810ab24b196c4ae57827abf25e76838edc0a86bfdd0c386f9c4b6a7fa94672b79177646323875cfff2da42106be46ff0e1b41e3b5706f1be19b582005fbedacf88d69370a2ccda66bffe6e7ef5f33a408707ebd3ddc20b1ff24d6bd94cc6db5e069277d78d56218cfddbfcd83f2be5e477734f548a6da0ae58bed212bf817914f3fe6f9afa4ad59fe5d97b0514e277dc6ba48672e074b23fe89a5f72c58ec139b3f1c09d557b6e7d2df5f52880ad409308204d0308203b8a003020102020107300d06092a864886f70d01010b0500308183310b30090603550406130255533110300e060355040813074172697a6f6e61311330110603550407130a53636f74747364616c65311a3018060355040a1311476f44616464792e636f6d2c20496e632e3131302f06035504031328476f20446164647920526f6f7420436572746966696361746520417574686f72697479202d204732301e170d3131303530333037303030305a170d3331303530333037303030305a3081b4310b30090603550406130255533110300e060355040813074172697a6f6e61311330110603550407130a53636f74747364616c65311a3018060355040a1311476f44616464792e636f6d2c20496e632e312d302b060355040b1324687474703a2f2f63657274732e676f64616464792e636f6d2f7265706f7369746f72792f313330310603550403132a476f2044616464792053656375726520436572746966696361746520417574686f72697479202d20473230820122300d06092a864886f70d01010105000382010f003082010a0282010100b9e0cb10d4af76bdd49362eb3064b881086cc304d962178e2fff3e65cf8fce62e63c521cda16454b55ab786b63836290ce0f696c99c81a148b4ccc4533ea88dc9ea3af2bfe80619d7957c4cf2ef43f303c5d47fc9a16bcc3379641518e114b54f828bed08cbef030381ef3b026f86647636dde7126478f384753d1461db4e3dc00ea45acbdbc71d9aa6f00dbdbcd303a794f5f4c47f81def5bc2c49d603bb1b24391d8a4334eeab3d6274fad258aa5c6f4d5d0a6ae7405645788b54455d42d2a3a3ef8b8bde9320a029464c4163a50f14aaee77933af0c20077fe8df0439c269026c6352fa77c11bc87487c8b993185054354b694ebc3bd3492e1fdcc1d252fb0203010001a382011a30820116300f0603551d130101ff040530030101ff300e0603551d0f0101ff040403020106301d0603551d0e0416041440c2bd278ecc348330a233d7fb6cb3f0b42c80ce301f0603551d230418301680143a9a8507106728b6eff6bd05416e20c194da0fde303406082b0601050507010104283026302406082b060105050730018618687474703a2f2f6f6373702e676f64616464792e636f6d2f30350603551d1f042e302c302aa028a0268624687474703a2f2f63726c2e676f64616464792e636f6d2f6764726f6f742d67322e63726c30460603551d20043f303d303b0604551d20003033303106082b06010505070201162568747470733a2f2f63657274732e676f64616464792e636f6d2f7265706f7369746f72792f300d06092a864886f70d01010b05000382010100087e6c9310c838b896a9904bffa15f4f04ef6c3e9c8806c9508fa673f757311bbebce42fdbf8bad35be0b4e7e679620e0ca2d76a637331b5f5a848a43b082da25d90d7b47c254f115630c4b6449d7b2c9de55ee6ef0c61aabfe42a1bee849eb8837dc143ce44a713700d911ff4c813ad8360d9d872a873241eb5ac220eca17896258441bab892501000fcdc41b62db51b4d30f512a9bf4bc73fc76ce36a4cdd9d82ceaae9bf52ab290d14d75188a3f8a4190237d5b4bfea403589b46b2c3606083f87d5041cec2a190c3bbef022fd21554ee4415d90aaea78a33edb12d763626dc04eb9ff7611f15dc876fee469628ada1267d0a09a72e04a38dbcf8bc0430012294020a0474657374121f08848e06121976a914314f713cd59781894277dcfdd31b5f179860b61e88ac18dbe793f80520dfee93f8052a5a5061796d656e74207265717565737420666f722042697450617920696e766f69636520526452684a4e3334426f45417a443151614c7067524b20666f72206d65726368616e7420536f6d6265724e696768745f74657374696e67323068747470733a2f2f746573742e6269747061792e636f6d2f692f526452684a4e3334426f45417a443151614c7067524b3a4c7b22696e766f6963654964223a22526452684a4e3334426f45417a443151614c7067524b222c226d65726368616e744964223a225372384b5774647158666b6a58563751704171773336227d450000803f2a80022896c21e381d423cd31672981c90f65919db4e30dd5a2f62b9be1e73a9a875ba176086e66507aeb67ad4b1c5cfd6bab57b49a6b8895b12bf14dbe868dfacc3ce04e5617c96bd1a4eb8c5fcb14122e569dfb9d860d67ed8d494f0a348a3e6ea00d27aab41c18ea625b872411543a52db374f97f7f01ed3df3be5f33551661ecb83dbab311d17d8942925934a0eab7965bb3e8a908bedd82001c6f1afe2ebca2616da288716452da8f4f32ca625a34f3f52b04b7ff32816b4875cfe3789e132ae1959d2a68d34aaed937c60b268cd7737889981071431e406318fd3619615d84647a0b900060f997ed8aa7ce50f65212d9d654409ae77001d3952b784a687a4082",
      "requestor": "test.bitpay.com",
      "txid": null
    },
    "tb1qh6qyzl3pds56azgjqkhkk7kfkavzxghjwsv0rl": {
      "hex": "2229121c08f8c70d12160014be80417e216c29ae891205af6b7ac9b7582322f2180020002a0574657374742a00",
      "requestor": null,
      "txid": null
    }
  }

convert_version_25 and convert_version_29 deal with related stuff, and they don't handle the non-bip70 invoice well; it ends up basically corrupted:

"invoices": {
  'c58dbd42b883d60433d9fb626b772406': {
    'type': 0,
    'message': 'Payment request for BitPay invoice RdRhJN34BoEAzD1QaLpgRK for merchant SomberNight_testing',
    'exp': 900,
    'time': 1594160091,
    'outputs': [
      (0,
      'mk1gaCn7HHqJRFwo15tKdhh9X2JppC8c5d',
      100100)
    ],
    'id': 'c58dbd42b883d60433d9fb626b772406',
    'bip70': '0801120b783530392b7368613235361a9f160ac50c3082064130820529a003020102020900e1222c5cacc7e4c2300d06092a864886f70d01010b05003081b4310b30090603550406130255533110300e060355040813074172697a6f6e61311330110603550407130a53636f74747364616c65311a3018060355040a1311476f44616464792e636f6d2c20496e632e312d302b060355040b1324687474703a2f2f63657274732e676f64616464792e636f6d2f7265706f7369746f72792f313330310603550403132a476f2044616464792053656375726520436572746966696361746520417574686f72697479202d204732301e170d3138303830323138343432305a170d3139313030313231313333385a303d3121301f060355040b1318446f6d61696e20436f6e74726f6c2056616c696461746564311830160603550403130f746573742e6269747061792e636f6d30820122300d06092a864886f70d01010105000382010f003082010a0282010100d14377b6f2b50aa73427c248915563d2c8804dd824b1d8f015dd1ae2a390c8022fd963355849b217309208c3450a016ff4001ee336a3189edadce12bc41e4999e45f6fcf0b00daa7e9a4b12e72dce19c51ae8e55b8bfec02b80a58ccb2c1b3e5bde81f679b9993bf52c871a1fadef6bb5f7d8d9208889400ba2be1d2baf82ec303470852570c3bbb6b89334d8974e61b8867bf299fb802c57e9b00d9b9f70572ac6d81fecd304c83aaf21f4f3b529e9898ea9b868f8f07b4189668e71854ae776bacd0d9706a8be03f528c68ad023e3b45bfa55e9b42e535aafc7eb8672645dcdeaf7204a468d2b84f27ed12072a411627647108e421abe7308e3bac305896f10203010001a38202ca308202c6300c0603551d130101ff04023000301d0603551d250416301406082b0601050507030106082b06010505070302300e0603551d0f0101ff0404030205a030370603551d1f0430302e302ca02aa0288626687474703a2f2f63726c2e676f64616464792e636f6d2f676469673273312d3835342e63726c305d0603551d20045630543048060b6086480186fd6d010717013039303706082b06010505070201162b687474703a2f2f6365727469666963617465732e676f64616464792e636f6d2f7265706f7369746f72792f3008060667810c010201307606082b06010505070101046a3068302406082b060105050730018618687474703a2f2f6f6373702e676f64616464792e636f6d2f304006082b060105050730028634687474703a2f2f6365727469666963617465732e676f64616464792e636f6d2f7265706f7369746f72792f67646967322e637274301f0603551d2304183016801440c2bd278ecc348330a233d7fb6cb3f0b42c80ce302f0603551d1104283026820f746573742e6269747061792e636f6d82137777772e746573742e6269747061792e636f6d301d0603551d0e041604142cd8def7d64c620cce78bdec365fd961cfd035e130820104060a2b06010401d6790204020481f50481f200f0007700a4b90990b418581487bb13a2cc67700a3c359804f91bdfb8e377cd0ec80ddc1000000164fbf55c500000040300483046022100d6a82312b4a16c7ed7911a2e559168ea77d01e3dcf9cfe63a6662e2661546405022100a43fe7cbe2fd9ae72a349ba1b19de71e7804cae2c3755774c4b8de4d432cbcd5007500747eda8331ad331091219cce254f4270c2bffd5e422008c6373579e6107bcc5600000164fbf55ed7000004030046304402201727366c115d0fb8f940cce989730727cc59d15c94120146f1c239989deeecaa0220602b946de08661ed7ff3ed64988a3d872151c89f20a9c0dbbc6edcc04870380a300d06092a864886f70d01010b05000382010100060cf534f7adea711fabcd9338fa89f4bc244da0a6af232f7d337f2d8ea79394343b5f5dfb07c3f2e71195755e0a8f3f51b0444e7e17b4b926cb0f9dc49253dc66f1ed8fd52297be8b515a4240ac59fc3f7f1fa810ab24b196c4ae57827abf25e76838edc0a86bfdd0c386f9c4b6a7fa94672b79177646323875cfff2da42106be46ff0e1b41e3b5706f1be19b582005fbedacf88d69370a2ccda66bffe6e7ef5f33a408707ebd3ddc20b1ff24d6bd94cc6db5e069277d78d56218cfddbfcd83f2be5e477734f548a6da0ae58bed212bf817914f3fe6f9afa4ad59fe5d97b0514e277dc6ba48672e074b23fe89a5f72c58ec139b3f1c09d557b6e7d2df5f52880ad409308204d0308203b8a003020102020107300d06092a864886f70d01010b0500308183310b30090603550406130255533110300e060355040813074172697a6f6e61311330110603550407130a53636f74747364616c65311a3018060355040a1311476f44616464792e636f6d2c20496e632e3131302f06035504031328476f20446164647920526f6f7420436572746966696361746520417574686f72697479202d204732301e170d3131303530333037303030305a170d3331303530333037303030305a3081b4310b30090603550406130255533110300e060355040813074172697a6f6e61311330110603550407130a53636f74747364616c65311a3018060355040a1311476f44616464792e636f6d2c20496e632e312d302b060355040b1324687474703a2f2f63657274732e676f64616464792e636f6d2f7265706f7369746f72792f313330310603550403132a476f2044616464792053656375726520436572746966696361746520417574686f72697479202d20473230820122300d06092a864886f70d01010105000382010f003082010a0282010100b9e0cb10d4af76bdd49362eb3064b881086cc304d962178e2fff3e65cf8fce62e63c521cda16454b55ab786b63836290ce0f696c99c81a148b4ccc4533ea88dc9ea3af2bfe80619d7957c4cf2ef43f303c5d47fc9a16bcc3379641518e114b54f828bed08cbef030381ef3b026f86647636dde7126478f384753d1461db4e3dc00ea45acbdbc71d9aa6f00dbdbcd303a794f5f4c47f81def5bc2c49d603bb1b24391d8a4334eeab3d6274fad258aa5c6f4d5d0a6ae7405645788b54455d42d2a3a3ef8b8bde9320a029464c4163a50f14aaee77933af0c20077fe8df0439c269026c6352fa77c11bc87487c8b993185054354b694ebc3bd3492e1fdcc1d252fb0203010001a382011a30820116300f0603551d130101ff040530030101ff300e0603551d0f0101ff040403020106301d0603551d0e0416041440c2bd278ecc348330a233d7fb6cb3f0b42c80ce301f0603551d230418301680143a9a8507106728b6eff6bd05416e20c194da0fde303406082b0601050507010104283026302406082b060105050730018618687474703a2f2f6f6373702e676f64616464792e636f6d2f30350603551d1f042e302c302aa028a0268624687474703a2f2f63726c2e676f64616464792e636f6d2f6764726f6f742d67322e63726c30460603551d20043f303d303b0604551d20003033303106082b06010505070201162568747470733a2f2f63657274732e676f64616464792e636f6d2f7265706f7369746f72792f300d06092a864886f70d01010b05000382010100087e6c9310c838b896a9904bffa15f4f04ef6c3e9c8806c9508fa673f757311bbebce42fdbf8bad35be0b4e7e679620e0ca2d76a637331b5f5a848a43b082da25d90d7b47c254f115630c4b6449d7b2c9de55ee6ef0c61aabfe42a1bee849eb8837dc143ce44a713700d911ff4c813ad8360d9d872a873241eb5ac220eca17896258441bab892501000fcdc41b62db51b4d30f512a9bf4bc73fc76ce36a4cdd9d82ceaae9bf52ab290d14d75188a3f8a4190237d5b4bfea403589b46b2c3606083f87d5041cec2a190c3bbef022fd21554ee4415d90aaea78a33edb12d763626dc04eb9ff7611f15dc876fee469628ada1267d0a09a72e04a38dbcf8bc0430012294020a0474657374121f08848e06121976a914314f713cd59781894277dcfdd31b5f179860b61e88ac18dbe793f80520dfee93f8052a5a5061796d656e74207265717565737420666f722042697450617920696e766f69636520526452684a4e3334426f45417a443151614c7067524b20666f72206d65726368616e7420536f6d6265724e696768745f74657374696e67323068747470733a2f2f746573742e6269747061792e636f6d2f692f526452684a4e3334426f45417a443151614c7067524b3a4c7b22696e766f6963654964223a22526452684a4e3334426f45417a443151614c7067524b222c226d65726368616e744964223a225372384b5774647158666b6a58563751704171773336227d450000803f2a80022896c21e381d423cd31672981c90f65919db4e30dd5a2f62b9be1e73a9a875ba176086e66507aeb67ad4b1c5cfd6bab57b49a6b8895b12bf14dbe868dfacc3ce04e5617c96bd1a4eb8c5fcb14122e569dfb9d860d67ed8d494f0a348a3e6ea00d27aab41c18ea625b872411543a52db374f97f7f01ed3df3be5f33551661ecb83dbab311d17d8942925934a0eab7965bb3e8a908bedd82001c6f1afe2ebca2616da288716452da8f4f32ca625a34f3f52b04b7ff32816b4875cfe3789e132ae1959d2a68d34aaed937c60b268cd7737889981071431e406318fd3619615d84647a0b900060f997ed8aa7ce50f65212d9d654409ae77001d3952b784a687a4082',
    'requestor': 'mk1gaCn7HHqJRFwo15tKdhh9X2JppC8c5d',
    'amount_sat': 100100
  },
  'tb1qh6qyzl3pds56azgjqkhkk7kfkavzxghjwsv0rl': {
    'type': 0,
    'message': '',
    'exp': 0,
    'time': 0,
    'outputs': None,
    'id': None,
    'bip70': None,
    'requestor': None,
    'amount_sat': 0
  }
}

I think we need to remove these invoices now, which is what this PR does.

@ecdsa
Copy link
Member

ecdsa commented Jul 8, 2020

thanks

@ecdsa ecdsa merged commit 3442dcc into spesmilo:master Jul 8, 2020
@ecdsa
Copy link
Member

ecdsa commented Jan 12, 2021

This was not sufficient to fix #6345. From the recent tracebacks:
File "/home/user/wspace/electrum/.buildozer/android/app/electrum/wallet_db.py", line 1199, in <genexpr>

The line number 1199 shows that it should have been applied to payment_requests too, not only to 'invoices'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

TypeError: 'NoneType' object is not iterable
2 participants