Handle when Multipart frame cannot be scanned in time

[pmespresso]

as discussed with Maciej, there is a possibility that multipart payloads could get stuck infinitely with a particular fps.

for an example with 3 frames:

• frame 1 cannot be scanned to completion before the the next frame is shown by hot wallet
• hot wallet loops through the other frames, cold signer successfully scans them
• we are now at 2/3 progress of scanning the frames
• hot wallet loops back to 1st frame, cold signer fails to scan it in time yet again,
• repeat steps 1 - 4 forever or till user gives up, cold signer remains at 2/3 progress, user is confused

The potential solutions brought up were to either:

1. show the user a slider to slow down the FPS after a few cycles
2. gradually slow down the FPS every cycle

in any case, the user should be alerted at the very least.

[pmespresso]

closing as the most we can do is show which frames were missed as in #427. Toggling frame size, frame delay, etc. should be an enhancement on the hot wallet end.

[burdges]

Are you guys following any standard for doing this? Assuming no..

Are your multi-part messages creating a two-way channel for ACKs? Or asking users to resend parts? If either, then you could maybe do better with erasure coding:

We encode the data using either some fountain/rateless erasure code like a Raptor code (code: raptorq), which entails some patent worries, or else some any erasure code with enough excess rate. We avoid any ACK process by simply sending forever at some safe speed until the receiver gets enough that their device reconstructs the full message. It still requires one ACK to stop the sender though.

Actually QR codes already use Reed-Solomon codes, so conceivably an optimal scheme would make the multi-part erasure coding play nicer with the per QR code code, not sure. I've found nothing about doing this the "right way" though.

Anyways @Tbaut and @yjkimjunior if "send forever until it works" sounds like a UX improvement then we could have someone look into this eventually.

[Tbaut]

Wow that's very interesting and could be better than showing what QR code we missed and let the user tell the sender to show the missed ones.

I couldn't answer these questions by quickly reading the articles

I wonder in practice for say a 15 parts QR code as we currently have for metadata:

• Does using erasure code increase the amount of part? I imagine it does, in which case how many part would be the minimum (mentioned as k in raptor code )?
• how many more QR codes does the user need to scan if they missed say 2 parts?

Depending on the values, this could be interesting to take into account.
For instance if you'd need at least 16 parts (instead of originally 15) and that you'd need say 3 more if you missed 2, that could work. But a click or 2 on the sender interface isn't super hard for a user and the advantage is that the user knows how long they'll need to wait or scan. With rateless they might wait infinitely if say their phone camera is super bad..

[burdges]

There is some tiny overhead due to part counters or whatever, but maybe not too significant. In expectation, if you miss two parts then you must scan two more parts, but occasionally you'll hit file sizes where the total number of parts that must be sent in the first place increases due to this overhead.

I think fountain code should occasionally have extra blocks that do not fully reconstruct the missing block too. I'm not sure about the frequency, but they make big claims about these codes, so fairly low I think.

There is already some increase from the Reed-Solomon code used in the QR code itself, so I wondered if making the codes mesh nicely improves that, but maybe not really since it's not so easy to know if the image go across. I know very little about this applied side of using erasure odes sadly. As an aside, https://github.com/gre/qrloop did some frame rate analysis, via https://stackoverflow.com/questions/10859043/using-multiple-qr-codes-to-encode-a-binary-image

Assuming we add just another layer of erasure coding without touching the QR code's existing erasure coding, then I'd think the interface could show some bar graph for how much it processed.

[burdges]

I just found one guy who did this by googling "animated QR code":
https://github.com/divan/txqr (https://github.com/divan/txqr-reader)
https://divan.dev/posts/animatedqr/
https://divan.dev/posts/fountaincodes/
https://hub.packtpub.com/introducing-txqr-data-transfer-via-animated-qr-codes/

Ain't clear from divan/txqr#9 if maybe some issues remain. Also color sounds interesting divan/txqr#2 And clearly nobody who really knows fountian codes weight in or else they could've answered divan/txqr#7 without testing.

[burdges]

Another TXQR implementation https://github.com/ThePlasmaRailgun/TXQR-Android although I still suspect you'd do better just using the crate https://github.com/cberner/raptorq