(light wallets) add support for e.g. "get_oldest_pending" ?

[ttcoder]

I'm writing a haiku light wallet, tailored for my needs (i.e. old fashioned power user rather than beautiful / visually appealing GUI :-)

Long Story Short:

I'm finding that implementing a light wallet (without a full node) can be done with ad-hoc code, except for one thing:
The wallet needs a way to retrieve the list of pending "sends", to be pocketed. To my understanding, the UDP based protocol does not provide for this currently, only access to a full-node's RPC port does.
There's also a few other less important improvements that could help.

I'll look at the source code of the iOS and Android wallets when available; but thought I'd file a ticket right now as I'm not sure when/whether those will answer my questions.

Details:

I'm writing this wallet to be as light as possible (it will be in C++ native to Haiku OS but that's another topic, if there's interest in porting the rai_node code to it?).
If the source code can be useful to others I could upload it to chiselapp after a clean-up.
The wallet does not store blocks at all, except for the pruned user's accounts' block chain.
It's 32bit and uses IPv4 code, since my router won't handle IPv6.

I'm at the stage where I can create "send" blocks:
After each send I keep track of the latest block hash, balance, and pre-calculate work for the next send.
There's also code to "catch up", if the app loses track, or if creating a wallet from seed and dealing with an account that has a pre-existing history to catch up on: I then use "bulk_pull", which returns the full account's block chain.

(suggestion 1: would that waste bandwidth for accounts with huge histories? Idea: with a special parameter, bulk_pull could return only the last block; after the transition to UTX blocks state_blocks that will be a bandwidth efficient way to catch-up).

What I haven't found a way to do yet: the equivalent of the RPC command "pending"...
As a stop-gap measure the wallet relies on www.nanode.co 's free services/API (thanks to whoever provides them BTW!) but that's not decentralized, and if I went to production with that I would have to provide my NaNode API key with copies of my application... Ouch.

Suggestion 2: Is the above something that should be addressed ? If so, how do we make life easier for light wallets ?
Do we add a new message type, "get_oldest_pending", working on UDP/IP port 7075 ?

Then the wallet calls "get_oldest_pending" to get the block hash of pending transactions (if any)..
Then I would run a "bulk_pull_blocks" on that hash (currently it's not necessary, but after the UTX state_blocks transition, a "receive" block will also have to add up the old balance to the amount, to provide the new balance). Or better yet, "get_oldest_pending" would provide not only the hash but also the balance, so that it's a one-message deal ?

Misc issues:

• keepalive listings "pollution":
  when I send a keepalive, nodes add me to their list of "contacts" and, I guess, will then provide my IP to others; yet I of course won't respond to connection attempts, won't honor requests to bulk_pull ..etc. (in fact the wallet closes the UDP port immediately when it's done).

Maybe there should be a way to say "I'm not a node, I just pinged you to say hello and publish a block or ask for a bulk_pull" ? Otherwise light wallets like me will 'pollute' the keepalive lists.. If too many wallets do that, nodes will have to sift through a bunch of non-responding addies before reaching a functional one.

• keepalive flags:
  E.g. there could be a variant of that message, like "keepalive_etc", that adds a two-bytes "flags" field in the header. When sending a keepalive_etc to hosts I would set the IM_NOT_A_FULL_NODE flag or some such, for the node to know who's talking to it.
  Also, a second flag that could be useful is a "GOOD_SAMARITAN" flag, to be used by full nodes, when responding to wallets.. Or a flag with reverse semantics, a flag signalling that a node is not willing to provide light-wallet specific messages (like get_oldest_pending) because it does not like freeloaders :-]

That's all that comes to mind for now. Hope it's of interest

[numtel]

That's exactly how my wallet works by querying nanode.co. @rawrmaan said it was fine for this use.

I have been wondering about this same issue. These suggestions are fantastic for creating true decentralized light wallets. The bulk_pull_blocks PR seems like a decent guide on how to add a new message type.

What is the significance of oldest in get_oldest_pending? Why not have a message be simply get_pending? AFAIK, there is no way to provide an order to pending blocks so the only parameter in the body along with the account public key would be a limit perhaps?

Is there a political issue making this RPC action, previously private, public on the peer-to-peer interface? Obviously, this change plays with the incentive to run a node. Like you mention, a boolean config option for respond_to_get_oldest_pending would work to mitigate this. Internal rate-limiting seems like a good idea.

[ttcoder]

@numtel -- Come to think of it, it probably makes sense to..

• retrieve the pending blocks for several accounts at once, not just one account (my app is shaping up to be account-intensive)
• retrieve the whole pending list for each account, rather than only one pending block

Pretty much like the RPC command "pendings" plural. And also (similar to RPC commands), there should be a parameter to provide a limit -- just like you suggest.

My first take was motivated by saving bandwidth, not being too much of a "freeloader" on full nodes. But if you end up spooling several requests, that actually ends up increasing network traffic, so..

Light wallets are not very much emphasized or technically detailed (in terms of cutting off the full-node RPC "ombilical") in the recently released roadmap.. So it's not quite clear how much of a priority they are. The "thumbs up" from dev is quite encouraging though.

I don't have previous experience with coding specifically a CC wallet or with cryptography but even at my level I realize it's a thin line to walk indeed, balancing light wallets vs. network health. Light wallets will probably want to have the very smallest footprint (don't overtax the nodes) they can, whilst still preserving security of the nano's they hold. Anyway.. will put this on the back burner for a little while, several things need to happen first: state_blocks/canary, open sourcing the official wallets ..etc.

[ttcoder]

Ouch, nanode.co's API is going offline soon

The email I received makes it sound like the API goes offlines in mid-june ("14 days notice") whereas this page https://www.nanode.co/node-api mentions Jul 15th
Anyhow, we'll have to do without soon

That has an impact for me as my perspective on nano is using it in a decentralized and light-weight way

Possible ways from here..

1. some leads from this ticket being applied in the future

2. a way for me to run my own node (optimized enough that I can shutdown in the evening and have it catch up to the latest blockchain activity when switching my computer back on in the morning; heavy pruning to have a small database, say a few dozen MB; run on IPv4 since that's what my ISP provides me)

3. failing all that, and that would s*ck, my wallet would have to open a web browser on a chain explorer and instruct the user to collect such and such value and paste it in such and such textfield

Just one guy's perspective, but I have a suspicion it would be good for the rest of the community too..

[rkeene]

This can be implemented using the new bootstrapping primitive "bulk_pull_account". See pull request #1039 and issue #988.

Let me know if this does not satisfy your requirements.

[ttcoder]

Kudos for all you've been doing those last few weeks rkeene.

Looks like the protocol (possibly TCP ..etc) is going to kick ass in the near future. I think it's safe to close this ticket right now.

I'll get back on working on my wallet over the next few months and will file a ticket if I encounter new issues. (and I still swear I'm going to get the code clean enough to upload to chiselapp at some point, just hard to find some spare time for Haiku projects these days)

[rkeene]

Thank you for your support !