Looking for help! Sending a KEYSEND Payment via the REST API. HOW????

[brianoflondon]

I've been driving myself crazy trying to send a Keysend payment via the REST API.

I can do most else that I've tried to do via the REST API: check invoices, pay invoices, create new invoices... but sending a simple keysend payment has got me stumped!

I'm working on Value 4 Value streaming payments in Podcasting and AMP is not an option.

I know something like this will work from the command line...

lncli sendpayment -d 0266ad2656c7a19a219d37e82b280046660f4d7f3ae0c00b64a1629de4ea567668 -a 10 --keysend --data 696969=77616c5f6872444873305242454d353736 --json

But why can't I get my code to work? I'm using:
dest_pubkey = "0266ad2656c7a19a219d37e82b280046660f4d7f3ae0c00b64a1629de4ea567668"

I don't believe the KeysendCustomRecord is the issue as I get exactly the same failure with or without trying to send custom records. Headers and cert are working just fine.

The failure always looks like and this is after I've gone to hell and back figuring out the correct (I think) hex and base64 encoding of the destination public key.

{
  "payment_error": "incorrect_payment_details",
  "payment_preimage": "",
  "payment_route": null,
  "payment_hash": "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
}

Code fragment:

def b64_hex_transform(plain_str: str) -> str:
    """Returns the b64 transformed version of a hex string"""
    a_string = bytes.fromhex(plain_str)
    return base64.b64encode(a_string).decode()


def b64_transform(plain_str: str) -> str:
    """Returns the b64 transformed version of a string"""
    return base64.b64encode(plain_str.encode()).decode()


async def local_pay_keysend(
    amt: int, custom_record: KeysendCustomRecord, dest_pubkey: str = ""
):
    """Pay a keysend invoice using the chosen node"""
    node = "https://umbrel.local:8080/"
    headers, cert = get_lnd_headers_cert(admin=True, node=node)
    if not dest_pubkey:
        dest_pubkey = my_voltage_public_key

    # Base 64 encoded destination bytes
    dest = b64_hex_transform(dest_pubkey)
    cust_rec_str = json.dumps(custom_record.dict(exclude_unset=True, exclude_none=True))
    dest_custom_records = {
        7629169: b64_transform(cust_rec_str),
        818818: b64_transform("blocktvnews"),
    }

    url = f"{node}v1/channels/transactions"
    data = {"dest": dest, "amt": amt, "dest_custom_records": dest_custom_records}

    async with httpx.AsyncClient(verify=cert) as client:
        response = await client.post(url=url, headers=headers, data=json.dumps(data))

    print(json.dumps(response.json(), indent=2))
    return json.dumps(response.json(), indent=2)

[guggero]

Since you're not paying an invoice, the receiver doesn't know the preimage for the given payment's hash. You need to add the preimage to your custom records. The number is 5482373484.
It also looks like you need to actually set the payment's hash (payment_hash=sha256(preimage)) of the payment.
See https://github.com/lightningnetwork/lnd/blob/master/cmd/lncli/cmd_payments.go#L358 on how it's done on the RPC level (what's behind the --keysend flag of lndcli sendpayment).

[brianoflondon]

Thank you!!! I'm making progress but I'm not there yet.... unless I've missed something this just isn't documented well anywhere!! I'll fix that if I can get this working.

I've now added generating a pre_image secrets.token_hex(32) and then doing an SHA256 Hash of that. I'm very unsure of where I should be b64 encoding things.

I feel this should be much easier than it is especially considering how easy it is to do with lncli. I've tried to figure out what the Go is doing but I've never really looked a Go before.

I'm now getting output like this:

{
  "payment_error": "incorrect_payment_details",
  "payment_preimage": "",
  "payment_route": null,
  "payment_hash": "8dd4424cb2ec5eae55108aeee1285d212e8942f0fdc="
}

async def local_pay_keysend(
    amt: int, custom_record: KeysendCustomRecord, dest_pubkey: str = ""
):
    """Pay a keysend invoice using the chosen node"""
    node = "https://umbrel.local:8080/"
    headers, cert = get_lnd_headers_cert(admin=True, node=node)
    if not dest_pubkey:
        dest_pubkey = my_voltage_public_key

    # Base 64 encoded destination bytes
    dest = b64_hex_transform(dest_pubkey)
    cust_rec_str = json.dumps(custom_record.dict(exclude_unset=True, exclude_none=True))

    pre_image = token_hex(32)
    payment_hash = sha256(pre_image.encode())

    dest_custom_records = {
        5482373484: b64_transform(pre_image),
        7629169: b64_transform(cust_rec_str),
        818818: b64_transform("blocktvnews"),
    }

    url = f"{node}v1/channels/transactions"
    data = {
        "dest": dest,
        "amt": amt,
        "payment_hash": payment_hash.hexdigest(),
        "dest_custom_records": dest_custom_records,
    }
    # data = {"dest": dest, "amt": amt}

    async with httpx.AsyncClient(verify=cert) as client:
        response = await client.post(url=url, headers=headers, data=json.dumps(data))

    print(json.dumps(response.json(), indent=2))
    return json.dumps(response.json(), indent=2)

[guggero]

The keysend BLIP is documented here: https://github.com/lightning/blips/blob/master/blip-0003.md

I'm not really a Python user, but it sounds to me like you need to use b64_transform(pre_image.encode()), instead of b64_transform(pre_image),? Otherwise it's base64 encoding the hex instead of the raw bytes?

We definitely need more documentation around this, I agree. So a PR to the docs would be much appreciated once you get it working!

[brianoflondon]

I feel like the head of a team of monkeys that just finished the last page typing out Shakespear's complete works!

I got it to work..... :-) Now if you can point me to a good place in the docs I'll figure out how to explain this and make it easy becasue honestly this is the 3rd time I've attempted this over the last 6 months and I've given up every other time and just used LNPay's API which is buttery smooth and easy.

    pre_image = token_hex(32)
    pre_image = "7e85e6bb1fdccd04c1085f1724ee11ebbe32ab6ff9411173fe53032635f37562"
    payment_hash = sha256(bytes.fromhex(pre_image))
    payment_hash_should_be = "972f041144a5d97d081360f2e8cb84f3bd569d93a1256a4cf721566e553f5ab0"
    print(payment_hash.hexdigest() == payment_hash_should_be)

and I only had to re-use the pre-image in the docs once to make sure it was hashing the right way....

def b64_hex_transform(plain_str: str) -> str:
    """Returns the b64 transformed version of a hex string"""
    a_string = bytes.fromhex(plain_str)
    return base64.b64encode(a_string).decode()


def b64_transform(plain_str: str) -> str:
    """Returns the b64 transformed version of a string"""
    return base64.b64encode(plain_str.encode()).decode()


async def local_pay_keysend(
    amt: int, custom_record: KeysendCustomRecord, dest_pubkey: str = ""
):
    """Pay a keysend invoice using the chosen node"""
    node = "https://umbrel.local:8080/"
    headers, cert = get_lnd_headers_cert(admin=True, node=node)
    if not dest_pubkey:
        dest_pubkey = my_voltage_public_key

    # Base 64 encoded destination bytes
    dest = b64_hex_transform(dest_pubkey)
    cust_rec_str = json.dumps(custom_record.dict(exclude_unset=True, exclude_none=True))

    pre_image = token_hex(32)
    payment_hash = sha256(bytes.fromhex(pre_image))

    dest_custom_records = {
        5482373484: b64_hex_transform(pre_image),
        7629169: b64_transform(cust_rec_str),
        818818: b64_transform("blocktvnews"),
    }

    url = f"{node}v1/channels/transactions"
    data = {
        "dest": dest,
        "amt": amt,
        "payment_hash": b64_hex_transform(payment_hash.hexdigest()),
        "dest_custom_records": dest_custom_records,
    }
    # data = {"dest": dest, "amt": amt}

    async with httpx.AsyncClient(verify=cert) as client:
        response = await client.post(url=url, headers=headers, data=json.dumps(data))

    print(json.dumps(response.json(), indent=2))
    return json.dumps(response.json(), indent=2)
  

[guggero]

Cool! I think a great place for a code example would be here: https://docs.lightning.engineering/lightning-network-tools/lnd/send-messages-with-keysend
The source for that repo can be found here: https://github.com/lightninglabs/docs.lightning.engineering/

Another option would be to create a keysend specific new document here: https://github.com/lightningnetwork/lnd/tree/master/docs/rest

Up to you. I think you could even do both if you wanted, to increase the chance of people finding it (could be the same content in both places, more or less).

[brianoflondon]

I put in a pull request https://github.com/lightningnetwork/lnd/tree/master/docs/rest

I'm not really hugely experienced at working with other people and big projects so forgive me. The style is probably a bit chatty.

#6360

[guggero]

Thanks a lot! Really appreciate your PR. Going to take a closer look today. One thing I noticed when scanning the PR is: We normally don't write docs in the "I/me" form, but either the "we" or passive form, so it sounds a bit more generic and less personal. Are you up to re-writing the text or should I ask our technical writer to create an article in docs.lightning.engineering out of your example?

[brianoflondon]

I'll try to de-personalise it in the next day or two.

[okemechris]

Hi @brianoflondon @guggero I have been trying to do same with Javascript but I keep running into the same issue with the hash length

{"code":2,"message":"invalid hash length of 36, want 32","details":[]}

Here is my sample code

let dest_pubkey_base64 = Buffer.from(state.destination, 'hex').toString('base64');
const secret = crypto.randomBytes(32);
console.log('secret: ' + secret.toString('hex'));
const hash = crypto.createHash('sha256').update(secret).digest();
console.log('hash: ' + hash);

const req = {
        "dest": dest_pubkey_base64,
        "amt": state.amount,
        "payment_hash": hash.toString("base64"),
        "timeout_seconds": 20,
    }

Any help will be appreciated, thanks

[guggero]

Hmm, looks correct. Can you try crypto.createHash('sha256').update(secret).digest('base64'); instead and paste an example output you get here (from the console.log('hash: ' + hash);?
Then further below you should be able to just use "payment_hash": hash,.

[okemechris]

I tried with crypto.createHash('sha256').update(secret).digest('base64');
output of console.log('hash: ' + hash); is n+mn+VbLeg0zMwfgK8wWQOGwK7uGaGeNgIQbiOLR1Ek=
but it still returns the same error {"code":2,"message":"invalid hash length of 36, want 32","details":[]}

[guggero]

What REST URL are you using? The hash looks correct. Are you doing a POST to v1/channels/transactions?

[okemechris]

Yes

[guggero]

Something's off. Check every detail again. Because it's weird, that URL shouldn't even produce the invalid hash length of 36... error when looking through the code...

[ok300]

Thanks for the great work @brianoflondon!

I was working on sending keysend payments with the LND gRPC interface. I ran into the same problem as you did above, and your solution worked for me too.