-
Notifications
You must be signed in to change notification settings - Fork 35.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bech32 address generation in regtest does not have the prefix tb
#12314
Comments
To distinguish regtest from testnet and mainnet, the human readable part of a bech32 address on regtest is |
Yes, this is intentional. I don't see a reason why regtest should produce addresses that can be confused with testnet. |
Agree on that. By reading the developer examples and the BIP173 i was lead to expecting a
Thanks for confirming 👍 |
There is no mention of the |
A mismatch between implementation occurred on the bech32 prefix for regtest mode. Bitcoin Core decided to use the `bcrt` prefix in order for it to be distinguishable from the testnet prefix. btcd on the other hand uses the same prefix as testnet because of lack of a separate prefix specified in this BIP. Core: bitcoin/bitcoin#12314 btcd: btcsuite/btcd#1150
regtest is a Bitcoin Core specific test network. It's not mentioned in the BIP because it's not something that affects Bitcoin. |
I do agree that regression testing should primarily be done within one repository, but it does make sense to have a similar structure for integration testing between implementations. Next to the official test network, I think it makes sense to have some standards on a single regression test network to facilitate cross-implementation testing. I realize that standards in Bitcoin currently mostly go hand in hand with Core development, but with Lightning the process has been way more diverse, creating a need for some form of standard integration testing. So, while the integration efforts could specify a custom regtest chain, it would be way easier if different implementations could already provide a compatible one. |
We tend to change the regtest parameter whenever it makes our testing or maintaining easier, so even if it is unlikely that the bech32 prefix will be changed, I am slightly -0 on adding it to the BIP. Also see #8994, which makes it easier to launch regtest like chains and thus impossible to documents all of them in the BIP. |
Is there a way to differentiate between a regtest and testnet for non bech32 addresses? |
Reasons:
There is still not too late to get rid of |
Can confirm that this has cost us several hours of troubleshooting and writing extra code. I agree with @dpc that I don't quite see the advantages of a different hrp. |
I think part of the issue here is the question whether regtest is an internal Bitcoin Core thing, or a framework for testing infrastructure on top. Historically at least, regtest was purely an internal thing for Bitcoin Core's own tests. There is no standardization, and it has undergone rule changes several times whenever it was convenient. On the other hand, testnet was explicitly designed for applications to test integration with, is dealt with in BIPs, though I can see why it may be hard to use in some cases. I don't know what the solution is. I'm opposed to adding regtest specifications to BIPs in general, as this would make it harder to adapt to changes that need testing in Core itself. We could consider changing the regtest prefix to something else; I don't think it's a strong reduction in test capability. Overall, the expectation of using regtest externally may bite us again in other situations. Maybe we need even more networks? (signet comes to mind, though global networks aren't what you always need either). |
@sipa I guess I can use this opportunity to share some thought of nature of Bitcoin "nets" from my experiences at Bitgo (thought are mine though). Please excuse if it is a bit scattered and random. I'll start with the main point:On the IDs, hashes, addresses, etc. level there's no reason to make any distinctions between testnet, regtest, simnet or whatever other non-mainnets. Conceptually the are just two types of networks: real one, and not real one. Mostly for the end user not to ever get confused. If the software needs to know if it is running against testnet vs regtest vs simnet, it can carry that information out of band - that's not a problem. It doesn't have to be embedded in every single address. Quite the opposite: having to potentially come up with additional logic in every single piece of code handling Bitcoin, to support every type of "net" that we could come up with is just a hindrance. General perspective on nets:There's mainnet - the real deal. Everybody understands mainnet. Then there's a testnet. What's not-obvious is that testnet is not a test network for companies integrating Bitcoin. It's a test network for end users of Bitcoin. It's global, shared and behaves almost exactly like a mainnet, so users can do testing of the whole ecosystem, send funds between different companies and so on. From the perspective of a company that integrates with Bitcoin, testnet is actually what I call a "prod-net" - production network, exposed to end users. When testnet systems are having problems, users report normal support cases to get it fixed, just like a mainnet prod systems. Testnet is also not really suitable for Bitcoin integration testing. It is just too slow and does not give enough control. Any practical yet extensive test suite needs to have an ability to control mining and what exactly goes on the block. Regtest is perfect for that. I don't know if it was intended for it or not, but in practice it works great. Regtest node is not a mock, fake or whatever. It's a real Bitcoin Core node. From an outside perspective of a blockchain data consumer it looks very much like a normal testnet node. But you can drive it manually like a mock. You can setup a test instance of your whole system, swap a node for a regtest node, drive it with rpc commands in your test suite and see if what comes in and out is what you expect. You don't have to wait for blocks being mined, and you can simulate stuff like reorgs. Consensus with anyone else irrelevant, because there's no way to coordinate anyway. |
For anyone in a similar situation: I eventually figured out that it is probably easier to patch bitcoin core node to just not use this custom prefix, than try to introduce and maintain all the code to support it. Indeed the fix is simple: dpc@99e6350 , though from what I can tell it will break a lot of existing tests that expect I asked around and I still haven't seen a single practical problem that different prefixes would solve, so I hope the whole idea can still be revised, especially that it is not baked into any consensus or even the BIP itself. |
I don't think the Bitcoin Core test suite cares what prefix is used for the regtest chain, so I don't mind changing it to If someone wants to pick this up, a good start would be to cherry-pick dpc@99e6350 and then fixup the test from there. |
I just hope there's an agreement on the reasoning here. I just point out that there is a substantial cost in making every downstream consumer of Bitcoin Core RPC deal specifically with minor details of distinct "testnets". If there's a good practical reason to have such a distinction that have benefits out-weighting the costs, than maybe there are alternative approaches like making it configurable, etc. But if no one has any use for it, than it seems simplest to just get rid of it. I have also not played with signet or any other potential testnet-like modes/setups, so if someone is familiar with them - it might be worthwhile to rethink if there are any cases like that there. Ideally, if a downstream consumer decides to e.g. switch from testnet to signet for a better user-experience, it would be great if that did not require any alterations. |
@dpc I don't think there is any reason why having a separate prefix for regtest is advantageous to us, but at the time it was introduced, it was also assumed to be of no cost. I had no idea people were using regtest for testing other things than Bitcoin Core itself. As I said, if there's a good reason (like you seem to have), changing it to 'tb' is fine by me. However, I am worried about encouraging people using regtest for external testing in general. We can't guarantee that regtest won't undergo radical changes as our test needs change. This makes me wonder if we shouldn't define a standard "external regtest" network instead, with an intention if it being as close as possible to testnet (or mainnet?) as possible while remaining usable for testing. PS: why 'tb' and not 'bc'? Regtest is closer to mainnet than to testnet in many aspects. |
It's not that rust-bitcoin can't work with the same prefix for multiple networks; it can: https://github.com/rust-bitcoin/rust-bitcoin/pull/291/files#diff-7769ce7612a45cca417dce18a5aa6a50R388 The problem is that when rust-bitcoin (or, I assume any other library) parses address, it needs somehow to distinguish whether |
Why? |
@dpc not sure I am getting your question. Because the code is written in that way... If you reconstruct network type from bech32 string you need to return some enum variant. |
It's very surprising to me that you at any point in time would want to decode an address without knowing what network it's for. I also don't see why that isn't a problem already - P2PKH and P2SH addresses for regtest are the same as for testnet. |
Good question, but this is problem not only from rust-bitcoin but also for c-lightning: ElementsProject/lightning#4068 (comment) and ElementsProject/lightning#4068 (comment) |
Just to confirm, at p2pkh and p2sh addresses for signet are identical to testnet? Otherwise, we have a problem with bolt11 invoices and fallback addresses:
But signet support in c-lightning was added while signet was still in flux, so maybe it doesn't use different address versions any more? |
Yes, @rustyrussell, the addresses are identical: #18267 (comment) |
I see that I'm too late to this party but for posterity I'd just like to say that I think that the decision made here was wrong because we have an hrp anyway and to decide to convey less information with these bytes makes no sense to me and complicates development (marginally) and helps no one. |
I think this issue can be closed. There seem to be valid arguments for and against the status quo (StackExchange question on the current status here) but I can't see it being changed now, over three years later since this issue was opened. Testnet and Signet addresses start |
Now that third party software likely supports duplicate prefixes, i.e. not treating the prefix as a unique network id, it could actually make sense to switch regtest over. I think it was good to hold back changing the regtest prefix before signet came out. Not saying whether it should be changed, but now seems a better time to change than it was before signet came out. |
@MarcoFalke: If the plan was for regtest, testnet and signet to all share the same prefix third party software would have had to have supported duplicate prefixes three years ago? I don't get the rationale that it potentially makes more sense today than it did three years ago. |
I have to say that this singe issue can be a massive productivity drain for people trying to build testable infrastructure on top of bitcoin core. I understand some internal tests might break, but how much work are we talking about here? I'd be willing to contribute in fixing those in my spare time if it means that we get rid of the |
I think there was software out there (citation needed) that treated bech32 address prefixes as a unique identifier for the whole chain. Now that there are two chains with the same prefix (and affected software likely removed that incorrect assumption), there is one less reason to not do the change, thus it makes more sense. |
I think it is quite the oppposite, most (if not all) bitcoin libraries today support the |
Information that, as has been stated repeatedly before is not only irrelevant, but can easily be communicated out of band. And I don't know what libraries you're using but bitcoinjs-lib is giving me trouble with that. Anyways, I don't want to over-extend an already extended thread, but keeping something that is clearly giving multiple devs more trouble than it is solving because in some hypothetical edge-case it might be useful doesn't sound like a good reason. |
Looks like bitcoinjs-lib has support for bcrt |
Add bitcoinj (https://github.com/bitcoinj/bitcoinj) to the list of implementations that use |
Closing due to lack of direction and progress. Pull requests with improvements are always welcome. |
Steps to reproduce:
By BIP173 Segwit address format
bitcoind
should return atb
prefixed address for testnet, and by developer examples, regtest is ablockchain with the same basic rules as testnet
.Expected behavior
bitcoind
should return a bech32 address with the same prefix as testnet.Bitcoind version
0.16.0rc1
The text was updated successfully, but these errors were encountered: