Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BIP-325: Signet support #16411

Closed
wants to merge 18 commits into from
Closed

BIP-325: Signet support #16411

wants to merge 18 commits into from

Conversation

@kallewoof
Copy link
Member

@kallewoof kallewoof commented Jul 18, 2019

This PR implements BIP 325 (https://github.com/bitcoin/bips/blob/master/bip-0325.mediawiki) -- Signet support.

Note: This PR is being split into more bite-sized PR's, the first of which is #18267.

Merged check boxes:

  • Signet consensus: #18267
  • Signet RPC tools: pending consensus merge
  • Signet utility scripts (contrib/signet): pending RPC tool merge

Signet is a proposed new type of test network that requires a signature in the blocks. While this kind of network is centralized, it allows for properties that are desirable in a test network, which testnet alone cannot provide.

Signet has a default global testnet baked in for easy usage, but is built around the concept of there being multiple simultaneous signets. Someone is already working on a signet with bip-taproot etc patched on top of it. When new features are tested globally, starting a temporary signet for that purpose is trivial.

See also: https://en.bitcoin.it/wiki/Signet

Noteworthy:

  • proof of work check now skips genesis block for all networks, including mainnet
  • a new global "is signet" flag is added, which affects consensus: it adds a new enforcement that all blocks must have a valid signature for the "signet block script", a global script
  • DecodeHexBlk will now attempt to load the block data from a file whose filename is the hex string argument, if it turns out to not be a valid hex string; this has the odd caveat of the relative directory being relative to the execution of bitcoind, not bitcoin-cli (see contrib/signet/mkblock.sh)

Update: proof of work check is back in place; instead, Signet now requires an additional -signet_genesisnonce parameter to operate.

@junderw
Copy link

@junderw junderw commented Jul 18, 2019

Brainstorming some things we should consider:

1. Does adding this put anyone at risk?
2. Does adding this somehow hurt Bitcoin's code base?
3. Does adding this require some long term maintenance / increase the weight on devs for support in the future?
  1. Since Testnet has been used at some points to scam people out of money, I could see how this could be used in a similar way. However, with something like this it would be easier for someone to create a signet and give it value (aka use it to create a permission-ed altcoin and mooch 100% off Bitcoin Core development. At least with current altcoin, they have a "keep up to date with merging Core patches" barrier of entry... this would not have such a barrier.)
  2. I would say no. But that would need more in-depth technical review.
  3. If scams run rampant in my scenario in number 1, some people might propose hard coding blacklists that will require scammers to maintain patches removing themselves from the signet blacklist etc... But other than that, I don't see much problem.

I am leaning towards concept ACK. But I am slightly concerned about someone abusing this like testnet has been abused in the past, and how signet's differences make those abuses easier or harder.

Loading

@kallewoof
Copy link
Member Author

@kallewoof kallewoof commented Jul 18, 2019

@junderw

Thanks for feedback, comments below:

1. Does adding this put anyone at risk?

Example?

2. Does adding this somehow hurt Bitcoin's code base?

There is a tiny amount of risk involved as it touches consensus validation code, but this is a tiny code change.

If for some reason someone managed to flip the "signet" flag on a running node, for example, that would cause those nodes to start rejecting bitcoin blocks. I don't foresee this as very likely, personally.

3. Does adding this require some long term maintenance / increase the weight on devs for support in the future?

Yes, same as adding any feature to a system. I assume the point of your question is if the added maintenance is worth it; in my opinion it is. Signet is fairly self-contained (3 main parts: signet, wallet RPC, miner RPC, and some sprinkled stuff; there are also the contrib/signet scripts but I see those as irrelevant in this case as they're not a part of the C++ codebase), so it shouldn't be that hard to maintain.

4. Since Testnet has been used at some points to scam people out of money, I could see how this could be used in a similar way. However, with something like this it would be easier for someone to create a signet and give it value (aka use it to create a permission-ed altcoin and mooch 100% off Bitcoin Core development. At least with current altcoin, they have a "keep up to date with merging Core patches" barrier of entry... this would not have such a barrier.)

I don't think this is an issue, to be honest. People can already copy the codebase and tweak it to make a new coin, or start up a regtest and ask their victim to connect to it using This Special Bitcoin.conf File [tm] which so happens to have regtest=1 in it.

Loading

@junderw
Copy link

@junderw junderw commented Jul 18, 2019

People can already copy the codebase and tweak it to make a new coin, or start up a regtest and ask their victim to connect to it using This Special Bitcoin.conf File [tm] which so happens to have regtest=1 in it.

Which is why I am mostly leaning toward concept ACK.

But tbh, if someone said "hey, we have a 7-of-13 multisig based faster Bitcoin!" as compared to "hey use this regtest thing, totally decentralized, we swear!" could be argued by pedantic people as "not technically false"... so a regtest based scam has less of a chance of someone understanding the tech aspect saying "yeah, that's not centralized" than a signet based scam.

Also, there are also many scams that copy-pasted the code base, sure... but with a signet based scam, they don't even need to maintain a patch. Just tell the victims to download Bitcoin Core. And it works out of the box (with a conf tweak)...

So while the risk is super low, it is objectively higher than regtest and testnet...

That being said. The benefit is great compared to that risk. Which, again, is why I say "leaning towards concept ACK."

It would be very powerful to have someone who already has Bitcoin Core downloaded and running to spin up another instance with some args that lets them try out taproot etc etc...

Loading

@kallewoof
Copy link
Member Author

@kallewoof kallewoof commented Jul 18, 2019

Yep, I see what you're saying. Ultimately both cases require a conf tweak, though (regtest or testnet scam vs signet scam), and signet case requires them to give you a big ugly signet challenge and signet seed node(s), so I'm not sure how viable the attack really is in the end.

Loading

@NicolasDorier
Copy link
Contributor

@NicolasDorier NicolasDorier commented Jul 18, 2019

Concept ACK, though I am unsure if signet should be configurable at all. (via signet_blockscript for example)

It should just be a shared public network, an alternative (or replacement) of testnet.

Loading

@DrahtBot
Copy link
Contributor

@DrahtBot DrahtBot commented Jul 18, 2019

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #18267 (BIP-325: Signet [consensus] by kallewoof)
  • #18002 (Abstract out script execution out of VerifyWitnessProgram() by sipa)
  • #17977 ([WIP] Implement BIP 340-342 validation (Schnorr/taproot/tapscript) by sipa)
  • #17693 (rpc: Add generateblock to mine a custom set of transactions by andrewtoth)
  • #17037 (Testschains: Many regtests with different genesis and default datadir by jtimon)
  • #17032 (Tests: Chainparams: Make regtest almost fully customizable by jtimon)
  • #16653 (script: add simple signature support (checker/creator) by kallewoof)
  • #15761 (Replace -upgradewallet startup option with upgradewallet RPC by achow101)
  • #15481 (Restrict timestamp when mining a diff-adjustment block to prev-600 by TheBlueMatt)
  • #13062 (Make script interpreter independent from storage type CScript by sipa)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

Loading

@NicolasDorier
Copy link
Contributor

@NicolasDorier NicolasDorier commented Jul 18, 2019

The code does not seems to be that much to maintain, I would advise though to open a separate pull requests for most changes in validation.cpp/h.

Loading

@laanwj laanwj added this to Chasing Concept ACK in High-priority for review Jul 18, 2019
@kallewoof kallewoof force-pushed the signet branch 5 times, most recently from 8789b74 to 587526d Jul 19, 2019
if (!CheckProofOfWork(block.GetHash(), block.nBits, consensusParams))
return error("ReadBlockFromDisk: Errors in block header at %s", pos.ToString());
// We skip POW checks for genesis block
if (block.GetHash() != consensusParams.hashGenesisBlock) {
Copy link
Contributor

@NicolasDorier NicolasDorier Jul 19, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would move this test inside CheckProofOfWork

Loading

Copy link
Member Author

@kallewoof kallewoof Jul 19, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need this if block for CheckBlockSolution too, though.

bitcoin/src/validation.cpp

Lines 945 to 952 in 587526d

// We skip POW checks for genesis block
if (block.GetHash() != consensusParams.hashGenesisBlock) {
// Check the header
if (!CheckProofOfWork(block.GetHash(), block.nBits, consensusParams))
return error("ReadBlockFromDisk: Errors in block header at %s", pos.ToString());
if (g_signet_blocks && !CheckBlockSolution(block, consensusParams)) return false; /* function calls error(..) */
}

Loading

if (!CheckBlockHeader(block, state, consensusParams, fCheckPOW))
return false;
// We skip POW checks for genesis block
if (block.GetHash() != consensusParams.hashGenesisBlock) {
Copy link
Contributor

@NicolasDorier NicolasDorier Jul 19, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would move this test inside CheckBlockHeader

Loading

Copy link
Member Author

@kallewoof kallewoof Jul 19, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need this if block for CheckBlockSolution too, though.

bitcoin/src/validation.cpp

Lines 2974 to 2982 in 587526d

// We skip POW checks for genesis block
if (block.GetHash() != consensusParams.hashGenesisBlock) {
// Check that the header is valid (particularly PoW). This is mostly
// redundant with the call in AcceptBlockHeader.
if (!CheckBlockHeader(block, state, consensusParams, fCheckPOW))
return false;
if (g_signet_blocks && fCheckPOW && !CheckBlockSolution(block, consensusParams)) return false;
}

Loading

Copy link
Contributor

@NicolasDorier NicolasDorier Jul 19, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CheckBlockSolution could also return true if block.GetHash() == consensusParams.hashGenesisBlock

Loading

Copy link
Member Author

@kallewoof kallewoof Jul 19, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fair. But the check appears in 3 places instead of 2. Not sure why that is better, tbh.

Edit: to clarify, moving the check to the three called-to functions means the check will happen 3 times in 3 different places, compared to 2 times in 2 places, as is the case right now.

Loading

@MarcoFalke
Copy link
Member

@MarcoFalke MarcoFalke commented Jul 19, 2019

Would be nice to have a contrib/signet/README.md to explain what those sh scripts are supposed to be doing

Loading

@kallewoof
Copy link
Member Author

@kallewoof kallewoof commented Jul 20, 2019

@MarcoFalke Added README file. Also missing release notes. Since it is Needs Concept Review stage, I am not gonna rush that just yet.

Loading

@kallewoof
Copy link
Member Author

@kallewoof kallewoof commented Jul 20, 2019

I added a "noteworthy" section to the OP of this PR, which points out a number of changes to existing functionality.

Loading

@practicalswift
Copy link
Contributor

@practicalswift practicalswift commented Jul 20, 2019

Concept ACK

Thanks for doing this - would be very useful for testing.

Loading

@kallewoof kallewoof force-pushed the signet branch 3 times, most recently from 43cfbfb to ac9da7d Jul 30, 2019

A script to generate one Signet block.

Syntax: `mkblock.sh <bitcoin-cli path> [<bitcoin-cli args>]`
Copy link
Member

@MarcoFalke MarcoFalke Jul 30, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this supposed to work? It looks like the chainparams are setting the pow to the pow of mainnet. This script calls grindblock, which is a non-optimized while loop on a single CPU core. Could this lead to the chain stalling?

Loading

Copy link

@junderw junderw Jul 30, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the signers compete with each other, or frequently change their hashpower, yes.

I think the idea is that signatures limit the miner pool to a few people, then those people each pledge a consistent low hashpower to the network.

Prevents people from hopping on with ASICs, driving up difficulty and then disappearing.

Loading

Copy link
Member Author

@kallewoof kallewoof Jul 31, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pow is not really the pow of mainnet unless you don't pass -signet or don't use a -datadir with signet in bitcoin.conf.

grindblock is basically equivalent to generate*. You can use other miners like existing CPU miners or even ASICs (though you can't use the extra nonce so you'd have to resign the block instead).

Loading

kallewoof added 4 commits Mar 6, 2020
Also adds a condition to test initialization code to not generate blocks for signet chains.
Running with -signet without any other parameters will use this network's parameters.
@DrahtBot
Copy link
Contributor

@DrahtBot DrahtBot commented Mar 11, 2020

🐙 This pull request conflicts with the target branch and needs rebase.

Loading

@jtimon
Copy link
Member

@jtimon jtimon commented Mar 11, 2020

But you need them to have different genesis blocks. For example, the genesis block is the chain_id in lightning networks.

@jtimon per bolt-00 you don't have to use the genesis block as the chain id, so I think it should be fine to use block 1 as the chain id for testing lightning on signet

Well, but in that case, you would need to hardcode the second block instead of the genesis block and you have the same problem @NicolasDorier complains about, no?

I understand that some use cases don't need a chain_id or anything like that, and for those it is fine to have the same the same genesis hash and they don't even care about the second block hash because they won't hardcode that either.
But for lightning, I think you need a chain_id.

Supporting multiple chains doesn't require external software to hardcode every new chain though, that's just a design decision.
Just like bitcoin core generates the hash in a deterministic way, other software can too.
Instead of implementing only a few hardcoded signets or only the default one, external software could implement them all possible signets at once, just like bitcoin core does.

For this, it would help to not mine the genesis block and thus not need to handle the nonce as configurable anywhere, but that's orthogonal.
Sorry to insist, but can we discuss what are the advantages of grinding the genesis block again?

Back to @NicolasDorier 's proposal of committing the challenge to the second block instead of the first one, I'm all fine with that, it's just that I also want to generate chains that do have different genesis blocks when I want to. This could be done, for example, by hashing the network name.
People wanting the same genesis hash could all just leave the default "signet" while others could customize different names, resulting in different genesis blocks, like in #17037

This again would be simpler if the genesis block wasn't mined.

Or perhaps we could even allow both having the challenge being committed on the genesis block or on the second one as a configuration option?

Loading

@kallewoof
Copy link
Member Author

@kallewoof kallewoof commented Mar 12, 2020

I just want to add to this that, with the hard coded genesis block, all software will still need to re-generate the magic number (message header / message start / whatever you wanna call it) when they switch signets. I think this is significantly easier than a dynamic genesis block hash, but it's still not "plug-and-play" to switch to a different signet. Ping @NicolasDorier.

Loading

@kallewoof
Copy link
Member Author

@kallewoof kallewoof commented Mar 24, 2020

Closing in favor of #18267.

Loading

@kallewoof kallewoof closed this Mar 24, 2020
laanwj added a commit that referenced this issue Sep 21, 2020
8258c4c test: some sanity checks for consensus logic (Anthony Towns)
e47ad37 test: basic signet tests (Karl-Johan Alm)
4c189ab test: add small signet fuzzer (practicalswift)
ec9b25d test: signet network selection tests (Karl-Johan Alm)
3efe298 signet: hard-coded parameters for Signet Global Network VI (2020-09-07) (Karl-Johan Alm)
c7898bc qt: update QT to support signet network (Karl-Johan Alm)
a8de47a consensus: add signet validation (Karl-Johan Alm)
e8990f1 add signet chain and accompanying parameters (Karl-Johan Alm)
404682b add signet basic support (signet.cpp) (Karl-Johan Alm)
a2147d7 validation: move GetWitnessCommitmentIndex to consensus/validation (Karl-Johan Alm)

Pull request description:

  This PR is a part of BIP-325 (https://github.com/bitcoin/bips/blob/master/bip-0325.mediawiki), and is a sub-PR of #16411.

  * Signet consensus (this)
  * Signet RPC tools (pending)
  * Signet utility scripts (contrib/signet) (pending)

ACKs for top commit:
  jonatack:
    re-ACK 8258c4c per `git diff dbeea65 8258c4c`, only change since last review is updated `-signet*` config option naming.
  fjahr:
    re-ACK 8258c4c
  laanwj:
    ACK 8258c4c
  MarcoFalke:
    Approach ACK 8258c4c 🌵

Tree-SHA512: 5d158add96755910837feafa8214e13695b769a6aec3a2da753cf672618bef377fac43b0f4b772a87b25dd9f0c1c9b29f2789785d7a7d47a155cdcf48f7c975d
sidhujag added a commit to syscoin/syscoin that referenced this issue Sep 22, 2020
8258c4c test: some sanity checks for consensus logic (Anthony Towns)
e47ad37 test: basic signet tests (Karl-Johan Alm)
4c189ab test: add small signet fuzzer (practicalswift)
ec9b25d test: signet network selection tests (Karl-Johan Alm)
3efe298 signet: hard-coded parameters for Signet Global Network VI (2020-09-07) (Karl-Johan Alm)
c7898bc qt: update QT to support signet network (Karl-Johan Alm)
a8de47a consensus: add signet validation (Karl-Johan Alm)
e8990f1 add signet chain and accompanying parameters (Karl-Johan Alm)
404682b add signet basic support (signet.cpp) (Karl-Johan Alm)
a2147d7 validation: move GetWitnessCommitmentIndex to consensus/validation (Karl-Johan Alm)

Pull request description:

  This PR is a part of BIP-325 (https://github.com/bitcoin/bips/blob/master/bip-0325.mediawiki), and is a sub-PR of bitcoin#16411.

  * Signet consensus (this)
  * Signet RPC tools (pending)
  * Signet utility scripts (contrib/signet) (pending)

ACKs for top commit:
  jonatack:
    re-ACK 8258c4c per `git diff dbeea65 8258c4c`, only change since last review is updated `-signet*` config option naming.
  fjahr:
    re-ACK 8258c4c
  laanwj:
    ACK 8258c4c
  MarcoFalke:
    Approach ACK 8258c4c 🌵

Tree-SHA512: 5d158add96755910837feafa8214e13695b769a6aec3a2da753cf672618bef377fac43b0f4b772a87b25dd9f0c1c9b29f2789785d7a7d47a155cdcf48f7c975d
KolbyML added a commit to KolbyML/bitcoin that referenced this issue Jan 17, 2021
… "regtest"

faf3683 test: Avoid hardcoding the chain name in combine_logs (MarcoFalke)
fa8a1d7 test: Adapt test framework for chains other than "regtest" (MarcoFalke)
68f5466 test: Fix “local variable 'e' is assigned to but never used” (Ben Woosley)

Pull request description:

  This is required for various work in progress:

  * testchains bitcoin#8994
  * signet bitcoin#16411
  * some of my locally written tests

  While it will be unused in the master branch as of now, it will make all of those pull requests shorter. Thus review for non-regtest tests can focus on the actual changes and not some test framework changes.

ACKs for top commit:
  jonatack:
    ACK faf3683, ran tests and reviewed the code.

Tree-SHA512: 35add66c12cab68f2fac8f7c7d47c604d3f24eae9336ff78f83e2c92b3dc08a25e7f4217199bac5393dd3fb72f945bba9c001d6fbb8efd298c88858075fcb3d6
KolbyML added a commit to KolbyML/bitcoin that referenced this issue Jan 17, 2021
… "regtest"

faf3683 test: Avoid hardcoding the chain name in combine_logs (MarcoFalke)
fa8a1d7 test: Adapt test framework for chains other than "regtest" (MarcoFalke)
68f5466 test: Fix “local variable 'e' is assigned to but never used” (Ben Woosley)

Pull request description:

  This is required for various work in progress:

  * testchains bitcoin#8994
  * signet bitcoin#16411
  * some of my locally written tests

  While it will be unused in the master branch as of now, it will make all of those pull requests shorter. Thus review for non-regtest tests can focus on the actual changes and not some test framework changes.

ACKs for top commit:
  jonatack:
    ACK faf3683, ran tests and reviewed the code.

Tree-SHA512: 35add66c12cab68f2fac8f7c7d47c604d3f24eae9336ff78f83e2c92b3dc08a25e7f4217199bac5393dd3fb72f945bba9c001d6fbb8efd298c88858075fcb3d6
PastaPastaPasta pushed a commit to dashpay/dash that referenced this issue Jan 22, 2021
* Merge bitcoin#16509: test: Adapt test framework for chains other than "regtest"

faf3683 test: Avoid hardcoding the chain name in combine_logs (MarcoFalke)
fa8a1d7 test: Adapt test framework for chains other than "regtest" (MarcoFalke)
68f5466 test: Fix “local variable 'e' is assigned to but never used” (Ben Woosley)

Pull request description:

  This is required for various work in progress:

  * testchains bitcoin#8994
  * signet bitcoin#16411
  * some of my locally written tests

  While it will be unused in the master branch as of now, it will make all of those pull requests shorter. Thus review for non-regtest tests can focus on the actual changes and not some test framework changes.

ACKs for top commit:
  jonatack:
    ACK faf3683, ran tests and reviewed the code.

Tree-SHA512: 35add66c12cab68f2fac8f7c7d47c604d3f24eae9336ff78f83e2c92b3dc08a25e7f4217199bac5393dd3fb72f945bba9c001d6fbb8efd298c88858075fcb3d6

* Add devnet support for tests

* test: make sure devnet can connect to each other and start

* Partial merge bitcoin#16681: Tests: Use self.chain instead of 'regtest' in almost all current tests, revert one TODO while at it

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
Co-authored-by: Jorge Timón <jtimon@jtimon.cc>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet