Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testchains: Introduce custom chain whose constructor... #8994

Closed
wants to merge 9 commits into from

Conversation

@jtimon
Copy link
Member

@jtimon jtimon commented Oct 22, 2016

...reads from runtime params and simplify the creation of partitioned chains by simply generating different gensis block hashes from a given custom name.

It also allows to customize any chain param in these custom chains (but not the other chains).

Dependencies:

  • Testschains: Many regtests with different genesis and default datadir #17037
  • Tests: Chainparams: Make regtest almost fully customizable #17032
  • Tests: Use self.chain instead of 'regtest' in all current tests #16681
  • Preparations for more testchains #16680
  • Use a proper factory for creating chainparams #8855
    - [ ] Really don't validate genesis block #9102
  • Introduce an ArgsManager class encapsulating cs_args, mapArgs and mapMultiArgs #9494
  • QA: segwit.py: s/find_unspent/find_spendable_utxo/ #11869
  • Refactor: One CBaseChainParams should be enough #12128

Other features:

  • Uses a custom chain for all python tests.
  • Create new testchains with different genesis hashes at will.
  • Load chainparams from separated file or command line. (file left for later, see https://github.com/jtimon/bitcoin/tree/b16-new-testnet-file )
  • New chains are neither orange, blue nor green: they're purple and have your custom chain petname shown in the GUI.
  • Extra context: some people asked for signed blocks but that's way more disruptive and this is already review-thirsty (see #9177 ).
@btcdrak
Copy link
Contributor

@btcdrak btcdrak commented Oct 22, 2016

Concept ACK

Loading

@btcdrak
Copy link
Contributor

@btcdrak btcdrak commented Oct 22, 2016

Some ideas from IRC today: Allow users to specify a .json file with all the parameters, and include an option for federated signing with list of signing keys.

Loading

@jtimon jtimon force-pushed the 0.13-new-testchain branch 3 times, most recently from ba59b47 to 62f928f Oct 29, 2016
@jtimon jtimon force-pushed the 0.13-new-testchain branch from 62f928f to 41207b1 Nov 3, 2016
@jtimon
Copy link
Member Author

@jtimon jtimon commented Nov 3, 2016

Needed rebase. Also, new configurable parameter was introduced in #9053

Loading

@jtimon jtimon force-pushed the 0.13-new-testchain branch 2 times, most recently from 977b564 to ba1c6c4 Nov 8, 2016
@jtimon
Copy link
Member Author

@jtimon jtimon commented Nov 8, 2016

Rebased on top of #9102

Loading

@jtimon jtimon force-pushed the 0.13-new-testchain branch 3 times, most recently from e528398 to 39ac540 Nov 14, 2016
@jtimon
Copy link
Member Author

@jtimon jtimon commented Nov 14, 2016

Since #8855 needed rebase, this one too. Also adapted some more rpc tests to use the custom chain instead of regtest (only 4 missing it seems, but travis should pass).

Loading

@jtimon jtimon force-pushed the 0.13-new-testchain branch from 39ac540 to d71e013 Nov 15, 2016
@jtimon jtimon force-pushed the 0.13-new-testchain branch from a2477b9 to 9c76ffb Nov 18, 2016
@jtimon jtimon force-pushed the 0.13-new-testchain branch 3 times, most recently from fa6547a to 31b68a9 Dec 2, 2016
@jtimon
Copy link
Member Author

@jtimon jtimon commented Dec 2, 2016

Needed rebase.

Update:

  • Now all rpc tests pass for self.chain="regtest"

"Magic" fixes:

  • pruning.py and mempool_packages.py now pass with both regtest and custom (they were passing travis before because they are in the extended set of tests).

"Magic" fails:

  • Now p2p-segwit.py only passes with regtest but not with custom like before

This is related to #9102 (see #9102 (comment) ping @laanwj ) in the sense that we cannot test that the genesis block is not validated unless we can run the system for a chain whose genesis block doesn't comply with the rules.
For example, the custom chain (unlike regtest) doesn't comply with pow (although some values for -chainpetname should make it).

@gmaxwell, you suggested that something like #9177 would need a lot of coverage testing and review, and I completely agree. Any suggestion to advance in that front in this PR instead of #9177 (which currently doesn't work and has one unittest that is not independent) would be welcomed.

You have recently made improvements in the rpc test framework, @MarcoFalke , and are familiar with it. If you get bored and can help fix one of the 3 tests that aren't passing for custom or just comment about the python changes in general, that would be great.

REM:

  • The last 2 commits are not necessary, but anyone feel free to comment on them.
  • Except for p2p-compactblocks.py, segwit.py (and now p2p-segwit.py too) all rpc tests pass with "custom" too

Loading

@jtimon
Copy link
Member Author

@jtimon jtimon commented Dec 2, 2016

@TheBlueMatt said in #9243 :

I think for things which are their own classes, they should either have the arguments they care about passed into the constructor, or passed in when they need them (eg the way mempool limiting is done now)

The longer version of CreateChainParams could indeed take all fields in CChainParams and Consensus::Params as arguments, but that would break the encapsulation. Every time a field is added or removed all calls would need to be adapted. Note that ideally, in the future, all tests would rely on CreateChainParams but none of them on the chainparams globals (accessed to via SelectParams and Params() ).

Another simpler option would be to simply let chainparams.cpp depend on globals mapArgs and mapMultiArgs (it depends on util.o anyway).

Yet another option is to undo some of that changes in #9243 if that gets merged first, which would be my personal preference.

If anyone has more options, I'm all ears.

Loading

@jtimon jtimon force-pushed the 0.13-new-testchain branch from 31b68a9 to 17528b6 Dec 3, 2016
@jtimon
Copy link
Member Author

@jtimon jtimon commented Dec 3, 2016

Rebased

Loading

@jtimon jtimon force-pushed the 0.13-new-testchain branch from 17528b6 to 8cb1b85 Dec 20, 2016
@jtimon
Copy link
Member Author

@jtimon jtimon commented Dec 20, 2016

Rebased

Loading

@jtimon jtimon force-pushed the 0.13-new-testchain branch from afaf7c9 to 797230a Oct 3, 2019
@jtimon
Copy link
Member Author

@jtimon jtimon commented Oct 3, 2019

Rebased on top of separated #17037

Loading

@jtimon jtimon force-pushed the 0.13-new-testchain branch 3 times, most recently from 59b7195 to 99343d1 Oct 5, 2019
@jtimon jtimon force-pushed the 0.13-new-testchain branch from 99343d1 to 0da5e72 Oct 8, 2019
@jtimon
Copy link
Member Author

@jtimon jtimon commented Oct 8, 2019

Rebased on top of rebased and modified #16681 , one less commit in total, but the total diff it's the same.

Loading

@jtimon jtimon mentioned this pull request Oct 8, 2019
3 tasks
@jtimon
Copy link
Member Author

@jtimon jtimon commented Oct 9, 2019

Since #16681 was closed, closing this too. There's also more discussions there with people saying they don't like the idea of moving the test framework default chain from "regtest" to "regtest2".

Loading

@jtimon jtimon closed this Oct 9, 2019
MarcoFalke added a commit that referenced this issue Feb 4, 2020
…nt tests

1abcecc Tests: Use self.chain instead of 'regtest' in almost all current tests (Jorge Timón)

Pull request description:

  Simply avoiding the hardcoded string in more places for consistency.
  It can also allow for more easily reusing tests for other chains other than regtest.

  Separated from #8994 .
  Continues #16509 .

  It is still not complete (ie to be complete, we need the -chain parameter in #16680 and make whether acceptnonstdtxs is allowed for that chain or not customizable for regtest [or for custom chains like in #8994 ] ). But while being incomplete like #16509 , it's quite simple to review and another step forward IMO.

ACKs for top commit:
  Sjors:
    re-ACK 1abcecc. I think it's an improvement even if incomplete and if some PR's might accidentally bring "regtest" back. Subsequent improvements hopefully don't have to touch 16 files.
  elichai:
    Code review ACK 1abcecc
  ryanofsky:
    Code review ACK 1abcecc.
  ryanofsky:
    Code review ACK 1abcecc

Tree-SHA512: 5620de6dab235ca8bd8670d6366c7b9f04f0e3ca9c5e7f87765b38e16ed80c17d7d1630c0d5fd7c5526f070830d94dc74cc2096d8ede87dc7180ed20569509ee
sidhujag added a commit to syscoin/syscoin that referenced this issue Feb 9, 2020
…l current tests

1abcecc Tests: Use self.chain instead of 'regtest' in almost all current tests (Jorge Timón)

Pull request description:

  Simply avoiding the hardcoded string in more places for consistency.
  It can also allow for more easily reusing tests for other chains other than regtest.

  Separated from bitcoin#8994 .
  Continues bitcoin#16509 .

  It is still not complete (ie to be complete, we need the -chain parameter in bitcoin#16680 and make whether acceptnonstdtxs is allowed for that chain or not customizable for regtest [or for custom chains like in bitcoin#8994 ] ). But while being incomplete like bitcoin#16509 , it's quite simple to review and another step forward IMO.

ACKs for top commit:
  Sjors:
    re-ACK 1abcecc. I think it's an improvement even if incomplete and if some PR's might accidentally bring "regtest" back. Subsequent improvements hopefully don't have to touch 16 files.
  elichai:
    Code review ACK 1abcecc
  ryanofsky:
    Code review ACK 1abcecc.
  ryanofsky:
    Code review ACK 1abcecc

Tree-SHA512: 5620de6dab235ca8bd8670d6366c7b9f04f0e3ca9c5e7f87765b38e16ed80c17d7d1630c0d5fd7c5526f070830d94dc74cc2096d8ede87dc7180ed20569509ee
sidhujag added a commit to syscoin-core/syscoin that referenced this issue Nov 10, 2020
…l current tests

1abcecc Tests: Use self.chain instead of 'regtest' in almost all current tests (Jorge Timón)

Pull request description:

  Simply avoiding the hardcoded string in more places for consistency.
  It can also allow for more easily reusing tests for other chains other than regtest.

  Separated from bitcoin#8994 .
  Continues bitcoin#16509 .

  It is still not complete (ie to be complete, we need the -chain parameter in bitcoin#16680 and make whether acceptnonstdtxs is allowed for that chain or not customizable for regtest [or for custom chains like in bitcoin#8994 ] ). But while being incomplete like bitcoin#16509 , it's quite simple to review and another step forward IMO.

ACKs for top commit:
  Sjors:
    re-ACK 1abcecc. I think it's an improvement even if incomplete and if some PR's might accidentally bring "regtest" back. Subsequent improvements hopefully don't have to touch 16 files.
  elichai:
    Code review ACK 1abcecc
  ryanofsky:
    Code review ACK 1abcecc.
  ryanofsky:
    Code review ACK 1abcecc

Tree-SHA512: 5620de6dab235ca8bd8670d6366c7b9f04f0e3ca9c5e7f87765b38e16ed80c17d7d1630c0d5fd7c5526f070830d94dc74cc2096d8ede87dc7180ed20569509ee
KolbyML added a commit to KolbyML/bitcoin that referenced this issue Jan 17, 2021
… "regtest"

faf3683 test: Avoid hardcoding the chain name in combine_logs (MarcoFalke)
fa8a1d7 test: Adapt test framework for chains other than "regtest" (MarcoFalke)
68f5466 test: Fix “local variable 'e' is assigned to but never used” (Ben Woosley)

Pull request description:

  This is required for various work in progress:

  * testchains bitcoin#8994
  * signet bitcoin#16411
  * some of my locally written tests

  While it will be unused in the master branch as of now, it will make all of those pull requests shorter. Thus review for non-regtest tests can focus on the actual changes and not some test framework changes.

ACKs for top commit:
  jonatack:
    ACK faf3683, ran tests and reviewed the code.

Tree-SHA512: 35add66c12cab68f2fac8f7c7d47c604d3f24eae9336ff78f83e2c92b3dc08a25e7f4217199bac5393dd3fb72f945bba9c001d6fbb8efd298c88858075fcb3d6
KolbyML added a commit to KolbyML/bitcoin that referenced this issue Jan 17, 2021
… "regtest"

faf3683 test: Avoid hardcoding the chain name in combine_logs (MarcoFalke)
fa8a1d7 test: Adapt test framework for chains other than "regtest" (MarcoFalke)
68f5466 test: Fix “local variable 'e' is assigned to but never used” (Ben Woosley)

Pull request description:

  This is required for various work in progress:

  * testchains bitcoin#8994
  * signet bitcoin#16411
  * some of my locally written tests

  While it will be unused in the master branch as of now, it will make all of those pull requests shorter. Thus review for non-regtest tests can focus on the actual changes and not some test framework changes.

ACKs for top commit:
  jonatack:
    ACK faf3683, ran tests and reviewed the code.

Tree-SHA512: 35add66c12cab68f2fac8f7c7d47c604d3f24eae9336ff78f83e2c92b3dc08a25e7f4217199bac5393dd3fb72f945bba9c001d6fbb8efd298c88858075fcb3d6
UdjinM6 added a commit to UdjinM6/dash that referenced this issue Jan 21, 2021
…t' in all current tests

1abcecc Tests: Use self.chain instead of 'regtest' in almost all current tests (Jorge Timón)

Pull request description:

  Simply avoiding the hardcoded string in more places for consistency.
  It can also allow for more easily reusing tests for other chains other than regtest.

  Separated from bitcoin#8994 .
  Continues bitcoin#16509 .

  It is still not complete (ie to be complete, we need the -chain parameter in bitcoin#16680 and make whether acceptnonstdtxs is allowed for that chain or not customizable for regtest [or for custom chains like in bitcoin#8994 ] ). But while being incomplete like bitcoin#16509 , it's quite simple to review and another step forward IMO.

ACKs for top commit:
  Sjors:
    re-ACK 1abcecc. I think it's an improvement even if incomplete and if some PR's might accidentally bring "regtest" back. Subsequent improvements hopefully don't have to touch 16 files.
  elichai:
    Code review ACK 1abcecc
  ryanofsky:
    Code review ACK 1abcecc.
  ryanofsky:
    Code review ACK 1abcecc

Tree-SHA512: 5620de6dab235ca8bd8670d6366c7b9f04f0e3ca9c5e7f87765b38e16ed80c17d7d1630c0d5fd7c5526f070830d94dc74cc2096d8ede87dc7180ed20569509ee
PastaPastaPasta pushed a commit to dashpay/dash that referenced this issue Jan 22, 2021
* Merge bitcoin#16509: test: Adapt test framework for chains other than "regtest"

faf3683 test: Avoid hardcoding the chain name in combine_logs (MarcoFalke)
fa8a1d7 test: Adapt test framework for chains other than "regtest" (MarcoFalke)
68f5466 test: Fix “local variable 'e' is assigned to but never used” (Ben Woosley)

Pull request description:

  This is required for various work in progress:

  * testchains bitcoin#8994
  * signet bitcoin#16411
  * some of my locally written tests

  While it will be unused in the master branch as of now, it will make all of those pull requests shorter. Thus review for non-regtest tests can focus on the actual changes and not some test framework changes.

ACKs for top commit:
  jonatack:
    ACK faf3683, ran tests and reviewed the code.

Tree-SHA512: 35add66c12cab68f2fac8f7c7d47c604d3f24eae9336ff78f83e2c92b3dc08a25e7f4217199bac5393dd3fb72f945bba9c001d6fbb8efd298c88858075fcb3d6

* Add devnet support for tests

* test: make sure devnet can connect to each other and start

* Partial merge bitcoin#16681: Tests: Use self.chain instead of 'regtest' in almost all current tests, revert one TODO while at it

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
Co-authored-by: Jorge Timón <jtimon@jtimon.cc>
rkarthik2k21 added a commit to rkarthik2k21/dash that referenced this issue Aug 12, 2021
6fa901f Don't edit Chainparams after initialization (Jorge Timón)
980b38f MOVEONLY: Move versionbits info out of versionbits.o (Jorge Timón)

Pull request description:

  This encapsulates the "-vbparams" option, which is only meant for regtest, directly on CRegTestParams.

  This is a refactor and doesn't change functionality.

  Related to bitcoin#8994

Tree-SHA512: 79771d729a63a720e743a9c77d5e2d80369f072d66202a43c1304e83a7d0ef7c6103d4968a03aea9666cc89a7203c618da972124a677b38cfe62ddaeb28f9f5d
rkarthik2k21 added a commit to rkarthik2k21/dash that referenced this issue Aug 25, 2021
6fa901f Don't edit Chainparams after initialization (Jorge Timón)
980b38f MOVEONLY: Move versionbits info out of versionbits.o (Jorge Timón)

Pull request description:

  This encapsulates the "-vbparams" option, which is only meant for regtest, directly on CRegTestParams.

  This is a refactor and doesn't change functionality.

  Related to bitcoin#8994

Tree-SHA512: 79771d729a63a720e743a9c77d5e2d80369f072d66202a43c1304e83a7d0ef7c6103d4968a03aea9666cc89a7203c618da972124a677b38cfe62ddaeb28f9f5d
UdjinM6 added a commit to dashpay/dash that referenced this issue Sep 2, 2021
* Merge bitcoin#13311: Don't edit Chainparams after initialization

6fa901f Don't edit Chainparams after initialization (Jorge Timón)
980b38f MOVEONLY: Move versionbits info out of versionbits.o (Jorge Timón)

Pull request description:

  This encapsulates the "-vbparams" option, which is only meant for regtest, directly on CRegTestParams.

  This is a refactor and doesn't change functionality.

  Related to bitcoin#8994

Tree-SHA512: 79771d729a63a720e743a9c77d5e2d80369f072d66202a43c1304e83a7d0ef7c6103d4968a03aea9666cc89a7203c618da972124a677b38cfe62ddaeb28f9f5d

* Resolve Merge with bitcoin#13311

* Incorporated review changes

* Apply suggestions from code review

* Update src/chainparams.cpp

* Update src/chainparams.cpp

Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet