TestFramework: TestShell.reset() will always fail

[edilmedeiros]

TestShell documentation mentions a TestShell.reset() method.

When experimenting with it, it will always fail with:

Traceback (most recent call last):
  File "/Users/jose.edil/2-development/bitcoin/vinteum/foss-program/infra-signet-server/signet-server.py", line 169, in <module>
    TestShell().reset()
  File "/Users/jose.edil/2-development/bitcoin/bitcoin-core-build/test/functional/test_framework/test_shell.py", line 64, in reset
    super().__init__()
TypeError: BitcoinTestFramework.__init__() missing 1 required positional argument: 'test_file'

Indeed, inspecting the source code, we see that TestShell is a wrapper class for BitcoinTestFramework. TestShell.reset() will call super().__init__() which refers to BitcoinTestFramework.__init__().

But we only have def __init__(self, test_file) -> None: which will require a mandatory test_file parameter. (https://github.com/bitcoin/bitcoin/blob/28ce159bc327e6dfec34077ff2e379b23a95db65/test/functional/test_framework/test_framework.py#L95C5-L95C43).

Not sure if TestShell.reset() should change to receive this parameter and pass along to the __init__() function it calls internally or any other workaround. In my specific case, I can't see what I can pass as test_file.

[brunoerg]

Thanks for it! I think this is similar to #30714 (which fixed the initialization).
cc: @theStack

[edilmedeiros]

Looks similar, indeed, but I think #30714 did not covered TestShell().setup().

[theStack]

It seems that the TestShell instructions generally don't work anymore since we switched to CMake. For me they fail already earlier at the TestShell instantiation, as the generated config.ini file is searched in a path that still assumes in-tree builds:

$ git rev-parse HEAD
ffe4261cb0669b1e1a926638e0498ae5b63f3599
$ python3
Python 3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.insert(0, "./test/functional")
>>> from test_framework.test_shell import TestShell
>>> test = TestShell().setup(num_nodes=2, setup_clean_chain=True)
tests_directory = /home/fas/bitcoin/test/functional
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/fas/bitcoin/./test/functional/test_framework/test_shell.py", line 79, in __new__
    TestShell.instance = TestShell.__TestShell(tests_directory / "testshell_dummy.py")
  File "/home/fas/bitcoin/./test/functional/test_framework/test_framework.py", line 106, in __init__
    self.parse_args(test_file)
  File "/home/fas/bitcoin/./test/functional/test_framework/test_framework.py", line 210, in parse_args
    config.read_file(open(self.options.configfile))
FileNotFoundError: [Errno 2] No such file or directory: '/home/fas/bitcoin/test/config.ini'

I assume that this was not noticed for some time as users probably had left-over files around from previous autotools builds (or, no-one just tried it after the switch to CMake, which was merged after the last TestShell fix #30714). A first naive try to fix this by adding the out-of-tree path "./build/test/functional" to sys.path instead of the in-tree one didn't help either.