tools: add t8n.sh to the t8n-dump-dir for evmone, besu & ethereum-specs-evm (and various t8n tool clean-up)

[danceratopz]

Implements the t8n.sh script added in #261 for the remaining supported t8n tools (evmone-t8n, besu t8n-server, ethereum-specs-evm t8n). The script gets created to the --t8n-dump-dir if specified on the command-line, e.g.,

fill --fork=Shanghai --traces --t8n-dump-dir=/tmp/geth-t8n

This PR also contains quite a lot of other small improvements and clean-up across the transition tools. Here's a quick overview:

1. Adds file extensions to all the files created in the t8n dump dir.
2. Writes input and output files to input/output subdirectories.
3. The result ofoutput.body is now also written to txs.rlp in the dump dir. Except ethereum-spec-evm cf Add output.body/txs rlp support for ethereum-spec-evm  #268.
4. The default evaluate() method (used by geth, nimbus, ethereum-specs-evm) no longer creates a temporary directory (unless tracing is enabled). All input/output is handled by stdin/stdout, so the directory is not required.
5. A default collect_traces() method has been added.
6. Besu t8n-server:
   i. Print the request's text.response in the test failure (by raising an exception in evaluate()) if the response is invalid.
   ii. Add more information about the request to the t8n dump dir (response.text, time elapsed, status code).
7. ethereum-spec-evm t8n: Required a few changes to get it running, see associated commit messages which reference the issues found; most of these are very small problems that should be easy to solve, but are out of the scope of this PR.
8. Add a small section to the docs describing --t8n-dump-dir and t8n.sh, preview:
   https://danceratopz.github.io/execution-spec-tests/main/getting_started/debugging_t8n_tools/

[danceratopz]

Force pushed to include #271.

[danceratopz]

Marked as draft: It's probably worth waiting for ethereum/execution-specs#823 as this will remove the additional workaround logic added in 32ac3bf when the transition tool is ethereum-specs-evm.

[marioevz]

Looks good overall, just a few comments 👍

[spencer-tb]

Super awesome addition!

Tried with all the evm t8n tools (minus nimbus):

• Besu works great (took me a while to figure out the server part.. :P)
• Evmone: note the comment on adding output via t8n.sh, and needs evmone: Fix paths' values for t8n danceratopz/execution-spec-tests#23 merged into your branch.
• Execution specs works well with your changes :D

Happy for you to merge when the extra changes for evmone/execution-specs are in!

[danceratopz]

@spencer-tb @marioevz thanks a lot for your fixes and suggestions.

Regarding execution-specs-evm: I reverted 32ac3bf(to reduce complexity) in 504b7e4, I wouldn't wait for ethereum/execution-specs#823 in order to merge this PR, I'm sure 823 (or a different fix) will get merged in due course.

@marioevz if you're happy with the changes regarding rm -rf in the t8n.sh script, see 3070d94 (and my comments on your review), I think we can merge.

execution-spec-tests/src/evm_transition_tool/transition_tool.py

Lines 330 to 334 in 504b7e4

	#!/bin/bash
	rm -rf {debug_output_path}/t8n.sh.out # hard-coded to avoid surprises
	mkdir {debug_output_path}/t8n.sh.out # unused if tracing is not enabled
	{t8n_call} < {debug_output_path}/stdin.txt
	"""

All points were addressed; re-requested review.

[marioevz]

LGTM :)

[danceratopz]

@rodiazet this is now merged into main, thanks for your contribution!