Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Factor substrate node runner into separate crate #913

Merged
merged 9 commits into from Apr 17, 2023
Merged

Conversation

jsdw
Copy link
Collaborator

@jsdw jsdw commented Apr 14, 2023

So that we can re-use the logic for starting a test substrate node up in both the test-runtime crate (to hopefully avoid the retry errors we've been seeing a bit lately) and the integration-tests crate which also spawns a node.

I added support for @niklasad1's upcoming PR in terms of capturing stdout but not in terms of handling the invalid flag; perhaps I should! (but until then, this should still be a bit of an improvement I hope)

I also removed the Subxt dep on the build.rs of test-runtime in the hopes that it would mean it is triggered much less often (ie not every time Subxt is changed), but let's see aye!

Closes #909

@jsdw jsdw requested a review from a team as a code owner April 14, 2023 12:38
@jsdw jsdw marked this pull request as draft April 14, 2023 12:47
@jsdw jsdw marked this pull request as ready for review April 14, 2023 13:34
@jsdw jsdw marked this pull request as draft April 14, 2023 13:34
@jsdw jsdw marked this pull request as ready for review April 14, 2023 16:26
@jsdw
Copy link
Collaborator Author

jsdw commented Apr 14, 2023

When #907 merges I'll remove the parity-scale-codec dependency from test-runtime too

.arg("--dev")
.stdout(process::Stdio::piped())
.stderr(process::Stdio::piped())
.arg("--port=0")
Copy link
Member

@niklasad1 niklasad1 Apr 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we care about the libp2p port? :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the reasoning is to prevent any overlap with the port used for other substrate instances we spawn; I guess Substrate will pick a random port for us already if the one we give isn't any good, but probably just getting the kernel to pick an OK port straight off is a touch better anyway?

.stdout(process::Stdio::piped())
.stderr(process::Stdio::piped())
.arg("--port=0")
.arg("--rpc-port=0")
Copy link
Member

@niklasad1 niklasad1 Apr 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could omit the port for both --rpc--port, --ws-port here because substrate would fallback to 0 anyway, this way it would be backward-compatible :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm but ah true; backward compat is a good reason not to have them; I'll remove :)

// Consume a stderr reader from a spawned substrate command and
// locate the port number that is logged out to it.
fn try_find_substrate_port_from_output(r: impl Read + Send + 'static) -> Option<u16> {
BufReader::new(r).lines().take(50).find_map(|line| {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe increase the number of a lines a bit to future proof?

Suggested change
BufReader::new(r).lines().take(50).find_map(|line| {
BufReader::new(r).lines().take(1024).find_map(|line| {

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I wanted it to fail reasonably quickly if for some reason it wasn't picking up the port; I started at 100 but I think I counted right now that it's something like 25 lines to get the port, so 50 was my "safe" number; maybe 100 to compromise a bit? It should never take that long for the port to be logged I'd hope :)

jsdw and others added 4 commits April 17, 2023 11:51
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
@jsdw jsdw merged commit 2f1b67b into master Apr 17, 2023
7 checks passed
@jsdw jsdw deleted the jsdw-substrate-runner branch April 17, 2023 10:42
@jsdw jsdw mentioned this pull request Jun 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Test-runtime: Run Substrate node on port 0 for more reliable execution
4 participants