Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the performance of the pooling instantiation strategy on macOS #42

Open
koute opened this issue May 20, 2022 · 0 comments
Open
Labels
I9-optimisation An enhancement to provide better overall performance in terms of time-to-completion for a task.

Comments

@koute
Copy link
Contributor

koute commented May 20, 2022

On macOS the copy-on-write memory reuse mechanism of the pooling strategy from paritytech/substrate#11232 requires the runtime to be loaded from the disk; we could automatically do this internally inside of the executor to also speed up the instantiation on macOS which should, in theory, make it as fast as on Linux.

Investigate whether that's actually the case and measure how big of a difference there is exactly, and if there is a significant positive difference transparently precompile the runtime on macOS and load it from disk (probably from macOS's equivalent of tmpfs).

(Although we don't run any production nodes on macOS and almost no one does according to our telemetry the performance on macOS is still somewhat important since many devs use macOS when working on substrate/polkadot.)

@koute koute added the I9-optimisation An enhancement to provide better overall performance in terms of time-to-completion for a task. label May 20, 2022
@the-right-joyce the-right-joyce transferred this issue from paritytech/substrate Aug 24, 2023
lexnv added a commit that referenced this issue Apr 3, 2024
Rename `runtimeUpdates` to `withRuntime`
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 8, 2024
* patch audit findings paritytech#42

* extend msg signature for substrate relay

* signature verification test

* make proof dependet on call_dispatch crate

* silence clippy

* revert deny exception

* address code review

* since it's not really a proof, call it digest
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 8, 2024
* patch audit findings paritytech#42

* extend msg signature for substrate relay

* signature verification test

* make proof dependet on call_dispatch crate

* silence clippy

* revert deny exception

* address code review

* since it's not really a proof, call it digest
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 8, 2024
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 8, 2024
* patch audit findings paritytech#42

* extend msg signature for substrate relay

* signature verification test

* make proof dependet on call_dispatch crate

* silence clippy

* revert deny exception

* address code review

* since it's not really a proof, call it digest
github-merge-queue bot pushed a commit that referenced this issue Apr 8, 2024
… import message (#4021)

Sometimes you need to debug some issues just by the logs and reconstruct
what happened.
In these scenarios it would be nice to know if a block was imported as
best block, and what it parent was.
So here I propose to change the output of the informant to this:

```
2024-04-05 20:38:22.004  INFO ⋮substrate: [Parachain] ✨ Imported #18 (0xe7b3…4555 -> 0xbd6f…ced7)    
2024-04-05 20:38:24.005  INFO ⋮substrate: [Parachain] ✨ Imported #19 (0xbd6f…ced7 -> 0x4dd0…d81f)    
2024-04-05 20:38:24.011  INFO ⋮substrate: [jobless-children-5352] 🌟 Imported #42 (0xed2e…27fc -> 0x718f…f30e)    
2024-04-05 20:38:26.005  INFO ⋮substrate: [Parachain] ✨ Imported #20 (0x4dd0…d81f -> 0x6e85…e2b8)    
2024-04-05 20:38:28.004  INFO ⋮substrate: [Parachain] 🌟 Imported #21 (0x6e85…e2b8 -> 0xad53…2a97)    
2024-04-05 20:38:30.004  INFO ⋮substrate: [Parachain] 🌟 Imported #22 (0xad53…2a97 -> 0xa874…890f)    
```

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 8, 2024
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 8, 2024
* patch audit findings paritytech#42

* extend msg signature for substrate relay

* signature verification test

* make proof dependet on call_dispatch crate

* silence clippy

* revert deny exception

* address code review

* since it's not really a proof, call it digest
Ank4n pushed a commit that referenced this issue Apr 9, 2024
… import message (#4021)

Sometimes you need to debug some issues just by the logs and reconstruct
what happened.
In these scenarios it would be nice to know if a block was imported as
best block, and what it parent was.
So here I propose to change the output of the informant to this:

```
2024-04-05 20:38:22.004  INFO ⋮substrate: [Parachain] ✨ Imported #18 (0xe7b3…4555 -> 0xbd6f…ced7)    
2024-04-05 20:38:24.005  INFO ⋮substrate: [Parachain] ✨ Imported #19 (0xbd6f…ced7 -> 0x4dd0…d81f)    
2024-04-05 20:38:24.011  INFO ⋮substrate: [jobless-children-5352] 🌟 Imported #42 (0xed2e…27fc -> 0x718f…f30e)    
2024-04-05 20:38:26.005  INFO ⋮substrate: [Parachain] ✨ Imported #20 (0x4dd0…d81f -> 0x6e85…e2b8)    
2024-04-05 20:38:28.004  INFO ⋮substrate: [Parachain] 🌟 Imported #21 (0x6e85…e2b8 -> 0xad53…2a97)    
2024-04-05 20:38:30.004  INFO ⋮substrate: [Parachain] 🌟 Imported #22 (0xad53…2a97 -> 0xa874…890f)    
```

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
* patch audit findings paritytech#42

* extend msg signature for substrate relay

* signature verification test

* make proof dependet on call_dispatch crate

* silence clippy

* revert deny exception

* address code review

* since it's not really a proof, call it digest
dharjeezy pushed a commit to dharjeezy/polkadot-sdk that referenced this issue Apr 9, 2024
… import message (paritytech#4021)

Sometimes you need to debug some issues just by the logs and reconstruct
what happened.
In these scenarios it would be nice to know if a block was imported as
best block, and what it parent was.
So here I propose to change the output of the informant to this:

```
2024-04-05 20:38:22.004  INFO ⋮substrate: [Parachain] ✨ Imported paritytech#18 (0xe7b3…4555 -> 0xbd6f…ced7)    
2024-04-05 20:38:24.005  INFO ⋮substrate: [Parachain] ✨ Imported paritytech#19 (0xbd6f…ced7 -> 0x4dd0…d81f)    
2024-04-05 20:38:24.011  INFO ⋮substrate: [jobless-children-5352] 🌟 Imported paritytech#42 (0xed2e…27fc -> 0x718f…f30e)    
2024-04-05 20:38:26.005  INFO ⋮substrate: [Parachain] ✨ Imported paritytech#20 (0x4dd0…d81f -> 0x6e85…e2b8)    
2024-04-05 20:38:28.004  INFO ⋮substrate: [Parachain] 🌟 Imported paritytech#21 (0x6e85…e2b8 -> 0xad53…2a97)    
2024-04-05 20:38:30.004  INFO ⋮substrate: [Parachain] 🌟 Imported paritytech#22 (0xad53…2a97 -> 0xa874…890f)    
```

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
* patch audit findings paritytech#42

* extend msg signature for substrate relay

* signature verification test

* make proof dependet on call_dispatch crate

* silence clippy

* revert deny exception

* address code review

* since it's not really a proof, call it digest
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
* patch audit findings paritytech#42

* extend msg signature for substrate relay

* signature verification test

* make proof dependet on call_dispatch crate

* silence clippy

* revert deny exception

* address code review

* since it's not really a proof, call it digest
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
* patch audit findings paritytech#42

* extend msg signature for substrate relay

* signature verification test

* make proof dependet on call_dispatch crate

* silence clippy

* revert deny exception

* address code review

* since it's not really a proof, call it digest
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
* patch audit findings paritytech#42

* extend msg signature for substrate relay

* signature verification test

* make proof dependet on call_dispatch crate

* silence clippy

* revert deny exception

* address code review

* since it's not really a proof, call it digest
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
* patch audit findings paritytech#42

* extend msg signature for substrate relay

* signature verification test

* make proof dependet on call_dispatch crate

* silence clippy

* revert deny exception

* address code review

* since it's not really a proof, call it digest
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 10, 2024
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 10, 2024
* patch audit findings paritytech#42

* extend msg signature for substrate relay

* signature verification test

* make proof dependet on call_dispatch crate

* silence clippy

* revert deny exception

* address code review

* since it's not really a proof, call it digest
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 10, 2024
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 10, 2024
* patch audit findings paritytech#42

* extend msg signature for substrate relay

* signature verification test

* make proof dependet on call_dispatch crate

* silence clippy

* revert deny exception

* address code review

* since it's not really a proof, call it digest
bkchr pushed a commit that referenced this issue Apr 10, 2024
* patch audit findings #42

* extend msg signature for substrate relay

* signature verification test

* make proof dependet on call_dispatch crate

* silence clippy

* revert deny exception

* address code review

* since it's not really a proof, call it digest
jonathanudd pushed a commit to jonathanudd/polkadot-sdk that referenced this issue Apr 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
I9-optimisation An enhancement to provide better overall performance in terms of time-to-completion for a task.
Projects
Status: backlog
Development

No branches or pull requests

1 participant