Prevent creation of unlimited number of compile workers

[0mkara]

Currently when solidity files are saved or changed frequently, it results creation of unlimited number of compile workers. This crashes the computer.

[0mkara]

• There should be always a finite number of node child process that can be initiated from Etheratom to run specific tasks.
• User should be able to cancel previous running task if any and start new task.
• A task runner indicating running tasks & able to cancel current running process should be implemented as well.

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

This issue now has a funding of 150.0 DAI (150.0 USD @ $1.0/DAI) attached to it as part of the ECF fund.

• If you would like to work on this issue you can 'start work' on the Gitcoin Issue Details page.
• Want to chip in? Add your own contribution here.
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $44,220.75 more funded OSS Work available on the Gitcoin Issue Explorer

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

Work has been started.

These users each claimed they can complete the work by 2 months, 3 weeks ago.
Please review their action plans below:

1) deleafgreen has applied to start work (Funders only: approve worker | reject worker).

Get access to project and reduce number oft workers

Learn more on the Gitcoin Issue Details page.

2) jeremigendron has been approved to start work.

I've submitted a basic PR outlining what I think is an OK approach to fixing the problem. It is by no means a complete solution but might offer grounds for another developer to make some progress on the issue. It is untested. #217.

Learn more on the Gitcoin Issue Details page.

[tcrowe]

Hi @JeremiGendron. I looked at your PR and it could work. I wonder if it would be better to find out what is causing the fork before even starting it. For example adding debugger; into compileWeb3 to see the stack trace. Using a debounce might help.

Some places to look:

etheratom/lib/web3/methods.js

Line 39 in c2523b2

async compileWeb3(sources) {

etheratom/lib/web3/web3.js

Line 268 in c2523b2

this.helpers.compileWeb3(sources);

etheratom/lib/web3/web3.js

Line 217 in c2523b2

this.compileSubscriptions.add(atom.workspace.observeTextEditors((editor) => {

---

I'm just a guy looking around at issues. You don't have to do anything based on what I say.

[tpscrpt]

I'll give it some thought, thanks for the indications!

…

On Fri, Nov 16, 2018, 4:22 PM Tony Crowe ***@***.*** wrote: Hi @JeremiGendron <https://github.com/JeremiGendron>. I looked at your PR and it could work. I wonder if it would be better to find out what is causing the fork before even starting it. For example adding debugger; into compileWeb3 to see the stack trace. Using a debounce might help. Some places to look: https://github.com/0mkara/etheratom/blob/c2523b28d2587da9f9f7ef02dca98e383a00f6e0/lib/web3/methods.js#L39 https://github.com/0mkara/etheratom/blob/c2523b28d2587da9f9f7ef02dca98e383a00f6e0/lib/web3/web3.js#L268 https://github.com/0mkara/etheratom/blob/c2523b28d2587da9f9f7ef02dca98e383a00f6e0/lib/web3/web3.js#L217 ------------------------------ *I'm just a guy looking around at issues. You don't have to do anything based on what I say.* — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#198 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AWfqZSdCIRtZg4f-dIDvPlft3StBmRjAks5uvyyEgaJpZM4XL9Rz> .

[0mkara]

what is causing the fork before even starting it

@tcrowe can you explain what do you mean ?

@JeremiGendron your approach of limiting total number of workers to 10 is quite simple and good. I also tried similar issue and it kinda does work. However there are few other issues I saw when I tried this solution earlier.

• Imagine you have submitted a.sol for compilation [i.e hit save and etheratom has started to compile it]. Now while rapidly developing you move to file b.sol and hit save 2 times. Now 2 more jobs has started. Then the first job finishes and the result you see, is compilation result for a.sol. This is quite confusing for users. Users would be expecting results for b.sol.
• When a compilation is going on there is no interactive way to stop that job. This is a required feature.
• There is a problem with the save event of atom. Even if I hit save shortcut button ctrl+s once I could see multiple event being triggered and that caused creation of multiple compiler worker when we should expect only one worker.

I think solution for this issue would be:

• find out the root cause of firing multiple events from save event in the editor and prevent creation of multiple workers
• limit maximum numbers of workers
• provide user interface to kill or cancel jobs

@JeremiGendron @tcrowe thank you both for looking into Etheratom IDE issues and contribute to them.

[tpscrpt]

- Save sha1 or sha256 of bytecode and create an object with all active hashes to compare before firing a job (avoids duplicate jobs) - Provide a UI shortcut to kill running solcWorkers - Change the compilation display priority mechanism

…

On Sat, Nov 17, 2018, 7:50 AM Omkara ***@***.*** wrote: what is causing the fork before even starting it @tcrowe <https://github.com/tcrowe> can you explain what do you mean ? @JeremiGendron <https://github.com/JeremiGendron> your approach of limiting total number of workers to 10 is quite simple and good. I also tried similar issue and it kinda does work. However there are few other issues I saw when I tried this solution earlier. - Imagine you have submitted a.sol for compilation [i.e hit save and etheratom has started to compile it]. Now while rapidly developing you move to file b.sol and hit save 2 times. Now 2 more jobs has started. Then the first job finishes and the result you see, is compilation result for a.sol. This is quite confusing for users. Users would be expecting results for b.sol. - When a compilation is going on there is no interactive way to stop that job. This is a *required* feature. - There is a problem with the save event of atom. Even if I hit save shortcut button ctrl+s once I could see multiple event being triggered and that caused creation of multiple compiler worker when we should expect only one worker. I think solution for this issue would be: - find out the root cause of firing multiple events from save event in the editor and prevent creation of multiple workers - limit maximum numbers of workers - provide user interface to kill or cancel jobs @JeremiGendron <https://github.com/JeremiGendron> @tcrowe <https://github.com/tcrowe> thank you both for looking into Etheratom IDE issues and contribute to them. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#198 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AWfqZdiHE1fvUUEui_WUYQ7lTt9HWIBuks5uwAYQgaJpZM4XL9Rz> .

[tpscrpt]

#217 has been update with a few changes, will now devote some time to setting up the environment and tuning it.

[tcrowe]

@0mkara I think you guys understood well. This problem comes up ALL the time when forking.

As an Atom user I am looking forward to trying this extension! Thanks guys :D

[tpscrpt]

@0mkara I'll be finishing up the theory and PoC tonight

[spm32]

Great work so far @JeremiGendron, just approved you formally to continue for the bounty!

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

Work for 150.0 DAI (150.0 USD @ $1.0/DAI) has been submitted by:

1. @jeremigendron

@ceresstation please take a look at the submitted work:

• PR by @JeremiGendron

---

• Learn more on the Gitcoin Issue Details page
• Want to chip in? Add your own contribution here.
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $37,902.91 more funded OSS Work available on the Gitcoin Issue Explorer

[0mkara]

PR #217 has been merged and published at a6a8225

[tpscrpt]

@ceresstation Have you reviewed the submitted work on gitcoin?

[0mkara]

No. I don't have any mod permission on gitcoin. I will contact @ceresstation.

[spm32]

Sorry for the delay @JeremiGendron, awesome job, paying you out now!

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

The funding of 150.0 DAI (150.0 USD @ $1.0/DAI) attached to this issue has been approved & issued to @JeremiGendron.

• Learn more on the Gitcoin Issue Details page
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $35,419.93 more funded OSS Work available on the Gitcoin Issue Explorer

[tpscrpt]

@ceresstation much love, thanks!