Plugin is spawning multiple flow server processes in different projects

[michalkvasnicak]

When I run eslint configuration with this plugin in different projects there are multiple flow processes running as a result.

Only way to get rid of these processes is to stop them manually.

This is screenshot when I run linting in 2 projects.

Is there a mechanism to stop them automatically? I know that it is hard because you start server on first run and then use flow check-contents to keep this plugin performant.

[michalkvasnicak]

I've tried to add the following line to collect.js#executeFlow

process.on('exit', () => childProcess.spawnSync(getFlowBin(), ['stop']));

This works nice with webpack in watching mode but is not good if you are using eslint in your editor because it kills flow server everytime lint process is done.

[amilajack]

What version are you using? Try both 1.2.0 and 1.3.0 separately. Do you get the same issues with both? From the code, it looks like you're using 1.2.0. 1.3.0 has huge performance improvements.

[michalkvasnicak]

I am using 1.3.0. It works good as expected. But problem with multiple servers starts if I open another project where I am using this plugin too. Then it will start another flow server for given project.

I don't see any nice solution to this because we can't detect if you've stopped your work on project. So flow server will stay running.

So how many separate projects you start, that many flow servers you will have running.

For example:

I am working on project A (/path/a), start npm run lint. This plugin starts flow server and validates source files.
Then I decide to not work on this project and move on separate project B (/path/b), start npm run lint. This plugin starts flow server for this directory too. There are 2 flow servers running as a result of this operations (each spawning multiple child processes as on screenshot from first post).

Only way to stop server from A is by running flow stop in this directory.

[amilajack]

What if the servers could have some kind of timeout option where they will automatically be killed after a certain time, say 1 hour? Also how impactful is the instantiation of new flow servers on your machine? I haven't noticed much one mine (but this is still an issue that i'm going to fix)

[michalkvasnicak]

I haven't noticed any impact on my machine. I came across it when I was debugging my app. It is just eating memory even if it is not used. Timeouts would be good but how should they work?

For example: I am using webpack in watch mode with eslint-loader which runs your plugin. So I can easily detect termination of process and kill flow server too.

Perhaps mentioning it in readme is enough so user will know that they need to stop flow server manually.

[amilajack]

I think that this module should be responsible for handling servers that it instantiates. I think the timeout method is a good one.

[amilajack]

@jdmota Would be great if you could look into this. I think it can be solved by small json object of when flow was last run on the project and killing the processes if flow hasn't been run on the project in an hour or so?

{
  '/Users/foo/project-1': '1490553978', // <- unix timestamp?
  '/Users/foo/project-2': '1490554013'
}

[jdmota]

I will take a look. I just need to understand how the flow server works. Maybe trying to start a flow server ahead of time can help... eslint-plugin-flow-check does that actually.

[jdmota]

I am working on project A (/path/a), start npm run lint. This plugin starts flow server and validates source files.
Then I decide to not work on this project and move on separate project B (/path/b), start npm run lint. This plugin starts flow server for this directory too. There are 2 flow servers running as a result of this operations (each spawning multiple child processes as on screenshot from first post).
Only way to stop server from A is by running flow stop in this directory.

Probably I'm not understanding the issue correctly... because when I run flow, there are processes that stay on the background, but they close immediately when I close the terminal. So I don't know if there is really a problem that needs to be fixed here...

[amilajack]

but they close immediately when I close the terminal

They actually stay open. This is good because flow can cache. The downside is that we have to kill them after a certain time period

[jdmota]

They actually stay open

In my case, they seem to close... Maybe because I'm on Windows?...

Anyway, how do we kill the processes after a certain time period? Can you explain exactly your ideia? I'm a little confused 😄

[amilajack]

You would probably do something like

execSync('flow stop path/to/server')

Here's the documentation

The issue gets pretty bad when you use eslint to check errors as you type.

[jdmota]

Yes. But, that json file you mentioned before; it will only help close processes from previous projects, and it would require someone to run eslint again so that we can check if flow hasn't been run in an hour, am I correct?

(Was I clear? My english is not perfect 😜 )

[amilajack]

so that we can check if flow hasn't been run in an hour, am I correct?

Yes. We may be able to use some kind of process manager like pm2. I think that will be a much more elegant solution than the json file one

[jdmota]

On the flow documentation I saw that there is a --from argument that you can give.

Do you think it can be useful to us? Maybe flow is able to close itself if it knows what started it???

[amilajack]

I haven't used it before so I'm not sure. I would recommend asking on the gitter chat or whatever chat the flow team uses now

[sarkistlt]

still have this issue, any update or fix?

[sarkistlt]

I have only one work around. In my case I'm using webstorm, so I need to turn off Flow type checking from webstorm, remove flow-bin from project and install it as global, then it works, but the are a couple problems.

[amilajack]

I believe i know the fix for this. But its will take some time, which i dont have a lot of right now. Hope someone can PR this.

[jdmota]

Maybe we could add a setting like stopOnExit, and if it is true, we do

process.on('exit', () => childProcess.spawnSync(getFlowBin(), ['stop']));

I think it would be good, at least as a work around.

Besides, although keeping the flow server alive is good for caching, I think in most cases, when people run eslint, they run it once, and expect it to stop. When using this with editors, I suspect the editor actually keeps the process alive (but I could be wrong).

Thoughts?

[amilajack]

@jdmota I think we should test this out and see how it impacts performance. How about creating a new branch. I can publish a new prerelease in v4.0.0-0 so we can test this out

[sobolevn]

This is still an issue for me. Maybe there is some workaround?

[skyrpex]

In m case, since this plugin uses flow server, other flow plugins like ide-flowtype (for Atom) or Flow Language Support (for VSCode) won't work.

Do you have similar experiences?

I reported the issue in the wrong repository... I'm ashamed, sorry! 🙏

[christophediprima]

I have the same issue? Any fix for this?

[jeffal]

I've had this issue occur when running multiple versions of flow-bin at the same time.

I recommend checking if you have a global version installed or if your repo has multiple flow-bin dependencies (monorepo), that they are all in sync across the board.

[Etheryte]

For anyone else running into this problem, in our case it was a problem with Flow running out of memory with its dependency hash table. Naturally, no error is logged about it, the server just crashes and is restarted over and over again which results in the above.

Increasing the hash table size in .flowconfig resolved the issue for us:

[options]
server.max_workers=1
sharedmemory.hash_table_pow=22