ERL-933: Port usage is maxing out CPU when migrating from 21.1 to 21.3

[OTP-Maintainer]

Original reporter: josevalim
Affected version: OTP-21.3
Component: erts
Migrated from: https://bugs.erlang.org/browse/ERL-933

---

This bug has been found on Windows Subsystem for Linux with Erlang installed via ESL debian repos. It is unclear if it happens on regular Linux systems. The minimum reproduction case requires inotify_wait. The bug does not happen on 21.1, but it happens on 21.3.

{code:text}
$ uname -a
Linux <snip> 4.4.0-43-Microsoft #1-Microsoft Wed Dec 31 14:42:53 PST 2014 x86_64 x86_64 x86_64 GNU/Linux
{code}

Take the following Erlang code:

{code:erlang}
-module(fsdebug).
-compile(export_all).

start_link() ->
  gen_server:start_link(?MODULE, ok, []).

init(ok) ->
  BashArgs = ["-c", os:find_executable("inotifywait") ++ " $0 $@ & PID=$!; read a; kill -KILL $PID"],
  PortArgs = ["-e", "modify", "--quiet", "-m", "-r", "/tmp"],
  Port = open_port(
    {spawn_executable, "/bin/sh"},
    [stream, exit_status, {line, 16384}, {args, BashArgs ++ PortArgs}, {cd, "/tmp"}]
  ),
  link(Port),
  process_flag(trap_exit, true),
  {ok, Port}.

handle_info({Port, Event}, Port) ->
  io:format("~p~n", [Event]),
  {noreply, Port}.
{code}

Now run it:

{code:text}
$ erl
1> c("fsdebug").
fsdebug.erl:2: Warning: export_all flag enabled - all functions will be exported
{ok,fsdebug}
2> fsdebug:start_link().
{code}

And on another terminal, write to the "/tmp" directory

{code:text}
$ for i in $(seq 1 20); do echo "koko" >> /tmp/koko; sleep 0.5; done
{code}

On 21.1, everything works as expected, but on 21.3 it stops printing the events and CPU usage grows very high. The original issue was [reported here](https://github.com/phoenixframework/phoenix/issues/3365). We are trying to see if we can reduce it even further to something that does not require `inotify_wait`.

[OTP-Maintainer]

josevalim said:

Here are some updates from the original issue.

1. The bug also happens on 21.2 (but not on 21.1)

2. The bug doesn't happen on native Linux (i.e. Ubuntu)

3. The bug can be reproduced by using `tail -f /tmp/koko` instead of inotify_wait. Here is an updated program:


{code:erlang}
-module(fsdebug).
-compile(export_all).

start_link() ->
  gen_server:start_link(?MODULE, ok, []).

init(ok) ->
  BashArgs = ["-c", os:find_executable("tail") ++ " $0 $@ & PID=$!; read a; kill -KILL $PID"],
  PortArgs = ["-f", "/tmp/koko"],
  Port = open_port(
    {spawn_executable, "/bin/sh"},
    [stream, exit_status, {line, 16384}, {args, BashArgs ++ PortArgs}, {cd, "/tmp"}]
  ),
  link(Port),
  process_flag(trap_exit, true),
  {ok, Port}.

handle_info({Port, Event}, Port) ->
  io:format("~p~n", [Event]),
  {noreply, Port}.
{code}

[OTP-Maintainer]

sverker said:

I reproduced this and it seems to be caused by the WSL emulation of epoll not behaving correctly. Events repeatedly triggers falsely causing the 100% CPU.

One workaround is to build OTP with {{--disable-kernel-poll}} configuration option.

To do some code workaround for WSL kernel poll does not seem high enough prio as WSL2 is announced to contain a real Linux kernel with a real epoll implementation.
https://devblogs.microsoft.com/commandline/announcing-wsl-2/