Meltano EDK utilizing code that is not implemented on Windows

[joshua-janicas]

I work with Meltano/DBT in a Windows environment, and containerize the solution into Debian. With the v0.2.0 release I started getting the issue which can be seen by the callstack below.

elt-pipeline: [main ≡ +1 ~1 -0 !]> meltano invoke dbt-snowflake:compile
2024-05-23T11:16:33.760512Z [info     ] Environment 'dev' is active
Extension executing `dbt clean`...
Traceback (most recent call last):
  File "C:\Git\elt-pipeline\.meltano\utilities\dbt-snowflake\venv\Lib\site-packages\meltano\edk\extension.py", line 126, in pass_through_invoker
    self.pre_invoke(None, *command_args)
  File "C:\Git\elt-pipeline\.meltano\utilities\dbt-snowflake\venv\Lib\site-packages\dbt_ext\extension.py", line 72, in pre_invoke
    self.dbt_invoker.run_and_log("clean")
  File "C:\Git\elt-pipeline\.meltano\utilities\dbt-snowflake\venv\Lib\site-packages\meltano\edk\process.py", line 179, in run_and_log
    result = asyncio.run(self._exec(sub_command, *args))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\asyncio\runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\asyncio\base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "C:\Git\elt-pipeline\.meltano\utilities\dbt-snowflake\venv\Lib\site-packages\meltano\edk\process.py", line 137, in _exec
    loop.add_signal_handler(
  File "C:\Program Files\Python311\Lib\asyncio\events.py", line 578, in add_signal_handler
    raise NotImplementedError
NotImplementedError
pre_invoke failed with uncaught exception, please report to maintainer
Traceback (most recent call last):
  File "C:\Git\elt-pipeline\.meltano\utilities\dbt-snowflake\venv\Lib\site-packages\meltano\edk\extension.py", line 126, in pass_through_invoker
    self.pre_invoke(None, *command_args)
  File "C:\Git\elt-pipeline\.meltano\utilities\dbt-snowflake\venv\Lib\site-packages\dbt_ext\extension.py", line 72, in pre_invoke
    self.dbt_invoker.run_and_log("clean")
  File "C:\Git\elt-pipeline\.meltano\utilities\dbt-snowflake\venv\Lib\site-packages\meltano\edk\process.py", line 179, in run_and_log
    result = asyncio.run(self._exec(sub_command, *args))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\asyncio\runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\asyncio\base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "C:\Git\elt-pipeline\.meltano\utilities\dbt-snowflake\venv\Lib\site-packages\meltano\edk\process.py", line 137, in _exec
    loop.add_signal_handler(
  File "C:\Program Files\Python311\Lib\asyncio\events.py", line 578, in add_signal_handler
    raise NotImplementedError
NotImplementedError
Exception ignored in: <function BaseSubprocessTransport.__del__ at 0x00000250E1B10540>
Traceback (most recent call last):
  File "C:\Program Files\Python311\Lib\asyncio\base_subprocess.py", line 126, in __del__
    self.close()
  File "C:\Program Files\Python311\Lib\asyncio\base_subprocess.py", line 104, in close
    proto.pipe.close()
  File "C:\Program Files\Python311\Lib\asyncio\proactor_events.py", line 109, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "C:\Program Files\Python311\Lib\asyncio\base_events.py", line 762, in call_soon
    self._check_closed()
  File "C:\Program Files\Python311\Lib\asyncio\base_events.py", line 520, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x00000250E1B11C60>
Traceback (most recent call last):
  File "C:\Program Files\Python311\Lib\asyncio\proactor_events.py", line 116, in __del__
    _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\asyncio\proactor_events.py", line 80, in __repr__
    info.append(f'fd={self._sock.fileno()}')
                      ^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\asyncio\windows_utils.py", line 102, in fileno
    raise ValueError("I/O operation on closed pipe")
ValueError: I/O operation on closed pipe
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x00000250E1B11C60>
Traceback (most recent call last):
  File "C:\Program Files\Python311\Lib\asyncio\proactor_events.py", line 116, in __del__
    _warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\asyncio\proactor_events.py", line 80, in __repr__
    info.append(f'fd={self._sock.fileno()}')
                      ^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\asyncio\windows_utils.py", line 102, in fileno
    raise ValueError("I/O operation on closed pipe")
ValueError: I/O operation on closed pipe

Thanks to Derek Visch's insight, I had to pin this EDK back to v0.1.0. It seems that the EDK is using a code here https://github.com/meltano/edk/blob/v0.4.1/meltano/edk/process.py#L139C39-L139C50 that breaks in Windows, but not in Linux environments. Supposedly the fix to this is found here https://stackoverflow.com/questions/35772001/how-to-handle-a-signal-sigint-on-a-windows-os-machine/72637975#72637975.

https://meltano.slack.com/archives/C069CQNHDNF/p1716463234362109

[edgarrmondragon]

Thanks for logging @joshua-janicas!

The change in question was introduced in 191b227.

PRs welcome!