warnings: log_vc_info - unclosed file

[oliver-sanders]

Address a Python warning:

cylc-flow/cylc/flow/install_plugins/log_vc_info.py:185: ResourceWarning: unclosed file <_io.TextIOWrapper name=5 encoding='UTF-8'>
  continue
ResourceWarning: Enable tracemalloc to get the object allocation traceback
cylc-flow/cylc/flow/install_plugins/log_vc_info.py:185: ResourceWarning: unclosed file <_io.TextIOWrapper name=7 encoding='UTF-8'>
  continue
ResourceWarning: Enable tracemalloc to get the object allocation traceback
cylc-flow/cylc/flow/install_plugins/log_vc_info.py:185: ResourceWarning: unclosed file <_io.TextIOWrapper name=5 encoding='UTF-8'>
  continue
ResourceWarning: Enable tracemalloc to get the object allocation traceback
cylc-flow/cylc/flow/install_plugins/log_vc_info.py:185: ResourceWarning: unclosed file <_io.TextIOWrapper name=7 encoding='UTF-8'>
  continue

To see this warning, run the tests or try adding:

import warnings
warnings.filterwarnings('once')

[MetRonnie]

I've narrowed this down to pipe_poller (by commenting out the first branch here and seeing the warnings disappear):

cylc-flow/cylc/flow/install_plugins/log_vc_info.py

Lines 257 to 260 in 8b2feae

	if stdout == PIPE:
	out, err = pipe_poller(proc, proc.stdout, proc.stderr)
	else:
	out, err = proc.communicate()

cylc-flow/cylc/flow/pipe_poller.py

Lines 29 to 30 in 8b2feae

	def pipe_poller(proc, *files, chunk_size=4096):
	"""Read from a process without hitting buffer issues.

[oliver-sanders]

Good spot, this may have the same underlying cause as #5990

[oliver-sanders]

Doesn't make sense to me, the pipe_poller does not open any files, it just reads from already open files. It wouldn't make sense for the pipe_poller to close the files, it might not even be provided with all of them.

Is proc.communicate closing proc.out and proc.err? If so, then we just need to add that into the pipe_poller branch. Or possible proc.wait if that also closes the files?

[MetRonnie]

Is proc.communicate closing proc.out and proc.err?

Yes looks like it. When it reaches the end of the file it closes it - https://github.com/python/cpython/blob/ecd9946c9d1df7cd1aa15d08d72bbcc0899d272d/Lib/subprocess.py#L1750-L1754

[oliver-sanders]

Ok, then we can bung a proc.communicate after or possibly even inside the proc_poller bit to resolve this.