You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When deploying to lots of hosts, using lots of files, pyinfra often hits the file open limit. This is because a new file object is opened per-file, per-host (so uploading the same file in parallel work). Use Python's resource module to guestimate good values for config.PARALLEL.
During an operation, possible open files/host:
Main one for commands
Second SFTP connection
Local file for SFTP
Two per nested local Popen commands (callbacks)
Nested fact commands (callbacks) uses main SSH connection
Max files / host should be 3 - which is SSH + Popen. SFTP uses main SSH connection and file handle is closed immediately after - so will never clash with Popen/etc.
The text was updated successfully, but these errors were encountered:
When deploying to lots of hosts, using lots of files, pyinfra often hits the file open limit. This is because a new file object is opened per-file, per-host (so uploading the same file in parallel work). Use Python's
resource
module to guestimate good values forconfig.PARALLEL
.During an operation, possible open files/host:
Second SFTP connectionPopen
commands (callbacks)Nested fact commands (callbacks)uses main SSH connectionMax files / host should be 3 - which is SSH + Popen. SFTP uses main SSH connection and file handle is closed immediately after - so will never clash with Popen/etc.
The text was updated successfully, but these errors were encountered: