sys.stdout not flushed on exit

[arxanas]

🌍 Environment

• Your operating system and version: MacOS 10.15.7
• Your python version: Python 3.9.1
• How did you install python (e.g. apt or pyenv)? Did you use a virtualenv?: Homebrew; python3 -m virtualenv .venv
• Your Rust version (rustc --version): rustc 1.50.0 (cb75ad5db 2021-02-10)
• Your PyO3 version: 0.13.2
• Have you tried using latest PyO3 master (replace version = "0.x.y" with git = "https://github.com/PyO3/pyo3")?: Yes, reproduces after fixing compilation errors.

💥 Reproducing

Clone https://github.com/arxanas/pyo3-bug and run PYTHONPATH=. cargo run in the root directory.

main.rs:

use pyo3::prelude::*;

fn main() -> PyResult<()> {
    Python::with_gil(|py| {
        let testing = PyModule::import(py, "testing")?;
        testing.call0("foo")?;
        Ok(())
    })
}

testing.py:

import sys

def foo():
    sys.stdout.write("Hello, world!")
    # Uncomment to fix issue:
    # sys.stdout.flush()

Note that the issue doesn't reproduce if the output ends with a newline character. Presumably that automatically flushes stdout.

This is probably due to #1355. I understand that running finalizers might break some C code, but perhaps a special case can be made for flushing stdout/stderr? It's an idempotent operation, and it might take a long time for someone to debug the issue otherwise.

If you don't want to change this behavior, then a note in the documentation somewhere would be helpful.

[davidhewitt]

Hi @arxanas we added the finalizers to flush stdout and then removed them in #1355 as you note because of the C code and also async issues. In the end we decided there's too many issues with finalizers to run them ourselves. So I think this is a documentation issue.

You may want to consider running with PYTHONUNBUFFERED=1 if you've got stdout being printed from both Rust and Python, so that ordering remains consistent. This will also avoid the need to call flush manually at program exit 😃

Alternatively you could add unsafe { pyo3::ffi::Py_Finalize() } at the end of your main function if it doesn't cause you problems.