Output is appended to the report file when using --output-file

[ghost]

We are using flake 4.0.1 in a clean virtual environment together with tox 3.42.2.

$ mkvirtualenv test
$ pip install flake8==4.0.1
$ pip install tox==3.42.2

We need to use the --output-file option to store the flake8 results to a file because tox does not support stdout redirection to a file. This report will then be used to publish the findings on our build server.
Also developers will use the same flow to check their code before committing it to the repository. During these checks we found out that flake8 does append new findings to an existing output file instead of replacing it with a new version. This makes it unnecessarily hard to check if reported findings have been fixed or new ones have been added to the report.

Appending to a log/report file without explicitly telling the tool to do so is not intuitive. In fact flake8 seems to be the only tool I know that does not overwrite output files by default. Also deleting the file manually before running flake8 again is error prone and has to be communicated to the developers.

Unfortunately there is no useful information in the issue #801 that is associated with the commit that changed the behavior from opening the file in write mode to opening it in append mode.

I propose to change the file mode back to 'w' to give flake8 the expected behavior.

[asottile]

you could probably use redirection in tox fairly easily:

allow_externals = bash
commands =
    bash -exc 'flake8 whatever > output.txt'

[asottile]

#535 is the rationale for #810 by the way

[ghost]

We are running our checks on Windows and Linux and unfortunately there is no bash available on our Windows machines. Therefore we would need to introduce OS-dependent solutions, which are also just workarounds for the real problem.

[asottile]

how are you getting the code onto the windows machines? if you're using git then you have bash

[ghost]

Sorry, but we are using SVN as version control for some Python packages and we are having some restrictions on what is allowed on out build nodes. So no chance for getting bash there.

[asottile]

surely you can use python then:

commands =
    python -c 'import subprocess; raise SystemExit(subprocess.call(("flake8",), stdout=open("logfile", "w")))'

[ghost]

After taking some time to look at the code it looks like the problem is that the logger and the formatter both open the same file and that the formatter does not know if the file has already been filled with log output.
Would it not be possible to just flush the file if it exists, so that both tools start with an empty file? This could be done before initializing the logger.

[sigmavirus24]

@smartSBA why can't you ensure the file is truncated before running flake8? This is a requirement you have

[ghost]

I just think that appending stuff to a report file by default is somehow unusual. For log files it would be OK. But in this case where the program is run once it does not make sense for me. At least it could be mentioned somewhere in the help output.

From my point of view changing this behavior would be an improvement. But it is your decision to do so or leave it as it is.