Clean up Tqdm bars when output is being piped or redirected (#4470)

* fix Tqdm formatting when output being redirected

* update changelog

CHANGELOG.md

@@ -27,9 +27,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Fixed issue #4330 by updating the `tokenizers` dependency.
 - Fixed a bug in `TextClassificationPredictor` so that it passes tokenized inputs to the `DatasetReader`
   in case it does not have a tokenizer.
-- reg_loss is only now returned for models that have some regularization penalty configured.
+- `reg_loss` is only now returned for models that have some regularization penalty configured.
 - Fixed a bug that prevented `cached_path` from downloading assets from GitHub releases.
 - Fixed a bug that erronously increased last label's false positive count in calculating fbeta metrics.
+- `Tqdm` output now looks much better when the output is being piped or redirected.
 
 ### Added
 

allennlp/common/tqdm.py

@@ -36,10 +36,12 @@ def replace_cr_with_newline(message: str) -> str:
     without adding more lines to the terminal output. Displaying those in a file won't work
     correctly, so we'll just make sure that each batch shows up on its one line.
     """
-    if "\r" in message:
-        message = message.replace("\r", "")
-        if not message or message[-1] != "\n":
-            message += "\n"
+    # In addition to carriage returns, nested progress-bars will contain extra new-line
+    # characters and this special control sequence which tells the terminal to move the
+    # cursor one line up.
+    message = message.replace("\r", "").replace("\n", "").replace("", "")
+    if message and message[-1] != "\n":
+        message += "\n"
     return message
 
 
@@ -48,7 +50,13 @@ def __init__(self):
         self.last_message_written_time = 0
 
     def write(self, message):
-        sys.stderr.write(message)
+        if sys.stderr.isatty():
+            # We're running in a real terminal.
+            sys.stderr.write(message)
+        else:
+            # The output is being piped or redirected.
+            sys.stderr.write(replace_cr_with_newline(message))
+        # Every 10 seconds we also log the message.
         now = time()
         if now - self.last_message_written_time >= 10 or "100%" in message:
             message = replace_cr_with_newline(message)