Java unable to read real-time output of youtube-dl

[dantheman213]

Please follow the guide below

• You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
• Put an x into all the boxes [ ] relevant to your issue (like this: [x])
• Use the Preview tab to see what your issue will actually look like

---

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.05.26. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

• I've verified and I assure that I'm running youtube-dl 2018.05.26

Before submitting an issue make sure you have:

• At least skimmed through the README, most notably the FAQ and BUGS sections
• Searched the bugtracker for similar issues including closed ones
• Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

• Bug report (encountered problems with youtube-dl)
• Site support request (request for adding support for a new site)
• Feature request (request for a new functionality)
• Question
• Other

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2018.05.26
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.17134
[debug] exe versions: ffmpeg N-84014-g9e6b269, ffprobe N-84014-g9e6b269
[debug] Proxy map: {}
Usage: youtube-dl.exe [OPTIONS] URL [URL...]

PROBLEM:

I'm trying to use Java to embed youtube-dl into my application. However, I cannot capture the output of the program until after the program has completely executed. The output is returned to me in one large batch. Here is my code.

               BufferedReader bufferedReaderStdOut = null;
               BufferedReader bufferedReaderStdErr = null;

               boolean isError = false;

               try {
                   System.out.println("Starting video download thread...");

                   String[] cmdArgs = {
                           App.config.settings.youtubeDlBin,
                           "-o %(title)s.%(ext)s",
                           "-f worstvideo[ext=mp4]",
                           url
                   };

                   ProcessBuilder procBuilder = new ProcessBuilder(cmdArgs);
                   procBuilder.directory(new File(App.config.settings.videoCollectionDir));
                   Process proc = procBuilder.start();

                   bufferedReaderStdOut = new BufferedReader(new InputStreamReader(proc.getInputStream()));
                   bufferedReaderStdErr = new BufferedReader(new InputStreamReader(proc.getErrorStream()));

                   System.out.println("Checking for program response...");

                   String stdOutLine = bufferedReaderStdOut.readLine();
                   String stdErrLine = bufferedReaderStdErr.readLine();
                   while(stdOutLine != null || stdErrLine != null) {
                       String logLine = "";
                       if(stdOutLine != null) {
                           System.out.println(stdOutLine);
                           logLine += stdOutLine + "\n";
                       }

                       if(stdErrLine != null) {
                           System.out.println(stdErrLine);
                           logLine += stdErrLine + "\n";
                           isError = true;
                       }

                       // On-the-fly class to pass data into runnable
                       class UpdateWindowLog implements Runnable {
                           private String line;

                           public UpdateWindowLog(String logLine) {
                               line = logLine;
                           }

                           @Override
                           public void run() {
                               textLog.setText(textLog.getText() + line);
                           }
                       }
                       Platform.runLater(new UpdateWindowLog(logLine));

                       stdOutLine = bufferedReaderStdOut.readLine();
                       stdErrLine = bufferedReaderStdErr.readLine();
                   }

                   proc.waitFor();
               } catch(Exception ex) {
                   // TBD
                   isError = true;
                   ex.printStackTrace();
               } finally {
                   System.out.println("Finished.. closing buffer.");
                   if(bufferedReaderStdOut != null) {
                       try {bufferedReaderStdOut.close();} catch(Exception ex) {}
                   }
               }

               // Create an on-the-fly custom Runnable that can receive data input into its thread
               class CloseWindowTask implements Runnable {
                   private boolean isError;

                   public CloseWindowTask(boolean error) {
                       isError = error;
                   }

                   @Override
                   public void run() {
                       Alert alert;

                       if(isError) {
                           alert = Utilities.showSimpleAlert("An error occurred!");

                       } else {
                           //System.out.println("Close dialog window...");
                           //closeWindow();

                           alert = Utilities.showSimpleAlert("Completed Successfully!");
                       }

                       alert.show();
                   }
               }
               Platform.runLater(new CloseWindowTask(isError));

               return null;
           }
       };

       // Execute the task described above
       Thread thread = new Thread(task);
       thread.setDaemon(true);
       thread.start();
   }

[dstftw]

So what? How is this even related to youtube-dl? If youtube-dl returned it as one chunk you'd get the same single chunk when running from command line but you get it gradually. So fix your code.

[dantheman213]

Whoa, why so rude? Nice to meet you too. I hope you don't treat your coworkers or other people in your life the way you do professionals on the Internet.

Please see:

MrS0m30n3/youtube-dl-gui#192

https://stackoverflow.com/questions/13530429/read-realtime-output-of-a-command

There appears to be an issue with the application treating console output and piping from another application different. I will admit I may be wrong about that but from what i've researched seems to suggest the issue is coming from the source application, e.g. youtube-dl. If the program was actually piping the data in real-time the Java code I have provided above should be able to retrieve in real-time as well.

I would ask that you look at this for more than a fraction of a second before closing out the ticket. Any assistance would be appreciated. Thanks.

[dstftw]

If something works everywhere apart from your code then it pretty much obvious whose code is broken. Yes, I've just wasted 15 minutes to check if this works properly in dummy Qt/C++ UI app running youtube-dl.exe via QProcess as well as in python script via subprocess (I don't do Java anymore, sorry). Both worked real-time just fine. So as already said fix your code.

And by the way, I either hope you don't blame and spread your incompetence to your coworkers or other people in your life the way you do it on the Internet. Thanks.