Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Single output file to stdout #128

Merged
merged 3 commits into from
May 19, 2024
Merged

Single output file to stdout #128

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
94a6fff
Open version 0.0.35-SNAPSHOT
hdsdi3g May 12, 2024
92f72ef
Single output file to stdout #112
hdsdi3g May 12, 2024
04cd8e4
Set version 0.0.35
hdsdi3g May 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .aspell-check/.aspell.en.prepl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
personal_repl-1.1 en 0
cumulate accumulate
stdout std out
uniq unique
14 changes: 7 additions & 7 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,14 +83,14 @@ Declared on OS (Windows/Linux/macOS) PATH.

Download the last application release, as a Linux RPM or DEB package, or as an executable JAR (autonomous *fat* JAR file), downloaded directly from [GitHub releases page](https://github.com/mediaexmachina/mediadeepa/releases), and build at each releases.

Install/update with
Install/update with

```bash
# DEB file on Debian/Ubuntu Linux distribs
sudo dpkg -i mediadeepa-0.0.34.deb
sudo dpkg -i mediadeepa-0.0.35.deb

# RPM file on RHEL/CentOS Linux distribs
sudo rpm -U mediadeepa-0.0.34.rpm
sudo rpm -U mediadeepa-0.0.35.rpm
```

Remove with `sudo dpkg -r mediadeepa` or `rpm -e mediadeepa`.
Expand All @@ -99,13 +99,13 @@ After, on Linux, run `mediadeepa [parameters]`, and `man mediadeepa` for the int

### Run simple JAR file

On Windows/macOS, just run `java -jar mediadeepa-0.0.34.jar [options]`.
On Windows/macOS, just run `java -jar mediadeepa-0.0.35.jar [options]`.

And simply run the application with `java -jar mediadeepa-0.0.34.jar`.
And simply run the application with `java -jar mediadeepa-0.0.35.jar`.

Mediadeepa contain embedded help, displayed with the `-h` parameter.

You can set the command line parameters with `java -jar mediadeepa-0.0.34.jar [parameters]`.
You can set the command line parameters with `java -jar mediadeepa-0.0.35.jar [parameters]`.

### Make a Java executable JAR file

Expand All @@ -119,7 +119,7 @@ cd mediadeepa
mvn install -DskipTests
```

Build jar will be founded on `target` directory as `mediadeepa-0.0.34.jar`
Build jar will be founded on `target` directory as `mediadeepa-0.0.35.jar`

<h2 id="examples">🛫 Examples</h2>

Expand Down
37 changes: 22 additions & 15 deletions docs/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<!DOCTYPE html>
<html lang="en"><head><title>Mediadeepa project documentation page</title><meta name="viewport" content="width=device-width, initial-scale=1.0"><meta data-http-equiv="Content-Type" content="text/html; charset=UTF-8"><link media="all" rel="stylesheet" href="style.css"><link rel="apple-touch-icon" type="image/png" sizes="180x180" href="https://mexm.media/a-propos/apple-touch-icon.png"><link rel="manifest" href="https://mexm.media/a-propos/site.webmanifest"><link rel="mask-icon" href="https://mexm.media/a-propos/safari-pinned-tab.svg"><meta name="msapplication-TileColor" content="#000000"><meta name="theme-color" content="#ffffff"><link rel="icon" type="image/png" sizes="16x16" href="https://mexm.media/a-propos/favicon-16x16.png"><link rel="icon" type="image/png" sizes="32x32" href="https://mexm.media/a-propos/favicon-32x32.png"></head><body><header><section id="aboutmediadeepa"><a class="anchor" href="#top"><h1>About Mediadeepa</h1></a><p>Audio/video medias and streams deep analyzer in Java with FFmpeg as back-end: extract/process technical information from audio/videos files/streams.</p>
<p><strong>This application is currently in alpha version, and should not be ready for production</strong></p>
<p><em>This document contain the full documentation for Mediadeepa version 0.0.34.</em></p>
<p><em>This document contain the full documentation for Mediadeepa version 0.0.35.</em></p>
<p><img src="https://mexm.media/project-logo.png" alt="&quot;Mediadeepa project logo&quot;" /></p>
<p>This application will run FFmpeg on a source video/audio file to apply some filters, and generate analysis raw data (mostly high verbosely text/XML streams). They are parsed and reduced/converted/drawn/summarized them to some output formats by Mediadeepa.</p>
<blockquote>
Expand Down Expand Up @@ -89,28 +89,28 @@ <h2>Current available export formats</h2>
<p>Declared on OS (Windows/Linux/macOS) PATH.</p>
<h3>Get Linux application packages</h3>
<p>Download the last application release, as a Linux RPM or DEB package, or as an executable JAR (autonomous <em>fat</em> JAR file), downloaded directly from <a href="https://github.com/mediaexmachina/mediadeepa/releases">GitHub releases page</a>, and build at each releases.</p>
<p>Install/update with</p>
<p>Install/update with</p>
<pre><code class="language-bash"># DEB file on Debian/Ubuntu Linux distribs
sudo dpkg -i mediadeepa-0.0.34.deb
sudo dpkg -i mediadeepa-0.0.35.deb

# RPM file on RHEL/CentOS Linux distribs
sudo rpm -U mediadeepa-0.0.34.rpm
sudo rpm -U mediadeepa-0.0.35.rpm
</code></pre>
<p>Remove with <code>sudo dpkg -r mediadeepa</code> or <code>rpm -e mediadeepa</code>.</p>
<p>After, on Linux, run <code>mediadeepa [parameters]</code>, and <code>man mediadeepa</code> for the internal doc man page.</p>
<h3>Run simple JAR file</h3>
<p>On Windows/macOS, just run <code>java -jar mediadeepa-0.0.34.jar [options]</code>.</p>
<p>And simply run the application with <code>java -jar mediadeepa-0.0.34.jar</code>.</p>
<p>On Windows/macOS, just run <code>java -jar mediadeepa-0.0.35.jar [options]</code>.</p>
<p>And simply run the application with <code>java -jar mediadeepa-0.0.35.jar</code>.</p>
<p>Mediadeepa contain embedded help, displayed with the <code>-h</code> parameter.</p>
<p>You can set the command line parameters with <code>java -jar mediadeepa-0.0.34.jar [parameters]</code>.</p>
<p>You can set the command line parameters with <code>java -jar mediadeepa-0.0.35.jar [parameters]</code>.</p>
<h3>Make a Java executable JAR file</h3>
<p>You can build yourself a JAR, with Git and Maven.</p>
<p>Run on Linux/WSL/macOS, after setup Git and Maven:</p>
<pre><code class="language-bash">git clone https://github.com/mediaexmachina/mediadeepa.git
cd mediadeepa
mvn install -DskipTests
</code></pre>
<p>Build jar will be founded on <code>target</code> directory as <code>mediadeepa-0.0.34.jar</code></p></section><section id="examples"><a class="anchor" href="#top"><h1>Examples</h1></a><h2>Process to export</h2>
<p>Build jar will be founded on <code>target</code> directory as <code>mediadeepa-0.0.35.jar</code></p></section><section id="examples"><a class="anchor" href="#top"><h1>Examples</h1></a><h2>Process to export</h2>
<p>Export to the current directory the analysis report for the file <code>videofile.mov</code>:</p>
<pre><code>mediadeepa -i videofile.mov -f report -e .
</code></pre>
Expand Down Expand Up @@ -231,7 +231,7 @@ <h3>Export to generated files</h3>
</ul>
<h3>Single export option</h3>
<ul>
<li><code>--single-export</code>, <strong>required</strong>, Export only this file, as: &quot;internal-file-name:outputfilename.ext&quot;</li>
<li><code>--single-export</code>, <strong>required</strong>, Export only this file, as: &quot;internal-file-name:outputfilename.ext&quot; or &quot;internal-file-name:-&quot; to stdout</li>
</ul>
<h2>Limit the scope of analysis</h2>
<p>By default, all analysis options and filters are activated in relation to your FFmpeg setup. Container analysis is still optional (via <code>-c</code>).</p>
Expand All @@ -256,17 +256,23 @@ <h2>Single export option</h2>
<p>Don't input (<code>-i</code>) more than one file.</p>
<h3>Syntax</h3>
<pre><code>--single-export &lt;internal-file-name&gt;:&lt;outputfilename.ext&gt;
--single-export &lt;internal-file-name&gt;:-
</code></pre>
<p>With <code>&lt;internal-file-name&gt;</code>, the internal app file name &quot;linked&quot; to a export format. See below the full list.</p>
<p>With <code>&lt;outputfilename.ext&gt;</code>, the full path file to produce as result.</p>
<p>With <code>&lt;outputfilename.ext&gt;</code>, the full path file to produce as result, <strong>or</strong> set <code>-</code> to send the result to <em>std out</em>.</p>
<blockquote>
<p>Use <code>;</code> instead on <code>:</code> in Windows as path separator.</p>
<p>The <code>-</code> output option will silent log messages output in std out</p>
</blockquote>
<p>Example:</p>
<pre><code>--single-export audio-loudness:/home/me/lufs.png
</code></pre>
<p>Will produce an <code>audio-loudness</code> graphic file on <code>/home/me/lufs.png</code> file.</p>
<pre><code>--single-export audio-loudness:-
</code></pre>
<p>Will produce and send the same file to <em>std out</em>, ready to be piped to another command.</p>
<blockquote>
<p>For information, <code>graphic</code> export format files has <em>no extension</em>. By default, it export <strong>PNG</strong> files. Change this with <code>--graphic-jpg</code>.</p>
</blockquote>
<p>Here the full available internal files that you can use (CSV FR format can't be selected):</p>
<h3>Values separated by tabs in text files (txt)</h3>
<ul>
Expand Down Expand Up @@ -449,6 +455,7 @@ <h3>Logging</h3>
<p>To inject a new logback configuration file, add in application command line:</p>
<pre><code>-Dlogging.config=&quot;path/to/new/logback.xml&quot;
</code></pre>
<p>For information, the use of <code>--single-export</code> option to <code>-</code> (std out) will cut all std out log messages, but you will stay able to send log messages to text file via <code>--log</code> option.</p>
<h2>Search path binaries</h2>
<p>Mediadeepa can search on several paths to found <code>ffmpeg</code>/<code>ffmpeg.exe</code> and <code>ffprobe</code>/<code>ffprobe.exe</code> (sorted by search order):</p>
<ul>
Expand Down Expand Up @@ -516,20 +523,20 @@ <h2>Road-map</h2>
<h2>Auto-generated documentation</h2>
<h3>Bash-completion</h3>
<p>The application provide a dynamic bash-completion script generated by:</p>
<pre><code class="language-bash">java -jar target/mediadeepa-0.0.34.jar --autocomplete
<pre><code class="language-bash">java -jar target/mediadeepa-0.0.35.jar --autocomplete
</code></pre>
<h3>Man page</h3>
<p>An auto-generated <strong>man</strong> page can be produced by an internal option:</p>
<pre><code class="language-bash">java -Dexportdocumentation.manpage=&quot;/full/path/to/file.man&quot; -jar target/mediadeepa-0.0.34.jar
<pre><code class="language-bash">java -Dexportdocumentation.manpage=&quot;/full/path/to/file.man&quot; -jar target/mediadeepa-0.0.35.jar
</code></pre>
<h3>README page</h3>
<p>Auto-generated by Mediadeepa, like the man page, and can be produced by the internal option:</p>
<pre><code class="language-bash">java -Dexportdocumentation.readme=&quot;README.md&quot; -jar target/mediadeepa-0.0.34.jar
<pre><code class="language-bash">java -Dexportdocumentation.readme=&quot;README.md&quot; -jar target/mediadeepa-0.0.35.jar
</code></pre>
<p>All source text for documentation is either auto-generated by the internal application options and static markdown files in the <code>src/main/resources/doc/en</code> directory.</p>
<h3>Project web page</h3>
<p>Auto-generated by Mediadeepa, like the man and README pages, and can be produced by the internal option:</p>
<pre><code class="language-bash">java -Dexportdocumentation.website=&quot;docs/index.html&quot; -jar target/mediadeepa-0.0.34.jar
<pre><code class="language-bash">java -Dexportdocumentation.website=&quot;docs/index.html&quot; -jar target/mediadeepa-0.0.35.jar
</code></pre>
<p>Same method to generate this file, like README/man files does.</p></section><section id="acknowledgments"><a class="anchor" href="#top"><h1>Acknowledgments</h1></a><p>Mediadeepa would never have been possible without the help of these magnificent and amazing OSS projects:</p>
<ul>
Expand Down
12 changes: 7 additions & 5 deletions docs/mediadeepa-1.man
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
.\" Title: mediadeepa
.\" Author: Media ex Machina / hdsdi3g
.\" Generator: mediadeepa
.\" Date: 05/12/2024
.\" Source: mediadeepa 0.0.34
.\" Date: 05/20/2024
.\" Source: mediadeepa 0.0.35
.\" Language: English
.TH "MEDIADEEPA" "1" "05/12/2024" "mediadeepa 0\&.0\&.34" ""
.TH "MEDIADEEPA" "1" "05/20/2024" "mediadeepa 0\&.0\&.35" ""
.\" -----------------------------------------------------------------
.ie \n(.g .ds Aq \(aq
.el .ds Aq '
Expand Down Expand Up @@ -531,7 +531,7 @@ Base file name for exported data file(s)
\fB\-\-single\-export\fR (required)
.PP
.RS 4
Export only this file, as: "internal\-file\-name:outputfilename\&.ext"
Export only this file, as: "internal\-file\-name:outputfilename\&.ext" or "internal\-file\-name:\-" to stdout
.PP
.RE

Expand All @@ -553,6 +553,8 @@ To inject a new logback configuration file, add in application command line:
\-Dlogging\&.config="path/to/new/logback\&.xml"
.RE
.PP
For information, the use of \fB\-\-single\-export\fR option to \fB\-\fR (std out) will cut all std out log messages, but you will stay able to send log messages to text file via \fB\-\-log\fR option\&.
.PP

Search path binaries
Mediadeepa can search on several paths to found \fBffmpeg\fR/\fBffmpeg\&.exe\fR and \fBffprobe\fR/\fBffprobe\&.exe\fR (sorted by search order):
Expand Down Expand Up @@ -617,7 +619,7 @@ Send bug reports on GitHub project page \fIhttps://github\&.com/mediaexmachina/m
If you have any questions, feel free to reach out via any contact method listed on https://mexm\&.media \fIhttps://mexm\&.media\fR\&.
.PP

Mediadeepa 0\&.0\&.34
Mediadeepa 0\&.0\&.35
.PP
Copyright (C) 2022\-2024 Media ex Machina, under the GNU General Public License\&.
.PP
2 changes: 1 addition & 1 deletion pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<groupId>media.mexm</groupId>
<artifactId>mediadeepa</artifactId>
<packaging>jar</packaging>
<version>0.0.34</version>
<version>0.0.35</version>

<name>MediaDeepA</name>
<url>https://gh.mexm.media/mediadeepa</url>
Expand Down
16 changes: 12 additions & 4 deletions src/main/java/media/mexm/mediadeepa/LoggerConfiguration.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,22 +89,22 @@ public void apply(final String[] args,
osa.setImmediateFlush(true);
osa.start();

rootLogger.addAppender(osa);
rootLogger.addAppender(osa);// NOSONAR S4792
}

if (appCommand.isVerbose()) {
verboseRootLoggersNames.stream()
.map(LoggerFactory::getLogger)
.distinct()
.map(l -> (ch.qos.logback.classic.Logger) l)
.forEach(l -> l.setLevel(ch.qos.logback.classic.Level.TRACE));
.forEach(l -> l.setLevel(ch.qos.logback.classic.Level.TRACE));// NOSONAR S4792
log.trace("Switch to verbose logger CLI mode");

} else if (appCommand.isQuiet()) {
} else if (appCommand.isQuiet() || isSingleExportToStdOut()) {
final var rootLogger = (ch.qos.logback.classic.Logger) getILoggerFactory().getLogger(ROOT_LOGGER_NAME);
rootLogger.getLoggerContext()
.getLoggerList()
.forEach(l -> l.setLevel(ch.qos.logback.classic.Level.ERROR));
.forEach(l -> l.setLevel(ch.qos.logback.classic.Level.ERROR));// NOSONAR S4792

externalCommandLine.setOut(new PrintWriter(new OutputStream() {
@Override
Expand All @@ -121,4 +121,12 @@ public void write(final int b) throws IOException {
internalCommandLine.execute(args);
}

private boolean isSingleExportToStdOut() {
try {
return appCommand.getOutputCmd().getSingleExportCmd().getSingleExport().endsWith(":-");
} catch (final NullPointerException e) {
return false;
}
}

}
2 changes: 1 addition & 1 deletion src/main/java/media/mexm/mediadeepa/cli/SingleExportCmd.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
public class SingleExportCmd {

@Option(names = { "--single-export" },
description = "Export only this file, as: \"internal-file-name:outputfilename.ext\"",
description = "Export only this file, as: \"internal-file-name:outputfilename.ext\" or \"internal-file-name:-\" to stdout",
required = true)
private String singleExport;

Expand Down
18 changes: 11 additions & 7 deletions src/main/java/media/mexm/mediadeepa/service/AppSessionServiceImpl.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@
*/
package media.mexm.mediadeepa.service;

import static java.io.File.pathSeparator;
import static java.nio.charset.Charset.defaultCharset;
import static java.nio.charset.StandardCharsets.UTF_8;
import static java.util.Collections.unmodifiableMap;
Expand Down Expand Up @@ -566,16 +565,21 @@ private void exportAnalytics(final DataResult dataResult) {
.flatMap(Optional::ofNullable)
.filter(not(String::isBlank));
if (oSingleExportParam.isPresent()) {
final var exportOnlyParams = StringUtils.split(oSingleExportParam.get(), pathSeparator, 2);
final var exportOnlyParams = StringUtils.split(oSingleExportParam.get(), ":", 2);
if (exportOnlyParams.length == 1) {
throw new ParameterException(commandLine,
"Can't manage singleExport param: missing path separator \"" + pathSeparator + "\"");
"Can't manage singleExport param: missing path separator \":\"");
}
final var internalFileName = exportOnlyParams[0];
final var outputFile = new File(exportOnlyParams[1]);

log.debug("Single export analytics {} file to {}...", internalFileName, outputFile);
mediaAnalyticsTransformerService.singleExportAnalytics(internalFileName, dataResult, outputFile);
if (exportOnlyParams[1].equals("-")) {
log.debug("Single export analytics {} file to stdout...", internalFileName);
mediaAnalyticsTransformerService.singleExportAnalyticsToOutputStream(
internalFileName, dataResult, System.out);// NOSONAR S106
} else {
final var outputFile = new File(exportOnlyParams[1]);
log.debug("Single export analytics {} file to {}...", internalFileName, outputFile);
mediaAnalyticsTransformerService.singleExportAnalytics(internalFileName, dataResult, outputFile);
}
} else {
log.debug("Export analytics");
mediaAnalyticsTransformerService.exportAnalytics(dataResult, appCommand.getOutputCmd().getExportToCmd());
Expand Down
3 changes: 3 additions & 0 deletions src/main/java/media/mexm/mediadeepa/service/MediaAnalyticsTransformerService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
package media.mexm.mediadeepa.service;

import java.io.File;
import java.io.OutputStream;

import media.mexm.mediadeepa.cli.ExportToCmd;
import media.mexm.mediadeepa.exportformat.DataResult;
Expand All @@ -30,6 +31,8 @@ void singleExportAnalytics(String internalFileName,
DataResult result,
File outputFile);

void singleExportAnalyticsToOutputStream(String internalFileName, DataResult dataResult, OutputStream out);

boolean isExportFormatExists(String name);

}
36 changes: 31 additions & 5 deletions src/main/java/media/mexm/mediadeepa/service/MediaAnalyticsTransformerServiceImpl.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,14 @@

import java.io.File;
import java.io.IOException;
import java.io.OutputStream;
import java.io.UncheckedIOException;
import java.util.List;
import java.util.Map.Entry;
import java.util.function.Consumer;

import org.apache.commons.io.FileUtils;
import org.apache.commons.io.IOUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

Expand Down Expand Up @@ -63,16 +66,23 @@ public void exportAnalytics(final DataResult result,
.forEach(s -> doExportAnalytic(result, s));
}

@Override
public void singleExportAnalytics(final String internalFileName,
final DataResult result,
final File outputFile) {
private void singleExportAnalytics(final String internalFileName,
final DataResult result,
final Consumer<? super byte[]> action) {
exportFormatList.stream()
.sorted(exportFormatComparator)
.filter(f -> f.getInternalProducedFileNames().contains(internalFileName))
.findFirst()
.flatMap(d -> d.makeSingleExport(result, internalFileName))
.ifPresent(bytes -> {
.ifPresent(action);
}

@Override
public void singleExportAnalytics(final String internalFileName,
final DataResult result,
final File outputFile) {
singleExportAnalytics(internalFileName, result,
bytes -> {
log.info("Single export result {} file produce {} bytes, save to {}",
internalFileName, bytes.length, outputFile);
try {
Expand All @@ -83,6 +93,22 @@ public void singleExportAnalytics(final String internalFileName,
});
}

@Override
public void singleExportAnalyticsToOutputStream(final String internalFileName,
final DataResult result,
final OutputStream out) {
singleExportAnalytics(internalFileName, result,
bytes -> {
try {
log.info("Single export result {} file produce {} bytes, save to stream",
internalFileName, bytes.length);
IOUtils.write(bytes, out);
} catch (final IOException e) {
throw new UncheckedIOException("Can't write to stream", e);
}
});
}

private void doExportAnalytic(final DataResult result, final ExportFormat s) {
final var now = System.currentTimeMillis();
log.info("Export with {} ({})...", s.getFormatLongName(), s.getFormatName());
Expand Down
2 changes: 1 addition & 1 deletion src/main/resources/doc/en/getting-started.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Declared on OS (Windows/Linux/macOS) PATH.

Download the last application release, as a Linux RPM or DEB package, or as an executable JAR (autonomous *fat* JAR file), downloaded directly from [GitHub releases page](https://github.com/mediaexmachina/mediadeepa/releases), and build at each releases.

Install/update with
Install/update with

```bash
# DEB file on Debian/Ubuntu Linux distribs
Expand Down
Loading
Loading