Skip to content
This repository has been archived by the owner on Mar 19, 2021. It is now read-only.

Commit

Permalink
autoxflate-sd plugin description
Browse files Browse the repository at this point in the history
  • Loading branch information
joergsteffens committed Oct 2, 2014
1 parent 68a4b86 commit d3fbe74
Show file tree
Hide file tree
Showing 6 changed files with 83 additions and 39 deletions.
2 changes: 1 addition & 1 deletion manuals/en/main/Makefile
Expand Up @@ -68,7 +68,7 @@ bareos-manual-main-reference.pdf: depend *.sty *.tex $(INDEXES) bareos-manual-ma
$(PDFLATEX) bareos-manual-main-reference.tex

check: bareos-manual-main-reference.pdf
@if ! grep "LaTeX Warning: Hyper reference .* undefined on input\|LaTeX Warning: There were undefined references." $(LOG); then true; else false; fi
@if ! grep "LaTeX Warning: Hyper reference .* undefined on input\|LaTeX Warning: There were undefined references." $(LOG); then echo "CHECK: OK"; true; else echo "CHECK: FAILED: undefined references"; false; fi

clean:
rm -f *.4ct *.4dx *.4ix *.4tc *.aux *.dvi *.htidx *.htind *.idx *.log *.out *.pdf *.toc *.idv *.lg *.tmp *.xref html/* bareos-manual-main-reference.html bareos-manual-main-reference.css
Expand Down
1 change: 1 addition & 0 deletions manuals/en/main/bareos-manual-main-reference.tex
Expand Up @@ -45,6 +45,7 @@
% but don't write them as marginal notes,
% instead only write remarks to the log file
\usepackage[norefs,nocites]{refcheck}
\usepackage{textcomp}

\usepackage{bareos}

Expand Down
21 changes: 21 additions & 0 deletions manuals/en/main/bareos.sty
Expand Up @@ -232,6 +232,27 @@
}{}


\lstnewenvironment{bmessage}[1]{%
% #1 caption
\renewcommand*{\lstlistingname}{Messages}
\footnotesize
\definecolor{colorConfig}{gray}{0.9}
\lstset{
backgroundcolor=\color{colorConfig},
frame=single,
framerule=0pt,
basicstyle=\ttfamily,
keepspaces=true,
columns=fullflexible,
%label=config:#1,
caption=#1,
captionpos=b,
moredelim=[is][\it]{<input>}{</input>},
moredelim=[is][\underline]{<strong>}{</strong>},
}
}{}



\lstnewenvironment{bconfig}[1]{%
% #1 caption
Expand Down
54 changes: 23 additions & 31 deletions manuals/en/main/plugins.tex
Expand Up @@ -97,38 +97,26 @@ \section{Storage Daemon Plugins}
\label{sdPlugins}
\subsection{autoxflate-sd}
\label{plugin-autoxflate-sd}
This plugin is part of the \package{bareos-storage} package.
The autoxflate-sd plugin can inflate(decompres) and deflate(compress)
The autoxflate-sd plugin can inflate (decompress) and deflate (compress)
the data being written to or read from a device. It can also do both.
[autoxflate-functionblocks.png]
\begin{center}
\includegraphics[width=0.8\textwidth]{\idir autoxflate-functionblocks}
\end{center}
Therefore the autoxflate plugin inserts a inflate and a deflate function block
into the stream going to the device (called OUT) and coming from the device (called IN).
Each stream passes first the inflate function block, then the deflate funcion block.
Each stream passes first the inflate function block, then the deflate function block.
The inflate blocks are controlled by the setting of the \directive{sd}{autoinflate}{yes{\textbar}no}{}{yes}{}
directive.
The inflate blocks are controlled by the setting of the \ilink{AutoInflate}{storage-device-autoinflate} directive.
The deflate blocks are controlled by the setting of the \directive{sd}{autodeflate}{yes{\textbar}no}{}{yes}{}
\directive{sd}{autodeflatealgorithm}{yes{\textbar}no}{}{yes}{} and \directive{sd}{autodeflatelevel}{yes{\textbar}no}{}{yes}{} .
\directive{sd}{autodeflate}{yes{\textbar}no}{}{yes}{} \directive{sd}{autoinflate}{yes{\textbar}no}{}{yes}{} control if the
function block is enabled at all.
Possible values are:
\begin{description}
\item [in] inflation / deflation is enabled on the incoming data path.
\item [out] inflation / deflation is enabled ont the outgoing data path.
\item [both] inflation / deflation is enabled both for incoming and outgoing data.
\end{description}
The \directive{sd}{autodeflatealgorithm}{yes{\textbar}no}{}{yes}{} selects the algorithm used for deflation,
and can be one of the supported compression algorithms: gzip / lzo / lzfast / lz4 / lz4hc.
If the compression algorithm supports the selection of compression level like gzip, this level can be configured in \directive{sd}{autodeflatelevel}{yes{\textbar}no}{}{yes}{}
The deflate blocks are controlled by the setting of the \ilink{Autodeflate}{storage-device-autodeflate},
\ilink{AutodeflateAlgorithm}{storage-device-autodeflatealgorithm} and \ilink{AutodeflateLevel}{storage-device-autodeflatelevel} directives.
The inflate blocks, if enabled, will uncompress data if it is compressed using the
algorithm that was used during compression.
Expand All @@ -138,26 +126,30 @@ \subsection{autoxflate-sd}
The series connection of the inflate and deflate function blocks makes the plugin very flexible.
Szenarios where this plugin can be used are for example:
- client computers with weak cpus can do backups without compression and
let the sd do the compression when writing to disk.
- compressed backups can be recompressed to a different compression format (e.g. gzip -> lzo) using migration jobs
- client backups can be compressed with compression algorithms that the client itself does not support.
\begin{itemize}
\item client computers with weak cpus can do backups without compression and let the sd do the compression when writing to disk
\item compressed backups can be recompressed to a different compression format (e.g. gzip \textrightarrow\ lzo) using migration jobs
\item client backups can be compressed with compression algorithms that the client itself does not support
\end{itemize}
Multi-core cpus will be utilized when using parallel jobs as the compression is done in each jobs' thread.
When the autoxflate plugin is configured, it will write some status information into the joblog.
This information shows what compression algorithm is used:
\begin{bmessage}{used compression algorithm}
autodeflation: compressor on device FileStorage is FZ4H
\end{bmessage}
How the inflation and deflation blocks are configured:
\begin{bmessage}{configured inflation and deflation blocks}
autoxflate-sd.c: FileStorage OUT:[SD->inflate=yes->deflate=yes->DEV] IN:[DEV->inflate=yes->deflate=yes->SD]
\end{bmessage}
And how the overall deflation/inflation ratio was:
\begin{bmessage}{overall deflation/inflation ratio}
autoxflate-sd.c: deflate ratio: 50.59%
\end{bmessage}
Additional \ilink{AutoXFlateOnReplication}{storage-storage-autoxflateonreplication} can be configured at the Storage resource.
\subsection{scsicrypto-sd}
Expand Down
27 changes: 21 additions & 6 deletions manuals/en/main/storedconf.tex
Expand Up @@ -223,7 +223,10 @@ \section{Storage Resource}
connections.
}

\xdirective{sd}{AutoXflateOnReplication}{yes{\textbar}no}{required}{yes}{13.4}{This directive controls the autoxflate-sd plugin when replicating data inside one or
\xdirective{sd}{AutoXflateOnReplication}{yes{\textbar}no}{required}{yes}{13.4}{%
\label{storage-storage-autoxflateonreplication}%
This directive controls the \ilink{autoxflate-sd plugin}{plugin-autoxflate-sd}
plugin when replicating data inside one or
between two storage daemons (Migration/Copy Jobs). Normally the storage daemon will
use the autoinflate/autodeflate setting of the device when reading and writing
data to it which could mean that while reading it inflates the compressed data
Expand All @@ -232,9 +235,10 @@ \section{Storage Resource}
while doing the replication of data you can set this option to no and it will
override any setting on the device for doing auto inflation/deflation when doing
data replication. This will not have any impact on any normal backup or restore jobs.}

\end{description}

The following is a typical Storage daemon Storage definition.
The following is a typical Storage daemon storage resource definition.

\begin{bconfig}{Storage daemon storage definition}
#
Expand Down Expand Up @@ -1163,7 +1167,10 @@ \section{Device Resource}
writing to a device that requires mount (USB). The default is to use the
working directory.

\xdirective{sd}{AutoDeflate}{in{\textbar}out{\textbar}both}{}{}{13.4}{This is a parameter used by the autoxflate-sd plugin which allow you to transform
\xdirective{sd}{AutoDeflate}{in{\textbar}out{\textbar}both}{}{}{13.4}{%
\label{storage-device-autodeflate}%
This is a parameter used by the \ilink{autoxflate-sd plugin}{plugin-autoxflate-sd}
which allow you to transform
a non compressed piece of data into a compressed piece of data on the storage daemon.
e.g. Storage Daemon compression. You can either enable compression on the client
and use the CPU cyclces there to compress your data with one of the supported
Expand All @@ -1190,7 +1197,9 @@ \section{Device Resource}
data you write to it but you want the data to be compressed when written.
}

\xdirective{sd}{AutoDeflateAlgorithm}{GZIP{\textbar}LZO{\textbar}LZFAST{\textbar}LZ4{\textbar}LZ4HC}{}{}{13.4}{This option specifies the compression algorithm used for the autodeflate option
\xdirective{sd}{AutoDeflateAlgorithm}{GZIP{\textbar}LZO{\textbar}LZFAST{\textbar}LZ4{\textbar}LZ4HC}{}{}{13.4}{%
\label{storage-device-autodeflatealgorithm}%
This option specifies the compression algorithm used for the autodeflate option
which is performed by the autoxflate-sd plugin. The algorithms supported are:
\begin{itemize}
\item GZIP - gzip level 1--9
Expand All @@ -1200,10 +1209,16 @@ \section{Device Resource}
\item LZ4HC
\end{itemize}}

\xdirective{sd}{AutoDeflateLevel}{0--9}{}{}{13.4}{This option specifies the level to be used when compressing when you select a
\xdirective{sd}{AutoDeflateLevel}{0--9}{}{}{13.4}{%
\label{storage-device-autodeflatelevel}%
This option specifies the level to be used when compressing when you select a
compression algorithm that has different levels.}

\xdirective{sd}{AutoInflate}{in{\textbar}out{\textbar}both}{}{}{13.4}{This is a parameter used by the autoxflate-sd plugin which allow you to transform

\xdirective{sd}{AutoInflate}{in{\textbar}out{\textbar}both}{}{}{13.4}{%
\label{storage-device-autoinflate}%
This is a parameter used by the \ilink{autoxflate-sd plugin}{plugin-autoxflate-sd}
which allow you to transform
a compressed piece of data into a non compressed piece of data on the storage daemon.
e.g. Storage Daemon decompression. You can either enable decompression on the client
and use the CPU cyclces there to decompress your data with one of the supported
Expand Down
17 changes: 16 additions & 1 deletion manuals/images/Makefile
Expand Up @@ -6,7 +6,14 @@ ORIG_EPS = $(wildcard *.eps)
CONV_EPS = $(patsubst %.eps,conv/%.eps,$(ORIG_EPS))
CONV_PNG = $(patsubst %.eps,conv/%.png,$(ORIG_EPS))

all: $(CONV_EPS) $(CONV_PNG) conv/bareos-full-logo.png conv/bareos-full-logo.eps
# conversion from fig to png does not work as expected,
# so we use the png as base.
#ORIG_FIG = $(wildcard *.fig)
#CONV_FIG_PNG = $(patsubst %.fig,conv/%.png,$(ORIG_FIG))
#CONV_FIG_EPS = $(patsubst %.fig,conv/%.eps,$(ORIG_FIG))


all: $(CONV_EPS) $(CONV_PNG) conv/bareos-full-logo.png conv/bareos-full-logo.eps conv/autoxflate-functionblocks.png conv/autoxflate-functionblocks.eps

# bareos-full-logo: png is of better quality than eps, therefore take that as source
conv/bareos-full-logo.eps: bareos-full-logo.png
Expand All @@ -17,6 +24,14 @@ conv/bareos-full-logo.png: bareos-full-logo.png
@$(MKDIRCONV)
$(COPY) bareos-full-logo.png conv/

conv/autoxflate-functionblocks.png: autoxflate-functionblocks.png
@$(MKDIRCONV)
$(COPY) autoxflate-functionblocks.png conv/

conv/autoxflate-functionblocks.eps: autoxflate-functionblocks.png
@$(MKDIRCONV)
$(CONVERT) autoxflate-functionblocks.png conv/autoxflate-functionblocks.eps

conv/%.png: %.eps
@$(MKDIRCONV)
$(CONVERT) $< conv/`basename $< .eps`.png
Expand Down

0 comments on commit d3fbe74

Please sign in to comment.