Make the number of parallel jobs configurable #362
Assignees
Labels
Comments
|
Using half the cores on e.g. a dual core machine (such as in a laptop) might mean using fewer cores than expected. It will also burn more energy as it will take longer before the processor can go to sleep again. Why not use all cores by default but allow to scale down? |
|
The following scenario always makes my desktop machine at home (AMD 6 Cores) freeze completely:
|
fsundermeyer
added a commit
that referenced
this issue
Jun 9, 2016
The new global switch --jobs lets you specify how mny parallel jobs to use when processing XML and image with DAPS. If not specified, the number is set to the number of CPU cores (as was done in previous versions).
|
Fixed with commit 53270b6, handing over to documentation. The new global switch --jobs lets you specify how mny parallel jobs to |
fsundermeyer
changed the title
fsundermeyer
added a commit
that referenced
this issue
Jun 13, 2016
Stable release DAPS 2.1.6:
* New Features:
- new subcommand "xmlformat" prettyfies the XML sources of a
given DC-file or ROOTID using daps-xmlformat with
/etc/daps/docbook-xmlformat.conf.
(Note: this command will modify the original XML sources!)
- SVG image support for HTML builds. To use it, make sure to
provide the images in SVG format ad reference to them as follows:
<imageobject role="html">
<imagedata fileref="<SVG-FILE>" format="SVG"/>
</imageobject>
- the new global switch --jobs lets you specify how many parallel
jobs to use. The default is set to the number of CPU cores (as was
done in previous versions).
- all commands generating file lists (e.g. "list-srcfiles") now
generate human readable output on STDOUT. When the output goes
to a pipe or subshell, it will be generated as a a one-liner.
- Debugging of XML errors has been made easier. Sometimes the error
message of the validator only points to an xi:include line, making
it impossible to find the real location of the error. The option
--novalid for the target "bigfile" will create the bigfile
without performing a validation check on the sources. A single XML
will be built and validated. Now the error message points to the
exact location of the error in the bigfile. From there it should be
easy to find the error in the original sources.
- Verbose and debugging output now also show all profiling
attributes set in the DC-file, plus version information for
the sytlesheets (if available)
- added the option --norefchecks to all output-generating targets
("html", "pdf", ...), "validate", "bigfile" and others. Allows
to build/validate documents without checking internal links
(<... linkend="<ID>">).
- subcommand "online-docs" now always converts the bigfile that is
created to NovDoc. To keep the same format as the original
sources, specify --dbnoconv. To create a DocBook4 bigfile from
DocBook5 sources, specify --db5todb4.
* Bugfixes:
- #367: daps does not validate fop output
- #366: Spellcheck trying to check the files "Cannot", "stat:", "No",
"such", "file" ...
- #364: DocBook5: Allow validating without checking xrefs
- #363: Allow building a bigfile from invalid sources
- #362: Make the number of parallel jobs configurable
- #361: Externel xrefs are resolved to "????"
- #360: Check Rootid fails on Debian 8.4.0
- #356: file list output should default to pretty format
- #354: Validate the result of online docs when it has been
converted from DB5 to DB4 or novdoc
- #353: Make db5 to db4 conversion default for online-docs
- #351, #348, #342, #282: Fixes for the daps spec-file
- #346: daps-xmlformat writes its messages into the output
- #343: daps-init warning messages
- #340: ePUB builds always show remarks
- #335: on Debian, make install creates incorrect group in
/etc/xml/config
- #330: Enable SVG2Grayscale stylesheet to handle new color
names
- #305: locdrop produces no "non-trans" image tarball for a
complete set
- #301: SVG to PNG conversion creates very large PNGs
- #289: Verbose / Debugging Output Should Show Profiling
Attributes
- #251: Write XSLT to Return xml-model PI
- #226: Let stylesheets know which version of DAPS calls them
- subcommand "clean-all" fails if run from within a directory
that will be removed with that command
- daps fails when called from a directory that no longer exists
* Cross-Distribution Support
- first release with full Debian/Ubuntu support--manual
adjustments after the installation are no longer needed
(was required in previous releases for DocBook5 support).
Many thanks to Tomáš Bažant and Christoph Berg for their help!
- GitHub checkouts now also work for Fedora/RedHat and
Debian/Ubuntu--refer to
https://github.com/openSUSE/daps/blob/develop/INSTALL.adoc
for details
- tested on Debian 8.5.0 / Fedora 23 / openSUSE 13.2 and
Leap 42.1 / SUSE Linux Enterprise 12 / Ubuntu 16.0
* Misc:
- removed "daps-envconvert", a script for converting susedoc
ENV-files to DAPS DC-files. susedoc was the predecessor of
DAPS.
fsundermeyer
added a commit
that referenced
this issue
Jun 13, 2016
Stable release DAPS 2.1.6:
* New Features:
- new subcommand "xmlformat" prettyfies the XML sources of a
given DC-file or ROOTID using daps-xmlformat with
/etc/daps/docbook-xmlformat.conf.
(Note: this command will modify the original XML sources!)
- SVG image support for HTML builds. To use it, make sure to
provide the images in SVG format ad reference to them as follows:
<imageobject role="html">
<imagedata fileref="<SVG-FILE>" format="SVG"/>
</imageobject>
- the new global switch --jobs lets you specify how many parallel
jobs to use. The default is set to the number of CPU cores (as was
done in previous versions).
- all commands generating file lists (e.g. "list-srcfiles") now
generate human readable output on STDOUT. When the output goes
to a pipe or subshell, it will be generated as a a one-liner.
- Debugging of XML errors has been made easier. Sometimes the error
message of the validator only points to an xi:include line, making
it impossible to find the real location of the error. The option
--novalid for the target "bigfile" will create the bigfile
without performing a validation check on the sources. A single XML
will be built and validated. Now the error message points to the
exact location of the error in the bigfile. From there it should be
easy to find the error in the original sources.
- Verbose and debugging output now also show all profiling
attributes set in the DC-file, plus version information for
the sytlesheets (if available)
- added the option --norefchecks to all output-generating targets
("html", "pdf", ...), "validate", "bigfile" and others. Allows
to build/validate documents without checking internal links
(<... linkend="<ID>">).
- subcommand "online-docs" now always converts the bigfile that is
created to NovDoc. To keep the same format as the original
sources, specify --dbnoconv. To create a DocBook4 bigfile from
DocBook5 sources, specify --db5todb4.
* Bugfixes:
- #367: daps does not validate fop output
- #366: Spellcheck trying to check the files "Cannot", "stat:", "No",
"such", "file" ...
- #364: DocBook5: Allow validating without checking xrefs
- #363: Allow building a bigfile from invalid sources
- #362: Make the number of parallel jobs configurable
- #361: Externel xrefs are resolved to "????"
- #360: Check Rootid fails on Debian 8.4.0
- #356: file list output should default to pretty format
- #354: Validate the result of online docs when it has been
converted from DB5 to DB4 or novdoc
- #353: Make db5 to db4 conversion default for online-docs
- #351, #348, #342, #282: Fixes for the daps spec-file
- #346: daps-xmlformat writes its messages into the output
- #343: daps-init warning messages
- #340: ePUB builds always show remarks
- #335: on Debian, make install creates incorrect group in
/etc/xml/config
- #330: Enable SVG2Grayscale stylesheet to handle new color
names
- #305: locdrop produces no "non-trans" image tarball for a
complete set
- #301: SVG to PNG conversion creates very large PNGs
- #289: Verbose / Debugging Output Should Show Profiling
Attributes
- #251: Write XSLT to Return xml-model PI
- #226: Let stylesheets know which version of DAPS calls them
- subcommand "clean-all" fails if run from within a directory
that will be removed with that command
- daps fails when called from a directory that no longer exists
* Cross-Distribution Support
- first release with full Debian/Ubuntu support--manual
adjustments after the installation are no longer needed
(was required in previous releases for DocBook5 support).
Many thanks to Tomáš Bažant and Christoph Berg for their help!
- GitHub checkouts now also work for Fedora/RedHat and
Debian/Ubuntu--refer to
https://github.com/openSUSE/daps/blob/develop/INSTALL.adoc
for details
- tested on Debian 8.5.0 / Fedora 23 / openSUSE 13.2 and
Leap 42.1 / SUSE Linux Enterprise 12 / Ubuntu 16.0
* Misc:
- removed "daps-envconvert", a script for converting susedoc
ENV-files to DAPS DC-files. susedoc was the predecessor of
DAPS.
* Documentation:
- the documentation has _not_ been updated for this release, so
new features mentioned above are not yet covered
fsundermeyer
added a commit
that referenced
this issue
Jun 15, 2016
New Features:
- new subcommand `xmlformat' prettyfies the XML sources of a given
DC-file or ROOTID using daps-xmlformat with
/etc/daps/docbook-xmlformat.conf. (Note: this command will modify the
original XML sources!)
- SVG image support for HTML builds. To use it, make sure to
provide the images in SVG format and reference to them as follows:
<imageobject role="html">
<imagedata fileref="SVG-FILE" format="SVG"/>
</imageobject>
- the new global switch `--jobs` lets you specify how many parallel
jobs to use. The default is set to the number of CPU cores (as was
done in previous versions).
- all commands generating file lists (e.g. `list-srcfiles`) now
generate human readable output on STDOUT. When the output goes
to a pipe or subshell, it will be generated as a a one-liner.
- Debugging of XML errors has been made easier. Sometimes the error
message of the validator only points to an xi:include line, making
it impossible to find the real location of the error. The option
`--novalid` for the target `bigfile` will create the bigfile
without performing a validation check on the sources. A single XML
will be built and validated. Now the error message points to the
exact location of the error in the bigfile. From there it should be
easy to find the error in the original sources.
- Verbose and debugging output now also show all profiling
attributes set in the DC-file, plus version information for
the sytlesheets (if available)
- added the option `--norefchecks` to all output-generating targets
(`html`, `pdf`, ...), `validate`, `bigfile` and others. Allows
to build/validate documents without checking internal links
(<... linkend="ID">).
- subcommand `online-docs` now always converts the bigfile that is
created to NovDoc. To keep the same format as the original
sources, specify `--dbnoconv`. To create a DocBook4 bigfile from
DocBook5 sources, specify `--db5todb4`.
Bugfixes:
- #368: db4tonovdoc.xsl generates invalid NovDoc
- #367: daps does not validate fop output
- #366: Spellcheck trying to check the files "Cannot", "stat:", "No",
"such", "file" ...
- #364: DocBook5: Allow validating without checking xrefs
- #363: Allow building a bigfile from invalid sources
- #362: Make the number of parallel jobs configurable
- #361: Externel xrefs are resolved to "????"
- #360: Check Rootid fails on Debian 8.4.0
- #356: file list output should default to pretty format
- #354: Validate the result of online docs when it has been
converted from DB5 to DB4 or novdoc
- #353: Make db5 to db4 conversion default for online-docs
- #351, #348, #342, #282: Fixes for the daps spec-file
- #346: daps-xmlformat writes its messages into the output
- #343: daps-init warning messages
- #340: ePUB builds always show remarks
- #335: on Debian, make install creates incorrect group in
/etc/xml/config
- #330: Enable SVG2Grayscale stylesheet to handle new color
names
- #305: locdrop produces no "non-trans" image tarball for a
complete set
- #301: SVG to PNG conversion creates very large PNGs
- #289: Verbose / Debugging Output Should Show Profiling
Attributes
- #251: Write XSLT to Return xml-model PI
- #226: Let stylesheets know which version of DAPS calls them
- subcommand "clean-all" fails if run from within a directory
that will be removed with that command
- daps fails when called from a directory that no longer exists
Cross-Distribution Support
- first release with full Debian/Ubuntu support--manual
adjustments after the installation are no longer needed
(was required in previous releases for DocBook5 support).
Many thanks to Tomáš Bažant and Christoph Berg for their help!
- GitHub checkouts now also work for Fedora/RedHat and
Debian/Ubuntu--refer to
https://github.com/openSUSE/daps/blob/develop/INSTALL.adoc
for details
- tested on Debian 8.5.0 / Fedora 23 / openSUSE 13.2 and
Leap 42.1 / SUSE Linux Enterprise 12 / Ubuntu 16.0
Misc:
- removed `daps-envconvert`, a script for converting susedoc
ENV-files to DAPS DC-files. susedoc was the predecessor of
DAPS.
Documentation:
- the documentation has _not_ been updated for this release, so
new features mentioned above are not yet covered
fsundermeyer
added a commit
that referenced
this issue
Jun 15, 2016
Stable release DAPS 2.2.0
New Features:
- new subcommand `xmlformat' prettyfies the XML sources of a given
DC-file or ROOTID using daps-xmlformat with
/etc/daps/docbook-xmlformat.conf. (Note: this command will modify the
original XML sources!)
- SVG image support for HTML builds. To use it, make sure to
provide the images in SVG format and reference to them as follows:
<imageobject role="html">
<imagedata fileref="SVG-FILE" format="SVG"/>
</imageobject>
- the new global switch `--jobs` lets you specify how many parallel
jobs to use. The default is set to the number of CPU cores (as was
done in previous versions).
- all commands generating file lists (e.g. `list-srcfiles`) now
generate human readable output on STDOUT. When the output goes
to a pipe or subshell, it will be generated as a a one-liner.
- Debugging of XML errors has been made easier. Sometimes the error
message of the validator only points to an xi:include line, making
it impossible to find the real location of the error. The option
`--novalid` for the target `bigfile` will create the bigfile
without performing a validation check on the sources. A single XML
will be built and validated. Now the error message points to the
exact location of the error in the bigfile. From there it should be
easy to find the error in the original sources.
- Verbose and debugging output now also show all profiling
attributes set in the DC-file, plus version information for
the sytlesheets (if available)
- added the option `--norefchecks` to all output-generating targets
(`html`, `pdf`, ...), `validate`, `bigfile` and others. Allows
to build/validate documents without checking internal links
(<... linkend="ID">).
- subcommand `online-docs` now always converts the bigfile that is
created to NovDoc. To keep the same format as the original
sources, specify `--dbnoconv`. To create a DocBook4 bigfile from
DocBook5 sources, specify `--db5todb4`.
Bugfixes:
- #368: db4tonovdoc.xsl generates invalid NovDoc
- #367: daps does not validate fop output
- #366: Spellcheck trying to check the files "Cannot", "stat:", "No",
"such", "file" ...
- #364: DocBook5: Allow validating without checking xrefs
- #363: Allow building a bigfile from invalid sources
- #362: Make the number of parallel jobs configurable
- #361: Externel xrefs are resolved to "????"
- #360: Check Rootid fails on Debian 8.4.0
- #356: file list output should default to pretty format
- #354: Validate the result of online docs when it has been
converted from DB5 to DB4 or novdoc
- #353: Make db5 to db4 conversion default for online-docs
- #351, #348, #342, #282: Fixes for the daps spec-file
- #346: daps-xmlformat writes its messages into the output
- #343: daps-init warning messages
- #340: ePUB builds always show remarks
- #335: on Debian, make install creates incorrect group in
/etc/xml/config
- #330: Enable SVG2Grayscale stylesheet to handle new color
names
- #305: locdrop produces no "non-trans" image tarball for a
complete set
- #301: SVG to PNG conversion creates very large PNGs
- #289: Verbose / Debugging Output Should Show Profiling
Attributes
- #251: Write XSLT to Return xml-model PI
- #226: Let stylesheets know which version of DAPS calls them
- subcommand "clean-all" fails if run from within a directory
that will be removed with that command
- daps fails when called from a directory that no longer exists
Cross-Distribution Support
- first release with full Debian/Ubuntu support--manual
adjustments after the installation are no longer needed
(was required in previous releases for DocBook5 support).
Many thanks to Tomáš Bažant and Christoph Berg for their help!
- GitHub checkouts now also work for Fedora/RedHat and
Debian/Ubuntu--refer to
https://github.com/openSUSE/daps/blob/develop/INSTALL.adoc
for details
- tested on Debian 8.5.0 / Fedora 23 / openSUSE 13.2 and
Leap 42.1 / SUSE Linux Enterprise 12 / Ubuntu 16.0
Misc:
- removed `daps-envconvert`, a script for converting susedoc
ENV-files to DAPS DC-files. susedoc was the predecessor of
DAPS.
Documentation:
- the documentation has _not_ been updated for this release, so
new features mentioned above are not yet covered
ghost
assigned 
fgerling
added a commit
that referenced
this issue
Mar 27, 2017
Add documentation for the --jobs option. Fixes #362
fsundermeyer
added a commit
that referenced
this issue
Jun 14, 2017
Stable release DAPS 2.4.0:
* New Features:
- added a dependency checker for DAPS (/usr/bin/daps-check-deps)
that makes it easy to check for missing programs/packages
- added a "--lean" parameter to PDF generation for creating PDFs
with a reduced file size (by reducing the quality of included
images); useful for sharing PDFs via e-mail
- added a srylesheet for returning all IDs from an XML file
(daps-xslt/common/get-all-xmlids.xsl)
* Bugfixes:
- #408: The list-file-* commands now work with ROOTIDs from all
possible elements
- #390: When an ID appears twice in the XML and this ID is used
with --rootid, DAPS's error message is unhelpful
- #365: Improved error messages on "File not found" errors
(now tells whether the path was provided by command-line
or config file)
- #371: Make path to the xmlformat config file configurable
- #372: Adjusted debian dependency list
- #373: Check unpack-locdrop
- #379: Output error of list-images-multisrc with
pretty | wc option
- #380: Proper error messages when binaries/packages are missing
- #392: daps-xmlformat writes name of config file into output
- SVGs were not included in ePUBs
(https://bugzilla.suse.com/show_bug.cgi?id=1006204)
- Setting a default value for db5_version in configure.ac
otherwise the DB5 URN in etc/config will be set to an invalid
value if DB5 is not installed when running make
- fixed a few minor issues with the config file parser that was
introduced with 2.3.0 (among them #387)
- Increased Java stacksize for jing
- Improved the DocBook5 -> DocBook 4 (-> NovDoc) conversion
- compatibility: Debian's version of "which" does not support
long parameters
* Documentation:
- Various updates to reflect changes in the code
- #345: added documentatioon for the xmlformat subcommand
- #362: added documentation for the --jobs option
- #363: added documentation about building a bigfile from
invalid sources
- #364: added documentation for the --norefchecks option
- #403: improved documentation about stylesheet customizing
(also see #407)
- #404: clarify doc about listing unused images
- completely revised the doc (spelling, language and grammar)
- doc is not yet 100% on par with the code, but we are getting
closer
* Compatibility:
- successfully tested DAPS on Linux Mint 18.1
fsundermeyer
added a commit
that referenced
this issue
Jun 14, 2017
Stable release DAPS 2.4.0:
* New Features:
- added a dependency checker for DAPS (/usr/bin/daps-check-deps)
that makes it easy to check for missing programs/packages
- added a "--lean" parameter to PDF generation for creating PDFs
with a reduced file size (by reducing the quality of included
images); useful for sharing PDFs via e-mail
- added a srylesheet for returning all IDs from an XML file
(daps-xslt/common/get-all-xmlids.xsl)
* Bugfixes:
- #408: The list-file-* commands now work with ROOTIDs from all
possible elements
- #390: When an ID appears twice in the XML and this ID is used
with --rootid, DAPS's error message is unhelpful
- #365: Improved error messages on "File not found" errors
(now tells whether the path was provided by command-line
or config file)
- #371: Make path to the xmlformat config file configurable
- #372: Adjusted debian dependency list
- #373: Check unpack-locdrop
- #379: Output error of list-images-multisrc with
pretty | wc option
- #380: Proper error messages when binaries/packages are missing
- #392: daps-xmlformat writes name of config file into output
- SVGs were not included in ePUBs
(https://bugzilla.suse.com/show_bug.cgi?id=1006204)
- Setting a default value for db5_version in configure.ac
otherwise the DB5 URN in etc/config will be set to an invalid
value if DB5 is not installed when running make
- fixed a few minor issues with the config file parser that was
introduced with 2.3.0 (among them #387)
- Increased Java stacksize for jing
- Improved the DocBook5 -> DocBook 4 (-> NovDoc) conversion
- compatibility: Debian's version of "which" does not support
long parameters
* Documentation:
- Various updates to reflect changes in the code
- #345: added documentatioon for the xmlformat subcommand
- #362: added documentation for the --jobs option
- #363: added documentation about building a bigfile from
invalid sources
- #364: added documentation for the --norefchecks option
- #403: improved documentation about stylesheet customizing
(also see #407)
- #404: clarify doc about listing unused images
- completely revised the doc (spelling, language and grammar)
- doc is not yet 100% on par with the code, but we are getting
closer
* Compatibility:
- successfully tested DAPS on Linux Mint 18.1
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
By default DAPS calls make with -j <all_cores>. This is not always desireable. Maybe we should use half of them by default and make the setting configurable.
The text was updated successfully, but these errors were encountered: