Skip to content

cym13/multiproc

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

13 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Description
===========

Multiprocessing from the command-line.

Multiproc allows one to use a pool of workers with bash. It provides a
reliable way to launch a lot of concurrent processes while keeping the
control of the number of these processes.

This program uses python3.

Documentation
=============

Usage: multiproc [-h] [-0] [-v] [-p N] [-1] FORMAT

Arguments:
    FORMAT     The command to be run in each process.
               "%s"    input string
               "%n"    number of the process
               "%%"    litteral "%"

Options:
    -h, --help          Print this help and exit.
    -v, --verbose       Set verbose output
    -0                  Use NUL as delimiter
                        This doesn't cover potential command output.
    -p, --process N     Number of processes to be used.
                        Default is the number of CPU.
    -1, --uniq          Equivalent to -p1

Example: download multiple files where urls.txt contains the urls list

    $ cat urls.txt | multiproc -p 10 "wget %s -O file.%n"

Example
=======

Download multiple files where urls.txt contains the urls list with 10 processes:

    $ cat urls.txt | multiproc -p 10 "wget %s -O file.%n"

Work with files

    $ find . -iname "*.c" -print0 | multiproc -v -0 "gcc %n"


Install
=======

The easiest is

    pip install multiproc

or, in this directory

    python3 setup install


Dependencies
============

docopt   https://github.com/docopt/docopt or "pip install docopt"

License
=======

This program is under the GPLv3 License.

You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.

Contact
=======

Main developper: Cédric Picard
Email:           cedric.picard@efrei.net
Twitter:         @Cym13
GPG:             383A 76B9 D68D 2BD6 9D2B  4716 E3B9 F4FE 5CED 42CB

About

Allows multiprocessing from the bash command-line.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages