Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
read chunks of a specific size from a pipe
C
branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
README
chunker.c

README

chunker - split streams into handy chunks

by Stefan Tomanek <stefan.tomanek@wertarbyte.de>

http://github.com/wertarbyte/chunker

The program chunker reads a specified amount of data from STDIN and
writes it to STDOUT. Its exit code indicates whether the end of
input data has been reached (code 1) or whether the specified limit
has been reached (code 0) and more data is waiting.

The program can be used to split data coming from a pipe and
redirect it to different output pipes.

This example reads from STDIN and calls ftpupload for every 500 MB
it reads. It then checks chunker's exit code if there is more data
waiting, repeating the process until no more data is arriving at
STDIN.

#!/bin/bash
# 500 MB
CHUNKSIZE=$((1024*1024*500))
MORE=1
while [ "$MORE" -eq 0 ]; do
    chunker "$CHUNKSIZE" | \
        ftpupload -
    # get exit status of chunker
    MORE="${PIPESTATUS[0]}"
done
Something went wrong with that request. Please try again.