Skip to content
master
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 

README.md

NAME

iconv-chunks - Process huge files with iconv

SYNOPSIS

iconv-chunks [iconv-options]

DESCRIPTION

The standard iconv program reads the entire input file into memory, which doesn't work for large files (such as database exports).

This script is just a wrapper that processes the input file in manageable chunks and writes it to standard output.

The first argument is the input filename (use - to specify standard input). Anything else is passed through to iconv.

The real iconv needs to be somewhere in your PATH.

EXAMPLES

# Convert latin1 to utf-8:
./iconv-chunks database.txt -f latin1 -t utf-8 > out.txt

# Input filename of - means standard input:
./iconv-chunks - -f iso8859-1 -t utf8 < database.txt > out.txt

# More complex example, using compressed input/output to minimize disk use:
zcat database.txt.gz | ./iconv-chunks - -f iso8859-1 -t utf8 | \
gzip - > database-utf.dump.gz

AUTHOR

Maurice Aubrey maurice.aubrey+iconv@gmail.com

LICENSE

MIT

About

Process huge files with iconv

Resources

License

Releases

No releases published

Languages

You can’t perform that action at this time.