Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100755 20 lines (17 sloc) 0.54 kb
db24328 @thiruvel Initial push
authored
1 #!/bin/bash
2 # Copyright (c) 2012 Yahoo! Inc. All rights reserved.
3 # Copyrights licensed under the New BSD License. See the accompanying LICENSE file for terms.
4 #
5
6 while read items
7 do
8 INPUTFILE=`echo $items | cut -d: -f1 | awk '{print $2}'`
9 OUTPUTFILE=`echo $items | cut -d: -f2`
10
11 echo "Processing files - $INPUTFILE - $OUTPUTFILE" >&2
12 $HADOOP_HOME/bin/hadoop fs -get $INPUTFILE .
13 FILE=`basename $INPUTFILE`
14 bunzip2 $FILE
15 NFILE=${FILE%%".bz2"}
16 gzip $NFILE
17 $HADOOP_HOME/bin/hadoop fs -put $NFILE.gz $OUTPUTFILE
18 rm -f $NFILE.gz
19 done
Something went wrong with that request. Please try again.