Python convenience function
canopener(filename, mode='r') for opening files.
Local files behave identically to
URLs can also be passed as the filename and opened for reading.
urllib2.urlopen() is used under the covers, so it has equivalent support:
S3 URLs can also be read if the boto module is installed and the
AWS_SECRET_ACCESS_KEY environment variables are set. The file is downloaded to a temporary local file on open:
>>> os.environ['AWS_ACCESS_KEY_ID'] = 'key' >>> os.environ['AWS_SECRET_ACCESS_KEY'] = 'secret' >>> canopener('s3://bucket/file.txt')
Any paths with ".gz" or ".bz2" extensions are transparently decompressed:
>>> canopener('local_file.txt.gz') >>> canopener('local_file.txt.bz2') >>> canopener('http://remote/file.txt.gz') >>> canopener('s3://bucket/file.txt.gz')
There's also transparent compression when writing to local files:
>>> canopener('local_file.txt.gz', 'w')