Skip to content

Latest commit

 

History

History
98 lines (63 loc) · 4.2 KB

gettingstarted.rst

File metadata and controls

98 lines (63 loc) · 4.2 KB
Raw

Getting Started

Linux

Download, extract and set permissions

wget -O bp_linux.tar.gz https://github.com/Azure/blobporter/releases/download/v0.6.20/bp_linux.tar.gz
tar -xvf bp_linux.tar.gz linux_amd64/blobporter
chmod +x ~/linux_amd64/blobporter
cd ~/linux_amd64

Set environment variables: :

export ACCOUNT_NAME=<STORAGE_ACCOUNT_NAME>
export ACCOUNT_KEY=<STORAGE_ACCOUNT_KEY>

Note

You can also set these values via options

Windows

Download BlobPorter.exe

Set environment variables (if using the command prompt): :

set ACCOUNT_NAME=<STORAGE_ACCOUNT_NAME>
set ACCOUNT_KEY=<STORAGE_ACCOUNT_KEY>

Set environment variables (if using PowerShell): :

$env:ACCOUNT_NAME="<STORAGE_ACCOUNT_NAME>"
$env:ACCOUNT_KEY="<STORAGE_ACCOUNT_KEY>"

Command Options

-f, --source_file (string) URL, Azure Blob or S3 Endpoint,

file or files (e.g. /data/*.gz) to upload.

-c, --container_name (string) Container name (e.g. mycontainer). -n, --blob_name (string) Blob name (e.g. myblob.txt) or prefix for download scenarios. -g, --concurrent_workers (int) Number of go-routines for parallel upload. -r, --concurrent_readers (int) Number of go-routines for parallel reading of the input. -b, --block_size (string) Desired size of each blob block.

Can be specified as an integer byte count or integer suffixed with B, KB or MB.

-a, --account_name (string) Storage account name (e.g. mystorage).

Can also be specified via the ACCOUNT_NAME environment variable.

-k, --account_key (string) Storage account key string.

Can also be specified via the ACCOUNT_KEY environment variable.

-s, --http_timeout (int) HTTP client timeout in seconds. Default value is 600s. -t, --transfer_type (string) Defines the source and target of the transfer.

Must be one of :

file-blockblob, file-pageblob, http-blockblob, 
http-pageblob, blob-file, pageblock-file (alias of blob-file), 
blockblob-file (alias of blob-file), http-file, 
blob-pageblob, blob-blockblob, s3-pageblob and s3-blockblob.
-m, --compute_blockmd5 (bool) If set, block level MD5 has will be computed and included

as a header when the block is sent to blob storage.

Default is false.

-q, --quiet_mode (bool) If set, the progress indicator is not displayed.

The files to transfer, errors, warnings and transfer completion summary is still displayed.

-x, --files_per_transfer (int) Number of files in a batch transfer. Default is 500. -h, --handles_per_file (int) Number of open handles for concurrent reads and writes per file. Default is 2. -i, --remove_directories (bool) If set blobs are downloaded or uploaded without keeping the directory structure of the source.

Not applicable when the source is a HTTP endpoint.

-o, --read_token_exp (int) Expiration in minutes of the read-only access token that will be generated to read from S3 or Azure Blob sources.

Default value: 360.

-l, --transfer_status (string) Transfer status file location.

If set, blobporter will use this file to track the status of the transfer.

By referencing the same status file after a failure, the transfer will skip files already transferred.

If the transfer is successful a summary will be appended.

-u, --endpoint_suffix Endpoint suffix to be used for blob sources or targets.

Default is blob.core.windows.net