[AZURE.INCLUDE storage-selector-blob-include]
Azure Blob storage is a service that stores unstructured data in the cloud as objects/blobs. Blob storage can store any type of text or binary data, such as a document, media file, or application installer. Blob storage is also referred to as object storage.
This tutorial shows how to write .NET code for some common scenarios using Azure Blob storage. Scenarios covered include uploading, listing, downloading, and deleting blobs.
Estimated time to complete: 45 minutes
Prerequisities:
- Microsoft Visual Studio
- Azure Storage Client Library for .NET
- Azure Configuration Manager for .NET
- An Azure storage account
[AZURE.INCLUDE storage-dotnet-client-library-version-include]
For additional examples using Blob storage, see Getting Started with Azure Blob Storage in .NET. You can download the sample application and run it, or browse the code on GitHub.
[AZURE.INCLUDE storage-blob-concepts-include]
[AZURE.INCLUDE storage-create-account-include]
[AZURE.INCLUDE storage-development-environment-include]
Add the following using statements to the top of the program.cs file:
using Microsoft.Azure; // Namespace for CloudConfigurationManager
using Microsoft.WindowsAzure.Storage; // Namespace for CloudStorageAccount
using Microsoft.WindowsAzure.Storage.Blob; // Namespace for Blob storage types
[AZURE.INCLUDE storage-cloud-configuration-manager-include]
The CloudBlobClient class enables you to retrieve containers and blobs stored in Blob storage. Here's one way to create the service client:
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
Now you are ready to write code that reads data from and writes data to Blob storage.
[AZURE.INCLUDE storage-container-naming-rules-include]
This example shows how to create a container if it does not already exist:
// Retrieve storage account from connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Create the blob client.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve a reference to a container.
CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");
// Create the container if it doesn't already exist.
container.CreateIfNotExists();
By default, the new container is private, meaning that you must specify your storage access key to download blobs from this container. If you want to make the files within the container available to everyone, you can set the container to be public using the following code:
container.SetPermissions(
new BlobContainerPermissions { PublicAccess = BlobContainerPublicAccessType.Blob });
Anyone on the Internet can see blobs in a public container, but you can modify or delete them only if you have the appropriate account access key or a shared access signature.
Azure Blob Storage supports block blobs and page blobs. In the majority of cases, block blob is the recommended type to use.
To upload a file to a block blob, get a container reference and use it to get a block blob reference. Once you have a blob reference, you can upload any stream of data to it by calling the UploadFromStream method. This operation will create the blob if it didn't previously exist, or overwrite it if it does exist.
The following example shows how to upload a blob into a container and assumes that the container was already created.
// Retrieve storage account from connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Create the blob client.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve reference to a previously created container.
CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");
// Retrieve reference to a blob named "myblob".
CloudBlockBlob blockBlob = container.GetBlockBlobReference("myblob");
// Create or overwrite the "myblob" blob with contents from a local file.
using (var fileStream = System.IO.File.OpenRead(@"path\myfile"))
{
blockBlob.UploadFromStream(fileStream);
}
To list the blobs in a container, first get a container reference. You
can then use the container's ListBlobs method to retrieve the blobs and/or directories
within it. To access the rich set of properties and methods for a
returned IListBlobItem, you must cast it to a CloudBlockBlob,
CloudPageBlob, or CloudBlobDirectory object. If the type is unknown, you can use a
type check to determine which to cast it to. The following code
demonstrates how to retrieve and output the URI of each item in
the photos container:
// Retrieve storage account from connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Create the blob client.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve reference to a previously created container.
CloudBlobContainer container = blobClient.GetContainerReference("photos");
// Loop over items within the container and output the length and URI.
foreach (IListBlobItem item in container.ListBlobs(null, false))
{
if (item.GetType() == typeof(CloudBlockBlob))
{
CloudBlockBlob blob = (CloudBlockBlob)item;
Console.WriteLine("Block blob of length {0}: {1}", blob.Properties.Length, blob.Uri);
}
else if (item.GetType() == typeof(CloudPageBlob))
{
CloudPageBlob pageBlob = (CloudPageBlob)item;
Console.WriteLine("Page blob of length {0}: {1}", pageBlob.Properties.Length, pageBlob.Uri);
}
else if (item.GetType() == typeof(CloudBlobDirectory))
{
CloudBlobDirectory directory = (CloudBlobDirectory)item;
Console.WriteLine("Directory: {0}", directory.Uri);
}
}
As shown above, you can name blobs with path information in their names. This creates a virtual directory structure that you can organize and traverse as you would a traditional file system. Note that the directory structure is virtual only - the only resources available in Blob storage are containers and blobs. However, the storage client library offers a CloudBlobDirectory object to refer to a virtual directory and simplify the process of working with blobs that are organized in this way.
For example, consider the following set of block blobs in a container named photos:
photo1.jpg
2010/architecture/description.txt
2010/architecture/photo3.jpg
2010/architecture/photo4.jpg
2011/architecture/photo5.jpg
2011/architecture/photo6.jpg
2011/architecture/description.txt
2011/photo7.jpg
When you call ListBlobs on the 'photos' container (as in the above sample), a hierarchical listing is returned. It contains both CloudBlobDirectory and CloudBlockBlob objects, representing the directories and blobs in the container, respectively. The resulting output looks like:
Directory: https://<accountname>.blob.core.windows.net/photos/2010/
Directory: https://<accountname>.blob.core.windows.net/photos/2011/
Block blob of length 505623: https://<accountname>.blob.core.windows.net/photos/photo1.jpg
Optionally, you can set the UseFlatBlobListing parameter of of the ListBlobs method to true. In this case, every blob in the container is returned as a CloudBlockBlob object. The call to ListBlobs to return a flat listing looks like this:
// Loop over items within the container and output the length and URI.
foreach (IListBlobItem item in container.ListBlobs(null, true))
{
...
}
and the results look like this:
Block blob of length 4: https://<accountname>.blob.core.windows.net/photos/2010/architecture/description.txt
Block blob of length 314618: https://<accountname>.blob.core.windows.net/photos/2010/architecture/photo3.jpg
Block blob of length 522713: https://<accountname>.blob.core.windows.net/photos/2010/architecture/photo4.jpg
Block blob of length 4: https://<accountname>.blob.core.windows.net/photos/2011/architecture/description.txt
Block blob of length 419048: https://<accountname>.blob.core.windows.net/photos/2011/architecture/photo5.jpg
Block blob of length 506388: https://<accountname>.blob.core.windows.net/photos/2011/architecture/photo6.jpg
Block blob of length 399751: https://<accountname>.blob.core.windows.net/photos/2011/photo7.jpg
Block blob of length 505623: https://<accountname>.blob.core.windows.net/photos/photo1.jpg
To download blobs, first retrieve a blob reference and then call the DownloadToStream method. The following example uses the DownloadToStream method to transfer the blob contents to a stream object that you can then persist to a local file.
// Retrieve storage account from connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Create the blob client.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve reference to a previously created container.
CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");
// Retrieve reference to a blob named "photo1.jpg".
CloudBlockBlob blockBlob = container.GetBlockBlobReference("photo1.jpg");
// Save blob contents to a file.
using (var fileStream = System.IO.File.OpenWrite(@"path\myfile"))
{
blockBlob.DownloadToStream(fileStream);
}
You can also use the DownloadToStream method to download the contents of a blob as a text string.
// Retrieve storage account from connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Create the blob client.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve reference to a previously created container.
CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");
// Retrieve reference to a blob named "myblob.txt"
CloudBlockBlob blockBlob2 = container.GetBlockBlobReference("myblob.txt");
string text;
using (var memoryStream = new MemoryStream())
{
blockBlob2.DownloadToStream(memoryStream);
text = System.Text.Encoding.UTF8.GetString(memoryStream.ToArray());
}
To delete a blob, first get a blob reference and then call the Delete method on it.
// Retrieve storage account from connection string.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Create the blob client.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve reference to a previously created container.
CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");
// Retrieve reference to a blob named "myblob.txt".
CloudBlockBlob blockBlob = container.GetBlockBlobReference("myblob.txt");
// Delete the blob.
blockBlob.Delete();
If you are listing a large number of blobs, or you want to control the number of results you return in one listing operation, you can list blobs in pages of results. This example shows how to return results in pages asynchronously, so that execution is not blocked while waiting to return a large set of results.
This example shows a flat blob listing, but you can also perform a hierarchical listing, by setting the useFlatBlobListing parameter of the ListBlobsSegmentedAsync method to false.
Because the sample method calls an asynchronous method, it must be prefaced with the async keyword, and it must return a Task object. The await keyword specified for the ListBlobsSegmentedAsync method suspends execution of the sample method until the listing task completes.
async public static Task ListBlobsSegmentedInFlatListing(CloudBlobContainer container)
{
//List blobs to the console window, with paging.
Console.WriteLine("List blobs in pages:");
int i = 0;
BlobContinuationToken continuationToken = null;
BlobResultSegment resultSegment = null;
//Call ListBlobsSegmentedAsync and enumerate the result segment returned, while the continuation token is non-null.
//When the continuation token is null, the last page has been returned and execution can exit the loop.
do
{
//This overload allows control of the page size. You can return all remaining results by passing null for the maxResults parameter,
//or by calling a different overload.
resultSegment = await container.ListBlobsSegmentedAsync("", true, BlobListingDetails.All, 10, continuationToken, null, null);
if (resultSegment.Results.Count<IListBlobItem>() > 0) { Console.WriteLine("Page {0}:", ++i); }
foreach (var blobItem in resultSegment.Results)
{
Console.WriteLine("\t{0}", blobItem.StorageUri.PrimaryUri);
}
Console.WriteLine();
//Get the continuation token.
continuationToken = resultSegment.ContinuationToken;
}
while (continuationToken != null);
}
An append blob is a new type of blob, introduced with version 5.x of the Azure storage client library for .NET. An append blob is optimized for append operations, such as logging. Like a block blob, an append blob is comprised of blocks, but when you add a new block to an append blob, it is always appended to the end of the blob. You cannot update or delete an existing block in an append blob. The block IDs for an append blob are not exposed as they are for a block blob.
Each block in an append blob can be a different size, up to a maximum of 4 MB, and an append blob can include a maximum of 50,000 blocks. The maximum size of an append blob is therefore slightly more than 195 GB (4 MB X 50,000 blocks).
The example below creates a new append blob and appends some data to it, simulating a simple logging operation.
//Parse the connection string for the storage account.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
Microsoft.Azure.CloudConfigurationManager.GetSetting("StorageConnectionString"));
//Create service client for credentialed access to the Blob service.
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
//Get a reference to a container.
CloudBlobContainer container = blobClient.GetContainerReference("my-append-blobs");
//Create the container if it does not already exist.
container.CreateIfNotExists();
//Get a reference to an append blob.
CloudAppendBlob appendBlob = container.GetAppendBlobReference("append-blob.log");
//Create the append blob. Note that if the blob already exists, the CreateOrReplace() method will overwrite it.
//You can check whether the blob exists to avoid overwriting it by using CloudAppendBlob.Exists().
appendBlob.CreateOrReplace();
int numBlocks = 10;
//Generate an array of random bytes.
Random rnd = new Random();
byte[] bytes = new byte[numBlocks];
rnd.NextBytes(bytes);
//Simulate a logging operation by writing text data and byte data to the end of the append blob.
for (int i = 0; i < numBlocks; i++)
{
appendBlob.AppendText(String.Format("Timestamp: {0:u} \tLog Entry: {1}{2}",
DateTime.UtcNow, bytes[i], Environment.NewLine));
}
//Read the append blob to the console window.
Console.WriteLine(appendBlob.DownloadText());
See Understanding Block Blobs, Page Blobs, and Append Blobs for more information about the differences between the three types of blobs.
By default, Azure Storage keeps your data secure by limiting access to the account owner, who is in possession of the account access keys. When you need to share blob data in your storage account, it is important to do so without compromising the security of your account access keys. Additionally, you can encrypt blob data to ensure that it is secure going over the wire and in Azure Storage.
[AZURE.INCLUDE storage-account-key-note-include]
By default, the blob data in your storage account is accessible only to storage account owner. Authenticating requests against Blob storage requires the account access key by default. However, you may wish to make certain blob data available to other users. You have two options:
- Anonymous access: You can make a container or its blobs publicly available for anonymous access. See Manage anonymous read access to containers and blobs for more information.
- Shared access signatures: You can provide clients with a shared access signature (SAS), which provides delegated access to a resource in your storage account, with permissions that you specify and over an interval that you specify. See Shared Access Signatures: Understanding the SAS Model for more information.
Azure Storage supports encrypting blob data both at the client and on the server:
- Client-side encryption: The Storage Client Library for .NET supports encrypting data within client applications before uploading to Azure Storage, and decrypting data while downloading to the client. The library also supports integration with Azure Key Vault for storage account key management. See Client-Side Encryption with .NET for Microsoft Azure Storage for more information. Also see Tutorial: Encrypt and decrypt blobs in Microsoft Azure Storage using Azure Key Vault.
- Server-side encryption: Azure Storage now supports server-side encryption. See Azure Storage Service Encryption for Data at Rest (Preview).
Now that you've learned the basics of Blob storage, follow these links to learn more.