Skip to content

Various updates to make the walkthrough run more smoothly #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Sep 22, 2016
Merged

Various updates to make the walkthrough run more smoothly #1

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5b90766
Various updates to make the walkthrough run more smoothly
dsyme Sep 22, 2016
59b95c8
Update blob-storage.md
dsyme Sep 22, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 62 additions & 49 deletions docs/fsharp/using-fsharp-on-azure/blob-storage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,14 @@ To parse the connection string, use:

This will return a `CloudStorageAccount`.

### Create some local dummy data

Before we begin, create some dummy local data in the directory of our script. Later you will upload this data.

// Create a dummy file in a local subdirectory to upload
let localFile = __SOURCE_DIRECTORY__ + "/myfile.txt"
File.WriteAllText(localFile, "some data")

### Create the Blob service client

The `CloudBlobClient` type enables you to retrieve containers and blobs stored in Blob storage. Here's one way to create the service client:
Expand All @@ -85,16 +93,15 @@ Now you are ready to write code that reads data from and writes data to Blob sto
This example shows how to create a container if it does not already exist:

// Retrieve a reference to a container.
let container = blobClient.GetContainerReference("mycontainer")
let container = blobClient.GetContainerReference("mydata")

// Create the container if it doesn't already exist.
container.CreateIfNotExists()

By default, the new container is private, meaning that you must specify your storage access key to download blobs from this container. If you want to make the files within the container available to everyone, you can set the container to be public using the following code:

container.SetPermissions(
BlobContainerPermissions(
PublicAccess = BlobContainerPublicAccessType.Blob))
let permissions = BlobContainerPermissions(PublicAccess=BlobContainerPublicAccessType.Blob)
container.SetPermissions(permissions)

Anyone on the Internet can see blobs in a public container, but you can modify or delete them only if you have the appropriate account access key or a shared access signature.

Expand All @@ -104,40 +111,32 @@ Azure Blob Storage supports block blobs and page blobs. In the majority of cases

To upload a file to a block blob, get a container reference and use it to get a block blob reference. Once you have a blob reference, you can upload any stream of data to it by calling the `UploadFromStream` method. This operation will create the blob if it didn't previously exist, or overwrite it if it does exist.

// Retrieve reference to a blob named "myblob".
let blockBlob = container.GetBlockBlobReference("myblob")
// Retrieve reference to a blob named "myblob.txt".
let blockBlob = container.GetBlockBlobReference("myblob.txt")

// Create or overwrite the "myblob" blob with contents from a local file.
// Create or overwrite the "myblob.txt" blob with contents from the local file.
do
use fileStream = File.OpenRead(@"path\myfile")
use fileStream = File.OpenRead localFile
blockBlob.UploadFromStream(fileStream)

## List the blobs in a container

To list the blobs in a container, first get a container reference. You can then use the container's `ListBlobs` method to retrieve the blobs and/or directories within it. To access the rich set of properties and methods for a returned `IListBlobItem`, you must cast it to a `CloudBlockBlob`, `CloudPageBlob`, or `CloudBlobDirectory` object. If the type is unknown, you can use a type check to determine which to cast it to. The following code demonstrates how to retrieve and output the URI of each item in the `photos` container:

// Retrieve reference to a previously created container.
let photosContainer = blobClient.GetContainerReference("photos")
To list the blobs in a container, first get a container reference. You can then use the container's `ListBlobs` method to retrieve the blobs and/or directories within it. To access the rich set of properties and methods for a returned `IListBlobItem`, you must cast it to a `CloudBlockBlob`, `CloudPageBlob`, or `CloudBlobDirectory` object. If the type is unknown, you can use a type check to determine which to cast it to. The following code demonstrates how to retrieve and output the URI of each item in the `mydata` container:

// Loop over items within the container and output the length and URI.
for item in photosContainer.ListBlobs(null, false) do
for item in container.ListBlobs(null, false) do
match item with
| :? CloudBlockBlob as blob ->
Console.WriteLine(
"Block blob of length {0}: {1}",
blob.Properties.Length, blob.Uri)
printfn "Block blob of length %d: %O" blob.Properties.Length blob.Uri

| :? CloudPageBlob as pageBlob ->
Console.WriteLine(
"Page blob of length {0}: {1}",
pageBlob.Properties.Length, pageBlob.Uri)
printfn "Page blob of length %d: %O" pageBlob.Properties.Length pageBlob.Uri

| :? CloudBlobDirectory as directory ->
Console.WriteLine("Directory: {0}", directory.Uri)
printfn "Directory: %O" directory.Uri

| _ ->
Console.WriteLine("Unknown blob type: {0}",
item.GetType().ToString())
printfn "Unknown blob type: %O" (item.GetType())

As shown above, you can name blobs with path information in their names. This creates a virtual directory structure that you can organize and traverse as you would a traditional file system. Note that the directory structure is virtual only - the only resources available in Blob storage are containers and blobs. However, the storage client library offers a `CloudBlobDirectory` object to refer to a virtual directory and simplify the process of working with blobs that are organized in this way.

Expand All @@ -163,9 +162,17 @@ Optionally, you can set the `UseFlatBlobListing` parameter of of the `ListBlobs`

// Loop over items within the container and output the length and URI.
for item in container.ListBlobs(null, true) do
...
match item with
| :? CloudBlockBlob as blob ->
Console.WriteLine(
"Block blob of length {0}: {1}",
blob.Properties.Length, blob.Uri)

and the results look like this:
| _ ->
Console.WriteLine("Unexpected blob type: {0}",
item.GetType().ToString())

and, depending on the current contents of your container, the results look like this:

Block blob of length 4: https://<accountname>.blob.core.windows.net/photos/2015/architecture/description.txt
Block blob of length 314618: https://<accountname>.blob.core.windows.net/photos/2015/architecture/photo3.jpg
Expand All @@ -181,22 +188,19 @@ and the results look like this:

To download blobs, first retrieve a blob reference and then call the `DownloadToStream` method. The following example uses the `DownloadToStream` method to transfer the blob contents to a stream object that you can then persist to a local file.

// Retrieve reference to a blob named "photo1.jpg".
let photoBlob = container.GetBlockBlobReference("photo1.jpg")
// Retrieve reference to a blob named "myblob.txt".
let blobToDownload = container.GetBlockBlobReference("myblob.txt")

// Save blob contents to a file.
do
use fileStream = File.OpenWrite(@"path\myfile")
photoBlob.DownloadToStream(fileStream)
use fileStream = File.OpenWrite(__SOURCE_DIRECTORY__ + "/path/download.txt")
blobToDownload.DownloadToStream(fileStream)

You can also use the `DownloadToStream` method to download the contents of a blob as a text string.

// Retrieve reference to a blob named "myblob.txt"
let photoBlob2 = container.GetBlockBlobReference("myblob.txt")

let text =
use memoryStream = new MemoryStream()
photoBlob2.DownloadToStream(memoryStream)
blobToDownload.DownloadToStream(memoryStream)
Text.Encoding.UTF8.GetString(memoryStream.ToArray())

## Delete blobs
Expand All @@ -205,10 +209,10 @@ To delete a blob, first get a blob reference and then call the
`Delete` method on it.

// Retrieve reference to a blob named "myblob.txt".
let blockBlob3 = container.GetBlockBlobReference("myblob.txt")
let blobToDelete = container.GetBlockBlobReference("myblob.txt")

// Delete the blob.
blockBlob3.Delete()
blobToDelete.Delete()

## List blobs in pages asynchronously

Expand Down Expand Up @@ -254,10 +258,23 @@ Because the sample method calls an asynchronous method, it must be prefaced with
if (continuationToken <> null) then
do! loop continuationToken (i+1)
}

do! loop null 1
}

We can now use this asynchronous routine as follows. First we upload some dummy data (using the local
file created earlier in this tutorial).

// Create some dummy data by upoading the same file over andd over again
for i in 1 .. 100 do
let blob = container.GetBlockBlobReference("myblob" + string i + ".txt")
use fileStream = System.IO.File.OpenRead(localFile)
blob.UploadFromStream fileStream

Now, call the routine. We use ``Async.RunSynchronously`` to force the execution of the asynchronous operation.

ListBlobsSegmentedInFlatListing container |> Async.RunSynchronously

## Writing to an append blob

An append blob is optimized for append operations, such as logging. Like a block blob, an append blob is comprised of blocks, but when you add a new block to an append blob, it is always appended to the end of the blob. You cannot update or delete an existing block in an append blob. The block IDs for an append blob are not exposed as they are for a block blob.
Expand Down Expand Up @@ -312,10 +329,10 @@ For further information see [Managing Concurrency in Microsoft Azure Storage](ht

## Naming containers

Every blob in Azure storage must reside in a container. The container forms part of the blob name. For example, `mycontainer` is the name of the container in these sample blob URIs:
Every blob in Azure storage must reside in a container. The container forms part of the blob name. For example, `mydata` is the name of the container in these sample blob URIs:

https://storagesample.blob.core.windows.net/mycontainer/blob1.txt
https://storagesample.blob.core.windows.net/mycontainer/photos/myphoto.jpg
https://storagesample.blob.core.windows.net/mydata/blob1.txt
https://storagesample.blob.core.windows.net/mydata/photos/myphoto.jpg

A container name must be a valid DNS name, conforming to the following naming rules:

Expand Down Expand Up @@ -345,25 +362,21 @@ For details on encrypting blob data, see [the .NET guide for blob storage sectio

## Next steps

Now that you've learned the basics of Blob storage, follow these links
to learn more.
Now that you've learned the basics of Blob storage, follow these links to learn more.

### Microsoft Azure Storage Explorer
### Tools
- [F# AzureStorageTypeProvider](http://fsprojects.github.io/AzureStorageTypeProvider/) An F# Type Provider which can be used to explore Blob, Table and Queue Azure Storage assets and easily apply CRUD operations on them.
- [FSharp.Azure.Storage](https://github.com/fsprojects/FSharp.Azure.Storage) An F# API for using Microsoft Azure Table Storage service
- [Microsoft Azure Storage Explorer (MASE)](https://azure.microsoft.com/en-us/documentation/articles/vs-azure-tools-storage-manage-with-storage-explorer/) is a free, standalone app from Microsoft that enables you to work visually with Azure Storage data on Windows, OS X, and Linux.

### Blob storage samples

- [Getting Started with Azure Blob Storage in C#](https://azure.microsoft.com/documentation/samples/storage-blob-dotnet-getting-started/)

### Blob storage reference

- [Storage Client Library for .NET reference](http://go.microsoft.com/fwlink/?LinkID=390731&clcid=0x409)
- [REST API reference](http://msdn.microsoft.com/library/azure/dd179355)

### Conceptual guides
### Related guides

- [Getting Started with Azure Blob Storage in C#](https://azure.microsoft.com/documentation/samples/storage-blob-dotnet-getting-started/)
- [Transfer data with the AzCopy command-line utility](https://azure.microsoft.com/en-us/documentation/articles/storage-use-azcopy/)
- [Azure Storage Team Blog](http://blogs.msdn.com/b/windowsazurestorage/)
- [Configuring Connection Strings](http://msdn.microsoft.com/library/azure/ee758697.aspx)
- [.NET client library reference](http://go.microsoft.com/fwlink/?LinkID=390731&clcid=0x409)
- [REST API reference](http://msdn.microsoft.com/library/azure/dd179355)
- [Azure Storage Team Blog](http://blogs.msdn.com/b/windowsazurestorage/)