From 22910cf09db3d2a7299d5525b1712e82e39b3f50 Mon Sep 17 00:00:00 2001 From: hzoppetti Date: Thu, 18 Jul 2019 10:11:30 -0400 Subject: [PATCH] added CLI back in, added note about bucket label rules --- .../how-to-use-object-storage/index.md | 168 ++++++++++++++++-- 1 file changed, 158 insertions(+), 10 deletions(-) diff --git a/docs/platform/object-storage/how-to-use-object-storage/index.md b/docs/platform/object-storage/how-to-use-object-storage/index.md index 1500ec7a8a5..4f3855add7e 100644 --- a/docs/platform/object-storage/how-to-use-object-storage/index.md +++ b/docs/platform/object-storage/how-to-use-object-storage/index.md @@ -15,7 +15,6 @@ contributor: external_resources: - '[S3cmd usage and command reference](https://s3tools.org/usage)' --- - {{< note >}} Object Storage is currently in a closed early access Beta, and you may not have access to Object Storage through the Cloud Manager or other tools. To gain access to the Early Access Program (EAP), open up a Customer Support ticket noting that you'd like to be included in the program, or e-mail objbeta@linode.com -- beta access is completely free. @@ -26,11 +25,21 @@ Linode's Object Storage is a globally-available, S3- and Swift-compatible method Additionally, **Object Storage does not require the use of a Linode.** Instead, Object Storage gives each object a unique URL with which you can access your data. An object can be publicly accessible, or you can set it to be private and only visible to you. This makes Object Storage great for sharing and storing unstructured data like images, documents, archives, streaming media assets, and file backups, and the amount of data you store can range from small collections of files up to massive libraries of information. Lastly, Linode Object Storage has the built-in ability to [host a static site](/docs/platform/object-storage/host-static-site-object-storage/). -Below you will find instructions on how to connect to Object Storage, and how to upload and access objects: +In this guide you will learn: + + - How to get set-up for object storage by [creating an Object Storage Key Pair](/docs/platform/object-storage/how-to-use-object-storage/#object-storage-key-pair). + + - About the variety of first-party and third-party [tools available](/docs/platform/object-storage/how-to-use-object-storage/#object-storage-tools) to access and use the service. + + - How to connect to Object Storage, how to upload and access objects, and how to host a static site using: + + - [Cloud Manager](/docs/platform/object-storage/how-to-use-object-storage/#cloud-manager) -1. First, you'll need to create a [*key pair*](#object-storage-key-pair) to access the service. + - [Linode CLI](/docs/platform/object-storage/how-to-use-object-storage/#linode-cli) -1. Then, you'll use choose from a variety of available [first-party and third-party tools](#object-storage-tools) to access and use the service. + - [s3cmd](/docs/platform/object-storage/how-to-use-object-storage/#s3cmd) + + - [Cyberduck](/docs/platform/object-storage/how-to-use-object-storage/#cyberduck) ## Object Storage Key Pair @@ -78,6 +87,8 @@ There are a number of tools that are available to help manage Linode Object Stor - The [Linode Cloud Manager](#cloud-manager) can be used to create buckets (you are currently not able to upload objects to a bucket from the Cloud Manager). +- The [Linode CLI](#linode-cli) has an Object Storage plugin and can be used to create and remove buckets, add and remove objects, and convert a bucket into a static site from the command line. + - [s3cmd](#s3cmd) is a powerful command line utility that can be used with any S3-compatible object storage service, including Linode's. s3cmd can be used to create and remove buckets, add and remove objects, convert a bucket into a static site from the command line, plus other functions like syncing entire directories up to a bucket. - [Cyberduck](#cyberduck) is a graphical utility available for Windows and macOS and is a great option if you prefer a GUI tool. @@ -98,12 +109,131 @@ The Cloud Manager provides a web interface for creating buckets. To create a buc ![The Create a Bucket menu.](object-storage-create-a-bucket.png) -3. Add a label for your bucket. A bucket's label needs to be unique within the cluster that it lives in, and this includes buckets of the same name on different Linode accounts. If the label you enter is already in use, you will have to choose a different label. +3. Add a label for your bucket. + + {{< note >}} +Bucket labels need to be unique within the same cluster, including buckets on other users' Linode accounts. If the label you enter is already in use, you will have to choose a different label. Additionally, bucket labels have the following rules:
+   •  Cannot be formatted as IP addresses
+   •  Must be between 3 and 63 characters in length
+   •  Can only contain lower-case characters, numbers, periods, and dashes
+   •  Must start with a lowercase letter or number
+   •  Cannot contain underscores (_), end with a dash (-) or period (.), have consecutive periods (.), or use dashes (-) adjacent to periods (.) +{{< /note >}} 4. Choose a cluster location for the bucket to reside in. 5. Click **Submit**. You are now ready to upload objects to your bucket using one of the other tools outlined in this guide. +## Linode CLI + +The Linode Command Line Interface (CLI) is a command line utility that allows you complete control over your Linode account. With the Object Storage plugin, you can also create and remove buckets, upload objects, and more. + +### Install and Configure the CLI + +1. Download the Linode CLI, or, if you have already downloaded it, make sure it has been upgraded to the latest version: + + pip install linode-cli --upgrade + +1. Configure the Object Storage plugin: + + linode-cli obj --help + + You will be prompted to enter in your Personal Access Token and default settings for deploying new Linodes. + +1. Install the `boto` module: + + pip install boto + +Now you are ready to create buckets and upload objects. + +### Create a Bucket with the CLI + +To create a bucket with the Linode CLI, issue the `mb` command. + + linode-cli obj mb my-example-bucket + +{{< note >}} +Bucket labels need to be unique within the same cluster, including buckets on other users' Linode accounts. If the label you enter is already in use, you will have to choose a different label. Additionally, bucket labels have the following rules:
+   •  Cannot be formatted as IP addresses
+   •  Must be between 3 and 63 characters in length
+   •  Can only contain lower-case characters, numbers, periods, and dashes
+   •  Must start with a lowercase letter or number
+   •  Cannot contain underscores (_), end with a dash (-) or period (.), have consecutive periods (.), or use dashes (-) adjacent to periods (.) +{{< /note >}} + +To delete a bucket, issue the `rb` command: + + linode-cli obj rb my-example-bucket + +If your bucket has objects in it, you will not be able to immediately delete it from the Linode CLI. Instead, remove the objects first, then delete the bucket. The [s3cmd](/docs/platform/object-storage/how-to-use-object-storage/#s3cmd) tool has commands for deleting all objects from a bucket, and it can also force-delete a bucket with objects in it. + +### Upload, Download, and Delete an Object with the CLI + +1. As an example object, create a text file and fill it with some example text. + + echo 'Hello World!' > example.txt + +1. To upload an object to a bucket using the Linode CLI, issue the `put` command. Supply the object name as the first parameter and the bucket label as the second: + + linode-cli obj put --acl-public example.txt my-example-bucket + + The file will now be accessible at the URL `http://my-example-bucket.us-east-1.linodeobjects.com/example.txt`. + + {{< note >}} +The `--acl-public` flag is used to make the object publicly accessible, meaning that you will be able to access the object from its URL. By default, all objects are set to private. To make a public file private, or a private file public, use the `setacl` command and supply the corresponding flag. + +For instance, if you want to make a public file private, you would supply the `--acl-private` flag: + + linode-cli obj setacl --acl-private my-example-bucket example.txt +{{}} + +1. To download an object, issue the `get` command. Supply the label of the bucket as the first parameter and the name of the file as the second: + + linode-cli obj get my-example-bucket example.txt + +1. To delete an object, issue the `rm` or `del` command. Supply the label of the bucket as the first parameter and the name of the object as the second: + + linode-cli obj rm my-example-bucket example.txt + +### Create a Static Site with the CLI + +To create a static website from a bucket: + +1. Issue the `ws-create` command, including the `--ws-index` and `--ws-error` flags: + + linode-cli obj ws-create my-example-bucket --ws-index=index.html --ws-error=404.html + + The `--ws-index` and `--ws-error` flags specify which objects the bucket should use to serve the static site's index page and error page, respectively. + +1. You need to separately upload the `index.html` and `404.html` files (or however you have named the index and error pages) to your bucket: + + echo 'Index page' > index.html + echo 'Error page' > 404.html + linode-cli obj put index.html 404.html my-example-bucket + +1. Set the `--aclpublic` flag on both the `index.html` and `404.html` files: + + linode-cli obj setacl --acl-public my-example-bucket index.html + linode-cli obj setacl --acl-public my-example-bucket 404.html + +1. Your static site is accessed from a different URL than the generic URL for your Object Storage bucket. Static sites are available at the `website-us-east-1` subdomain. Using `my-example-bucket` as an example, navigate to `http://my-example-bucket.website-us-east-1.linodeobjects.com`. + +For more information on hosting static websites from Linode Object Storage, see our [Host a Static Site on Linode's Object Storage](/docs/platform/object-storage/host-static-site-object-storage/) guide. + +### Other CLI Commands + +To get a list of all available buckets, issue the `ls` command: + + linode-cli obj ls + +To get a list of all objects in a bucket, issue the `ls` command with the label of a bucket: + + linode-cli obj ls my-example-bucket + +For a complete list of commands available with the Object Storage plugin, use the `--help` flag: + + linode-cli obj --help + ## s3cmd s3cmd is a command line utility that you can use for any S3-compatible Object Storage. @@ -165,10 +295,19 @@ You are now ready to use s3cmd to create a bucket in Object Storage. ### Create a Bucket with s3cmd -You can create a bucket with s3cmd issuing the following `mb` command, replacing `my-example-bucket` with the name of the bucket you would like to create. Bucket names need to be unique within the same cluster, including buckets on other Linode accounts. If you choose a name for your bucket that someone else has already created, you will have to choose a different name: +You can create a bucket with s3cmd issuing the following `mb` command, replacing `my-example-bucket` with the label of the bucket you would like to create. s3cmd mb s3://my-example-bucket +{{< note >}} +Bucket labels need to be unique within the same cluster, including buckets on other users' Linode accounts. If the label you enter is already in use, you will have to choose a different label. Additionally, bucket labels have the following rules:
+   •  Cannot be formatted as IP addresses
+   •  Must be between 3 and 63 characters in length
+   •  Can only contain lower-case characters, numbers, periods, and dashes
+   •  Must start with a lowercase letter or number
+   •  Cannot contain underscores (_), end with a dash (-) or period (.), have consecutive periods (.), or use dashes (-) adjacent to periods (.) +{{< /note >}} + To remove a bucket, you can use the `rb` command: s3cmd rb s3://my-example-bucket @@ -206,7 +345,7 @@ If you chose to enable encryption when configuring s3cmd, you can store encrypte Public URL of the object is: http://us-east-1.linodeobjects.com/my-example-bucket/example.txt {{< note >}} -The URL for the object that s3cmd provides is one of two valid ways to access your object. The first, which s3cmd provides, places the name of your bucket after the domain name. You can also access your object by affixing your bucket name as a subdomain: `http://my-example-bucket.us-east-1.linodeobjects.com/example.txt`. The latter URL is generally favored. +The URL for the object that s3cmd provides is one of two valid ways to access your object. The first, which s3cmd provides, places the label of your bucket after the domain name. You can also access your object by affixing your bucket label as a subdomain: `http://my-example-bucket.us-east-1.linodeobjects.com/example.txt`. The latter URL is generally favored. {{< /note >}} 1. To retrieve a file, issue the `get` command: @@ -297,7 +436,16 @@ To create a bucket in Cyberduck: ![Right click or click 'Action', then click 'New Folder'](object-storage-cyberduck-create-bucket.png) -1. Enter your bucket's name and then click **Create**. Bucket names need to be unique within the same cluster, including buckets on other Linode accounts. If the name of your bucket is already in use, you will have to choose a different name. +1. Enter your bucket's label and then click **Create**. + + {{< note >}} +Bucket labels need to be unique within the same cluster, including buckets on other users' Linode accounts. If the label you enter is already in use, you will have to choose a different label. Additionally, bucket labels have the following rules:
+   •  Cannot be formatted as IP addresses
+   •  Must be between 3 and 63 characters in length
+   •  Can only contain lower-case characters, numbers, periods, and dashes
+   •  Must start with a lowercase letter or number
+   •  Cannot contain underscores (_), end with a dash (-) or period (.), have consecutive periods (.), or use dashes (-) adjacent to periods (.) +{{< /note >}} To delete the bucket using Cyberduck, right click on the bucket and select **Delete**. @@ -319,7 +467,7 @@ To delete the bucket using Cyberduck, right click on the bucket and select **Del ![Set the permissions for 'Everyone' to READ.](object-storage-cyberduck-object-permissions2.png) - Your object is now accessible via the internet, at the URL `http://my-example-bucket.us-east-1.linodeobjects.com/example.txt`, where `my-example-bucket` is the name of your bucket, and `example.txt` is the name of your object. + Your object is now accessible via the internet, at the URL `http://my-example-bucket.us-east-1.linodeobjects.com/example.txt`, where `my-example-bucket` is the label of your bucket, and `example.txt` is the name of your object. 1. To download an object, right click on the object and select **Download**, or click **Download As** if you'd like to specify the location of the download. @@ -345,4 +493,4 @@ To create a static site from your bucket: ## Next Steps -There are S3 bindings available for a number of programming languages, including the popular [Boto](https://github.com/boto/boto3) library for Python, that allow you to interact with Object Storage programmatically. +There are S3 bindings available for a number of programming languages, including the popular [Boto](https://github.com/boto/boto3) library for Python, that allow you to interact with Object Storage programmatically. \ No newline at end of file