diff --git a/ci/vale/dictionary.txt b/ci/vale/dictionary.txt index 213a6d1deb2..d4b5d3010eb 100644 --- a/ci/vale/dictionary.txt +++ b/ci/vale/dictionary.txt @@ -102,6 +102,7 @@ bootable bootloader bootup bootups +boto bounceback brackley bram diff --git a/ci/vale/styles/Linode/Terms.yml b/ci/vale/styles/Linode/Terms.yml index 2751dd20533..b0fced56646 100644 --- a/ci/vale/styles/Linode/Terms.yml +++ b/ci/vale/styles/Linode/Terms.yml @@ -35,3 +35,4 @@ swap: uris: URIs Cuda: CUDA gpu: GPU + markdown: Markdown diff --git a/docs/applications/configuration-management/automate-a-static-site-deployment-with-salt/index.md b/docs/applications/configuration-management/automate-a-static-site-deployment-with-salt/index.md index 74f40126841..1aeee6a1783 100644 --- a/docs/applications/configuration-management/automate-a-static-site-deployment-with-salt/index.md +++ b/docs/applications/configuration-management/automate-a-static-site-deployment-with-salt/index.md @@ -34,7 +34,7 @@ Setting up these mechanisms offers an array of benefits: ## Development and Deployment Workflow -The static site generator used in this guide is [Hugo](https://gohugo.io), a fast framework written in Go. Static site generators compile [markdown](https://en.wikipedia.org/wiki/Markdown) or other content files into HTML files. This guide can easily be adapted to other frameworks. +The static site generator used in this guide is [Hugo](https://gohugo.io), a fast framework written in Go. Static site generators compile [Markdown](https://en.wikipedia.org/wiki/Markdown) or other content files into HTML files. This guide can easily be adapted to other frameworks. Two Git repositories will be created: one will track changes to the Hugo site, and the other will track Salt's configuration files. Remote repositories will be created for both on GitHub. @@ -482,7 +482,7 @@ The Salt minion's formula needs to be updated in order to serve the Hugo site. S - Install Git and clone the Hugo site repository from GitHub. -- Install Hugo and build the HTML files from the markdown content. +- Install Hugo and build the HTML files from the Markdown content. - Update the NGINX configuration to serve the built site. @@ -993,7 +993,7 @@ Hook rules were not satisfied.⏎ hugo new post/test-post.md -1. This command creates a new partially filled in markdown document in `content/post/`. Open this file in your editor, remove the `draft: true` line from the [*frontmatter*](https://gohugo.io/content-management/front-matter/), and add some body text: +1. This command creates a new partially filled in Markdown document in `content/post/`. Open this file in your editor, remove the `draft: true` line from the [*frontmatter*](https://gohugo.io/content-management/front-matter/), and add some body text: {{< file "example-hugo-site/content/post/test-post.md" >}} --- diff --git a/docs/applications/project-management/how-to-create-a-private-python-package-repository/index.md b/docs/applications/project-management/how-to-create-a-private-python-package-repository/index.md index 486318c8ff2..2cf8bfa45d7 100644 --- a/docs/applications/project-management/how-to-create-a-private-python-package-repository/index.md +++ b/docs/applications/project-management/how-to-create-a-private-python-package-repository/index.md @@ -85,7 +85,7 @@ def hello_word(): {{< /file >}} -5. The `setup.cfg` file lets PyPI know the README is a markdown file: +5. The `setup.cfg` file lets PyPI know the README is a Markdown file: {{< file "setup.cfg" >}} [metadata] diff --git a/docs/applications/project-management/jupyter-notebook-on-jekyll/index.md b/docs/applications/project-management/jupyter-notebook-on-jekyll/index.md index 1c7ad7cc365..d1fc768992a 100644 --- a/docs/applications/project-management/jupyter-notebook-on-jekyll/index.md +++ b/docs/applications/project-management/jupyter-notebook-on-jekyll/index.md @@ -140,7 +140,7 @@ The steps in this section can be completed from either your local machine or fro This section demonstrates some common features of a Jupyter Notebook that can be rendered as HTML on a Jekyll blog. There are four types of outputs from a Jupyter Notebook cell covered here: MathJex through LaTeX in Markdown, an HTML table, console output, and graphs from a plotting function. The Iris dataset will be used as an example to generate the output in this guide. -1. Open the notebook of interest, or use the code below to create an example notebook. Run all of the relevant cells so that the output you want to display on your Jekyll blog is visible on the page. Navigate to `File > Download As > Markdown (.md)`. The markdown file will save to the default `Downloads` folder of the browser. +1. Open the notebook of interest, or use the code below to create an example notebook. Run all of the relevant cells so that the output you want to display on your Jekyll blog is visible on the page. Navigate to `File > Download As > Markdown (.md)`. The Markdown file will save to the default `Downloads` folder of the browser. ![Jupyter Menu](jupyter_menu.png "Jupyter Menu") @@ -175,11 +175,11 @@ sns.pairplot(x_vars=["petal-length"], y_vars=["petal-width"], data=iris, hue="cl {{< /file >}} -3. Inside the `_posts` folder of the Jekyll project, create a new markdown file called `YYYY-MM-DD-example-post.md`. If the date format is incorrect, the post may not be displayed on the blog: +3. Inside the `_posts` folder of the Jekyll project, create a new Markdown file called `YYYY-MM-DD-example-post.md`. If the date format is incorrect, the post may not be displayed on the blog: touch YYYY-MM-DD-example-post.md -4. The markdown file should begin with three dashes and contain headers which provide information for Jekyll to populate the post with the appropriate page data. The date must be in the format specified. The hours, minutes, seconds, and timezone adjustment are optional: +4. The Markdown file should begin with three dashes and contain headers which provide information for Jekyll to populate the post with the appropriate page data. The date must be in the format specified. The hours, minutes, seconds, and timezone adjustment are optional: {{< file "YYYY-MM-DD-example-post.md" yaml >}} --- @@ -193,7 +193,7 @@ categories: {{< /file >}} -5. Copy the contents of the markdown file exported from Jupyter into the new post. +5. Copy the contents of the Markdown file exported from Jupyter into the new post. To do this from the command line, use: @@ -253,11 +253,11 @@ margin-bottom: 0; } ### Add an Image in Jekyll -Adding an image through markdown requires having the images stored in the project directory. +Adding an image through Markdown requires having the images stored in the project directory. 1. Move all of the images exported from Jupyter into the `/assets/images` folder. -2. Modify the references to images within the markdown to the appropriate path. Wrap the path in two curly braces and double quotes. +2. Modify the references to images within the Markdown to the appropriate path. Wrap the path in two curly braces and double quotes. {{< file "YYYY-MM-DD-example-post.md" >}} ![png]({{ "/assets/images/example_notebook_5_0.png" }}) diff --git a/docs/linode-writers-formatting-guide/index.md b/docs/linode-writers-formatting-guide/index.md index 7978a8ae1e2..807fecc0c0d 100644 --- a/docs/linode-writers-formatting-guide/index.md +++ b/docs/linode-writers-formatting-guide/index.md @@ -28,7 +28,7 @@ We only accept new guides and authors through our guide submission process. To a ## General Layout -Linode Guides & Tutorials are written in [Markdown](https://en.wikipedia.org/wiki/Markdown). Our documentation site uses [Hugo](https://gohugo.io), a static site generator. Hugo-specific markdown formatting notes are given [further below](#markdown-formatting). +Linode Guides & Tutorials are written in [Markdown](https://en.wikipedia.org/wiki/Markdown). Our documentation site uses [Hugo](https://gohugo.io), a static site generator. Hugo-specific Markdown formatting notes are given [further below](#markdown-formatting). ### Header @@ -209,7 +209,7 @@ If you wish to provide links to external sites for the user to review after goin ### Extend Markdown Using Shortguides -Using shortcodes, it is possible to extend a markdown file with another. For common tasks such as basic software installation, consider using the `content` shortcode. This allows our library to maintain consistent and up to date installation instructions for frequently used tools such as Python, MySQL, and Docker. +Using shortcodes, it is possible to extend a Markdown file with another. For common tasks such as basic software installation, consider using the `content` shortcode. This allows our library to maintain consistent and up to date installation instructions for frequently used tools such as Python, MySQL, and Docker. Markdown files intended to be inserted into multiple guides are called shortguides. To create a shortguide, create a directory with the name of your shortguide anywhere within `docs/`, and then create an index.md within the directory for your content (e.g. `example-shortguide-name/index.md`). @@ -270,7 +270,7 @@ Adding `os: ["mac", "linux", "windows"]` to the front matter inserts a jQuery sc The shortcode should contain two parameters: filepath and operating system. -For example, `{{}}` will insert a markdown snippet that will only be visible when the Mac button is in the active state. +For example, `{{}}` will insert a Markdown snippet that will only be visible when the Mac button is in the active state. ### Files and File Excerpts diff --git a/docs/platform/object-storage/_index.md b/docs/platform/object-storage/_index.md new file mode 100644 index 00000000000..e28250f5d9b --- /dev/null +++ b/docs/platform/object-storage/_index.md @@ -0,0 +1,5 @@ +--- +description: 'Content storage and delivery for unstructured data. Great for multimedia, static sites, software delivery, archives, and data backups.' +title: 'Linode Object Storage' +show_in_lists: true +--- diff --git a/docs/platform/object-storage/host-static-site-object-storage/hugo-index.png b/docs/platform/object-storage/host-static-site-object-storage/hugo-index.png new file mode 100644 index 00000000000..7d9f32d3eb7 Binary files /dev/null and b/docs/platform/object-storage/host-static-site-object-storage/hugo-index.png differ diff --git a/docs/platform/object-storage/host-static-site-object-storage/index.md b/docs/platform/object-storage/host-static-site-object-storage/index.md new file mode 100644 index 00000000000..d46b990d14a --- /dev/null +++ b/docs/platform/object-storage/host-static-site-object-storage/index.md @@ -0,0 +1,297 @@ +--- +author: + name: Linode Community + email: docs@linode.com +description: "Host a Static Site using Linode\'s Object Storage." +keywords: ['list','of','keywords','and key phrases'] +license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' +published: 2019-04-09 +modified: 2019-04-09 +modified_by: + name: Linode +title: "Host a Static Site using Linode Object Storage" +contributor: + name: Linode +external_resources: +- '[Hugo Documentation](https://gohugo.io/documentation/)' +- '[s3cmd Options and Commands](https://s3tools.org/usage)' +- '[s3cmd Sync How-To](https://s3tools.org/s3cmd-sync)' +--- +{{< note >}} +Object Storage is currently in a closed early access Beta, and you may not have access to Object Storage through the Cloud Manager or other tools. To gain access to the Early Access Program (EAP), open up a Customer Support ticket noting that you'd like to be included in the program, or e-mail objbeta@linode.com + +Additionally, because Object Storage is in Beta, there may be breaking changes to how you access and manage Object Storage. This guide will be updated to reflect these changes if and when they occur. +{{}} + +## Why Host a Static Site on Object Storage? + +[Static site generators](/docs/websites/static-sites/how-to-choose-static-site-generator/) are a popular solution for creating simple, fast, flexible, and attractive websites that are easy to update. You can contribute new pages and content to a static site in two steps: + +1. First, write the content for your site's new page using [Markdown](https://www.markdownguide.org), an easy-to-learn and light-weight markup language. + +1. Then, tell your static site generator to compile your Markdown (along with other relevant assets, like CSS styling, images, and JavaScript) into static HTML files. + +The second compilation step only needs to happen once for each time that you update your content. This is in contrast with a dynamic website framework like WordPress or Drupal, which will reference a relational database and compile your HTML every time a visitor loads your site. + +### Benefits of Hosting on Object Storage + +Traditionally, these static HTML files would be served by a web server (like [NGINX](/docs/web-servers/nginx/) or [Apache](/docs/web-servers/apache/)) running on a Linode. Using Object Storage to host your static site files means you do not have to worry about maintaining your site's infrastructure. It is no longer necessary to perform typical server maintenance tasks, like software upgrades, web server configuration, and security upkeep. + +Object Storage provides an HTTP REST gateway to objects, which means a unique URL over HTTP is available for every object. Once your static site is built, making it available publicly over the Internet is as easy uploading files to an Object Storage bucket. + +### Object Storage Hosting Workflow + +At a high-level, the required steps to host a static site using Object Storage are: + +1. [Install](#install-the-hugo-static-site-generator) the static site generator of your choice to your local computer. + +1. [Create](#create-a-hugo-site) the desired content and build the site (using your static site generator). + +1. [Upload](#upload-your-static-site-to-linode-s-object-storage) the static files to your Object Storage bucket to make the content publicly available over the Internet. + +This guide will use [Hugo](https://gohugo.io/) to demonstrate how to create a static site and host it on Linode Object Storage. However, there are [many other static site generators](https://www.staticgen.com) to choose from--[Jekyll](https://jekyllrb.com/) and [Gatsby](https://www.gatsbyjs.org/) are popular choices, and the general steps outlined in this guide could be adapted to them. For more information on choosing a static site generator, see the [How to Choose a Static Site Generator](/docs/websites/static-sites/how-to-choose-static-site-generator/) guide. + +## Before You Begin + +1. Read the [How to Use Linode Object Storage](/docs/platform/object-storage/how-to-use-object-storage/) guide to familiarize yourself with Object Storage on Linode. Specifically, be sure that you have: + + - Created your Object Storage access and secret keys. + - Installed and configure the [s3cmd tool](https://s3tools.org/download). + +1. [Install and configure Git](https://linode.com/docs/development/version-control/how-to-install-git-and-clone-a-github-repository/#install-and-configure-git) on your local computer. + +## Install the Hugo Static Site Generator + +Hugo is written in [Go](https://golang.org/) and is known for being extremely fast to compile sites, even very large ones. It is well-supported, [well-documented](https://gohugo.io/documentation/), and has an [active community](https://discourse.gohugo.io/). Some useful Hugo features include [*shortcodes*](https://gohugo.io/content-management/shortcodes/), which are an easy way to include predefined templates inside of your Markdown, and built-in [*LiveReload*](https://gohugo.io/getting-started/usage/#livereload) web server, which allows you to preview your site changes locally as you make them. + +1. Install Hugo on your computer: + + **macOS**: + + - Use the Homebrew package manager for macOS to install Hugo: + + brew install hugo + + **Linux/Ubuntu**: + + - Determine your Linux kernel's architecture: + + uname -r + + Your output will resemble the following: + + {{< output >}} +4.9.0-8-amd64 +{{}} + + - Navigate to [Hugo's GitHub releases page](https://github.com/gohugoio/hugo/releases) and download the appropriate version for your platform. This example command downloads version 0.55, but a newer release may be available: + + wget https://github.com/gohugoio/hugo/releases/download/v0.55.0/hugo_0.55.0_Linux-64bit.deb + + - Install the package using `dpkg`: + + sudo dpkg -i hugo*.deb + +1. Verify that Hugo is installed. You should see output indicating your installed Hugo's version number: + + hugo version + +## Create a Hugo Site + +In this section, you will use the [Hugo CLI](https://gohugo.io/commands/) (command line interface) to create your Hugo site, initialize a Hugo theme, and add content to your site. Hugo's CLI provides several useful commands for common tasks needed to build, configure, and interact with your Hugo site. + +1. Create a new Hugo site on your local computer. This command will create a folder named `example-site` and scaffold [Hugo's directory structure](https://gohugo.io/getting-started/directory-structure/) inside it: + + hugo new site example-site + +1. Move into your Hugo site's root directory: + + cd example-site + + {{< note >}} +All commands in this section of the guide should be issued from your site's root directory. +{{< /note >}} + +1. You will use Git to add a theme to your Hugo site's directory. Initialize your Hugo site's directory as a Git repository: + + git init + +1. Install the [Ananke theme](https://github.com/budparr/gohugo-theme-ananke) as a submodule of your Hugo site's Git repository. [Git submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules) allow one Git repository to be stored as a subdirectory of another Git repository, while still being able to maintain each repository's version control information separately. The Ananke theme's repository will be located in the `~/example-site/themes/ananke` directory of your Hugo site. + + git submodule add https://github.com/budparr/gohugo-theme-ananke.git themes/ananke + + {{< note >}} +Hugo has many [available themes](https://themes.gohugo.io/) that can be installed as a submodule of your Hugo site's directory. +{{< /note >}} + +1. Add the theme to your Hugo site's [configuration file](https://gohugo.io/getting-started/configuration/). The configuration file (`config.toml`) is located at the root of your Hugo site's directory. + + echo 'theme = "ananke"' >> config.toml + +1. Create a new content file for your site. This command will generate a Markdown file with an auto-populated date and title: + + hugo new posts/my-first-post.md + +1. You should see a similar output. Note that the file is located in the `content/posts/` directory of your Hugo site: + + {{< output >}} +/home/username/example-site/content/posts/my-first-post.md created +{{}} + +1. Open the Markdown file in the text editor of your choice to begin modifying its content; you can copy and paste the example snippet into your file, which contains an updated *front matter* section at the top and some example Markdown body text. + + Set your desired value for `title`. Then, set the `draft` state to `false` and add your content below the `---` in Markdown syntax, if desired: + + {{< file "/home/username/example-site/content/posts/my-first-post.md" >}} +--- +title: "My First Post" +date: 2019-04-11T11:25:11-04:00 +draft: false +--- + +# Host a Static Site on Linode Object Storage + +There are many benefits to using a static site generator. Here is a list of a few of them: + +- Run your own website without having to manage a Linode. +- You don't need to worry about running a web server like Apache or NGINX. +- Static website performance is typically very fast. +- Use Git to version control your static website's content. +{{}} + + {{< disclosure-note "About front matter" >}} +[*Front matter*](https://gohugo.io/content-management/front-matter/) is a collection of metadata about your content, and it is embedded at the top of your file within opening and closing `---` delimiters. + +Front matter is a powerful Hugo feature that provides a mechanism for passing data that is attached to a specific piece of content to Hugo's rendering engine. Hugo accepts front matter in TOML, YAML, and JSON formats. In the example snippet, there is YAML front matter for the title, date, and draft state of the Markdown file. These variables will be referenced and displayed by your Hugo theme. +{{< /disclosure-note >}} + +1. Once you have added your content, you can preview your changes by building and serving the site using Hugo's built-in webserver: + + hugo server + +1. You will see a similar output: + + {{< output >}} +                   | EN ++------------------+----+ + Pages              | 11 + Paginator pages    | 0 + Non-page files     | 0 + Static files       | 3 + Processed images   | 0 + Aliases            | 1 + Sitemaps           | 1 + Cleaned            | 0 + +Total in 7 ms +Watching for changes in /home/username/example-site/{content,data,layouts,static,themes} +Watching for config changes in /home/username/example-site/config.toml +Serving pages from memory +Running in Fast Render Mode. For full rebuilds on change: hugo server --disableFastRender +Web Server is available at http://localhost:1313/ (bind address 127.0.0.1) +Press Ctrl+C to stop +{{}} + +1. The output will provide a URL to preview your site. Copy and paste the URL into a browser to access the site. In the above example Hugo's web server URL is `http://localhost:1313/`. + +1. When you are happy with your site's content you can *build* your site: + + hugo -v + + Hugo will generate your site's static HTML files and store them in a `public` directory that it will create inside your project. The static files that are generated by Hugo are the files that you will upload to your Object Storage bucket to make your site accessible via the Internet. + +1. View the contents of your site's `public` directory: + + ls public + + Your output should resemble the following example. When you built the site, the Markdown file you created and edited in steps 6 and 7 was used to generate its corresponding static HTML file in the `public/posts/my-first-post/index.html` directory. + + {{< output >}} + 404.html categories dist images index.html index.xml posts sitemap.xml tags + {{}} + + {{< disclosure-note "Track your Static Site Files with Git">}} +It's not necessary to version control your site files in order to host them on Object Storage, but we still recommended that you do so: + +1. Display the state of your current working directory (root of your Hugo site): + + git status + +2. Stage all your files to be committed: + + git add -A + +3. Commit all your changes and add a meaningful commit message: + + git commit -m 'Add my first post.' + +Once you have used Git to track your local Hugo site files, you can easily push them to a remote Git repository, like [GitHub](https://github.com/) or [GitLab](https://about.gitlab.com/). Storing your static site files on a remote Git repository opens up many possibilities for collaboration and automating your static site's deployment to Linode Object Storage. To learn more about Git, see the [Getting Started with Git](/docs/development/version-control/how-to-configure-git/) guide. +{{}} + +## Upload your Static Site to Linode Object Storage + +Before proceeding with this section ensure that you have already created your Object Storage access and secret keys and have installed the s3cmd tool. + +1. Create a new Object Storage bucket; prepend `s3://` to the beginning of the bucket's name: + + s3cmd mb s3://my-bucket + + {{< note >}} +Buckets names must be unique within the Object Storage cluster. You might find the bucket name `my-bucket` is already in use by another Linode customer, in which case you will need to choose a new bucket name. +{{}} + +1. Initialize your Object Storage bucket as a website. You must tell your bucket which files to serve as the index page and the error page for your static site. This is done with the `--ws-index` and `--ws-error` options: + + s3cmd ws-create --ws-index=index.html --ws-error=404.html s3://my-bucket + + In our Hugo example, the site's index file is `index.html` and the error file is `404.html`. Whenever a user visits your static site's URL, the Object Storage service will serve the `index.html` page. If a site visitor tries to access an invalid path, they will be presented with the `404.html` page. + +1. The command will return the following message: + + {{< output >}} + Bucket 's3://my-bucket/': website configuration created. + {{}} + +1. Display information about your Object Storage's website configuration to obtain your site's URL: + + s3cmd ws-info s3://my-bucket + +1. You should see a similar output. Be sure to take note of your Object Storage bucket's URL: + + {{< output >}} + Bucket s3://my-bucket/: Website configuration +Website endpoint: http://website-us-east-1.linodeobjects.com/ +Index document: index.html +Error document: 404.html + {{}} + + {{< note >}} +The Linode Object Storage early access Beta provides SSL enabled by default. This means you can access your Object Storage bucket using `https`, as well. +{{}} + +1. Use s3cmd's `sync` command to upload the contents of your static site's `public` directory to your Object Storage bucket. This step will make your site available publicly on the Internet. Ensure you are in your site's root directory on your computer (e.g. `/home/username/example-site`): + + s3cmd --no-mime-magic --acl-public --delete-removed --delete-after sync public/ s3://my-bucket + + | Option                          | Description | + |--------|-------------| + | `no-mime-magic` | Tells Object Storage not to use file signatures when guessing the object's MIME-type. | + | `acl-public` | Sets the access level control of the objects to public. | + | `delete-removed` | Deletes any destination objects with no corresponding source file. | + | `delete-after` | Deletes destination files that are no longer found at the source after all files are uploaded to the bucket. | + +1. Use a browser to navigate to your Object Storage bucket's URL to view your Hugo site: + + ![Hugo Index Page](hugo-index.png) + + {{< note >}} +It may take a minute or two after your s3cmd sync completes for the page to appear at your bucket's website URL. +{{< /note >}} + +1. If needed, you can continue to update your static site locally and upload any changes using s3cmd's `sync` command from step 3 of this section. + +## (Optional) Next Steps + +After uploading your static site to Linode Object Storage, you may want to use a custom domain for your site. To do this, you can add a CNAME entry to your domain's DNS records that aliases it to your Object Storage bucket's website URL. To learn about managing DNS records on Linode, see the [DNS Manager](/docs/platform/manager/dns-manager/) and [DNS Records: An Introduction](/docs/networking/dns/dns-records-an-introduction/) guides. + +As noted before, it's possible to trigger automatic deployments to the Object Storage service when you push new content updates to GitHub or GitLab. This is done by leveraging a CI/CD (continuous integration/continuous delivery) tool like [Travis CI](https://travis-ci.org). Essentially, you would build your Hugo site within the Travis environment and then run the `s3cmd sync` command from it to your bucket. \ No newline at end of file diff --git a/docs/platform/object-storage/how-to-use-object-storage/index.md b/docs/platform/object-storage/how-to-use-object-storage/index.md new file mode 100644 index 00000000000..b0d78f9567c --- /dev/null +++ b/docs/platform/object-storage/how-to-use-object-storage/index.md @@ -0,0 +1,348 @@ +--- +author: + name: Linode Community + email: docs@linode.com +description: "Get started using Linode's Object Storage." +keywords: ['object','storage','bucket'] +license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' +published: 2019-04-12 +modified: 2019-04-12 +modified_by: + name: Linode +title: "How to Use Linode Object Storage" +contributor: + name: Linode +external_resources: +- '[S3cmd usage and command reference](https://s3tools.org/usage)' +--- + +{{< note >}} +Object Storage is currently in a closed early access Beta, and you may not have access to Object Storage through the Cloud Manager or other tools. To gain access to the Early Access Program (EAP), open up a Customer Support ticket noting that you'd like to be included in the program, or e-mail objbeta@linode.com + +Additionally, because Object Storage is in Beta, there may be breaking changes to how you access and manage Object Storage. This guide will be updated to reflect these changes if and when they occur. +{{}} + +Linode's Object Storage is a globally-available, S3- and Swift-compatible method for storing and accessing data. Object Storage differs from traditional hierarchical data storage (as in a Linode's disk) and [Block Storage Volumes](https://www.linode.com/docs/platform/block-storage/). Under Object Storage, files (also called *objects*) are stored in flat data structures (referred to as *buckets*) alongside their own rich metadata. + +Additionally, **Object Storage does not require the use of a Linode.** Instead, Object Storage gives each object a unique URL with which you can access your data. An object can be publicly accessible, or you can set it to be private and only visible to you. This makes Object Storage great for sharing and storing unstructured data like images, documents, archives, streaming media assets, and file backups, and the amount of data you store can range from small collections of files up to massive libraries of information. Lastly, Linode Object Storage has the built-in ability to [host a static site](/docs/platform/object-storage/host-static-site-object-storage/). + +Below you will find instructions on how to connect to Object Storage, and how to upload and access objects: + +1. First, you'll need to create a [*key pair*](#object-storage-key-pair) to access the service. + +1. Then, you'll use choose from a variety of available [first-party and third-party tools](#object-storage-tools) to access and use the service. + +## Object Storage Key Pair + +The first step towards using Object Storage is to create a pair of keys for the service. This pair is composed of an *access key* and a *secret key*: + +- The access key allows you to access any objects that you set to have private read permissions. + + {{< note >}} +To use your access key when viewing a private object, you first need to generate a *signed* URL for the object. The signed URL is much like the standard URL for your object, but some extra URL parameters are appended to it, including the access key. Instructions for generating a signed URL can be found for each of the tools outlined in this guide. +{{< /note >}} + +- Your secret key is used together with your access key to authenticate the various Object Storage tools with your Linode account. You should not share the secret key. + + {{< note >}} +Each Object Storage key pair on your Linode account has complete access to all of the buckets on your account. +{{< /note >}} + +### Generate a Key Pair + +1. Log in to the [Linode Cloud Manager](https://cloud.linode.com). + + {{< note >}} +Object Storage is not available in the Linode Classic Manager. +{{}} + +1. Click on the **Object Storage** link in the sidebar, click the **Access Keys** tab, and then click the **Create an Access Key** link. + + ![Click on the 'Access Keys' tab.](object-storage-access-keys-tab.png) + +1. The **Create an Access Key** menu will appear. + + ![The 'Create an Access Key' menu.](object-storage-create-key.png) + +1. Enter a label for the key pair. This label will be how you reference your key pair in the Linode Cloud Manager. Then, click **Submit**. + +1. A window will appear that contains your access key and your secret key. Write these down somewhere secure. The access key will be visible in the Linode Cloud Manager, but **you will not be able to retrieve your secret key again once you close the window.** + + ![Your access key and secret key.](object-storage-acces-keys.png) + + You now have the credentials needed to connect to Linode Object Storage. + +## Object Storage Tools + +There are a number of tools that are available to help manage Linode Object Storage. This guide explains how to install and use the following options: + +- The [Linode Cloud Manager](#cloud-manager) can be used to create buckets (you are currently not able to upload objects to a bucket from the Cloud Manager). + +- [s3cmd](#s3cmd) is a powerful command line utility that can be used with any S3-compatible object storage service, including Linode's. s3cmd can be used to create and remove buckets, add and remove objects, convert a bucket into a static site from the command line, plus other functions like syncing entire directories up to a bucket. + +- [Cyberduck](#cyberduck) is a graphical utility available for Windows and macOS and is a great option if you prefer a GUI tool. + +## Cloud Manager + +### Create a Bucket + +The Cloud Manager provides a web interface for creating buckets. To create a bucket: + +1. If you have not already, log in the [Linode Cloud Manager](https://cloud.linode.com). + +1. Click on the **Object Storage** link in the sidebar, and then click on **Add a Bucket**. + + ![The Object Storage menu.](object-storage-add-a-bucket.png) + +1. The **Create a Bucket** menu will appear. + + ![The Create a Bucket menu.](object-storage-create-a-bucket.png) + +1. Add a label for your bucket. A bucket's label needs to be unique within the cluster that it lives in, and this includes buckets of the same name on different Linode accounts. If the label you enter is already in use, you will have to choose a different label. + +1. Choose a cluster location for the bucket to reside in. + +1. Click **Submit**. You are now ready to upload objects to your bucket using one of the other tools outlined in this guide. + +## s3cmd + +s3cmd is a command line utility that you can use for any S3-compatible Object Storage. + +### Install and Configure s3cmd + +1. s3cmd can be downloaded using `apt` on Debian and Ubuntu, and [Homebrew](https://brew.sh/) on macOS. To download s3cmd using Homebrew, run the following command: + + brew install s3cmd + + {{< note >}} +On macOS, s3cmd might fail to install if you do not have XCode command line tools installed. If that is the case, run the following command: + + xcode-select --install + +You will be prompted to agree to the terms and conditions. +{{}} + + To install s3cmd on Debian or Ubuntu, run the following command: + + apt install s3cmd + +1. Once s3cmd has been installed, you will need to configure it: + + s3cmd --configure + + You will be presented with a number of questions. To accept the default answer that appears within the brackets, press enter. Here is an example of the answers you will need to provide: + + Access Key: 4TQ5CJGZS92LLEQHLXB3 + Secret Key: enteryoursecretkeyhere + Default Region: US + S3 Endpoint: us-east-1.linodeobjects.com + DNS-style bucket+hostname:port template for accessing a bucket: us-east-1.linodeobjects.com + Encryption password: YOUR_GPG_KEY + Path to GPG program: /usr/local/bin/gpg + Use HTTPS protocol: False + HTTP Proxy server name: + HTTP Proxy server port: 0 + + {{< note >}} +It is not necessary to supply a GPG key when configuring s3cmd, though it will allow you to store and retrieve encrypted files. If you do not wish to configure GPG encryption, you can leave the `Encryption password` and `Path to GPG program` fields blank. +{{}} + +1. When you are done, enter `Y` to save your configuration. + + {{< note >}} +s3cmd offers a number of additional configuration options that are not presented as prompts by the `s3cmd --configure` command. One of those options is `website_endpoint`, which instructs s3cmd on how to construct an appropriate URL for a bucket that is hosting a static site, similar to the `S3 Endpoint` in the above configuration. This step is optional, but will ensure that any commands that contain your static site's URL will output the right text. To edit this configuration file, open the `~/s3.cfg` file on your local computer: + + nano ~/.s3cfg + +Scroll down until you find the `website_endpoint`, then add the following value: + + http://%(bucket)s.website-us-east-1.linodeobjects.com/ + + +{{}} + +You are now ready to use s3cmd to create a bucket in Object Storage. + +### Create a Bucket with s3cmd + +You can create a bucket with s3cmd issuing the following `mb` command, replacing `my-example-bucket` with the name of the bucket you would like to create. Bucket names need to be unique within the same cluster, including buckets on other Linode accounts. If you choose a name for your bucket that someone else has already created, you will have to choose a different name: + + s3cmd mb s3://my-example-bucket + +To remove a bucket, you can use the `rb` command: + + s3cmd rb s3://my-example-bucket + +{{< caution >}} +To delete a bucket that has files in it, include the `--recursive` (or `-r`) option *and* the `--force` (or `-f`) option. Use caution when using this command: + + s3cmd rb -r -f s3://my-example-bucket/ +{{< /caution >}} + +### Upload, Download, and Delete an Object with s3cmd + +1. As an example object, create a text file and fill it with some example text. + + echo 'Hello World!' > example.txt + +1. Now, transfer the text file object to your bucket using s3cmd's `put` command, replacing `my-example-bucket` with the label of the bucket you gave in the last section: + + s3cmd put example.txt s3://my-example-bucket -P + + {{< note >}} +The `-P` flag at the end of the command instructs s3cmd to make the object public. To make the object private, which means you will only be able to access it from a tool such as s3cmd, simply leave the '-P' flag out of the command. +{{}} + + {{< note >}} +If you chose to enable encryption when configuring s3cmd, you can store encrypted objects by supplying the `-e` flag: + + s3cmd put -e encrypted_example.txt s3://my-example-bucket +{{}} + +1. The object will be uploaded to your bucket, and s3cmd will provide a public URL for the object: + + upload: 'example.txt' -> 's3://my-example-bucket/example.txt' [1 of 1] + 13 of 13 100% in 0s 485.49 B/s done + Public URL of the object is: http://us-east-1.linodeobjects.com/my-example-bucket/example.txt + + {{< note >}} +The URL for the object that s3cmd provides is one of two valid ways to access your object. The first, which s3cmd provides, places the name of your bucket after the domain name. You can also access your object by affixing your bucket name as a subdomain: `http://my-example-bucket.us-east-1.linodeobjects.com/example.txt`. The latter URL is generally favored. +{{< /note >}} + +1. To retrieve a file, issue the `get` command: + + s3cmd get s3://my-example-bucket/example.txt + + If the file you are attempting to retrieve is encrypted, you can retrieve it using the `-e` flag: + + s3cmd get -e s3://my-example-bucket/encrypted_example.txt + +1. To delete a file, you can issue the `rm` command: + + s3cmd rm s3://my-example-bucket/example.txt + + {{< caution >}} +To delete all files in a bucket, include the `--recursive` (or `-r`) option *and* the `--force` (or `-f`) option. Use caution when using this command: + + s3cmd rm -r -f s3://my-example-bucket/ +{{< /caution >}} + +1. To list all available buckets, issue the `ls` command: + + s3cmd ls + +1. To list all objects in a bucket, issue the `ls` command and supply a bucket: + + s3cmd ls s3://my-example-bucket + +### Create a Static Site with s3cmd + +You can also create a static website using Object Storage and s3cmd: + +1. To create a website from a bucket, issue the `ws-create` command: + + s3cmd ws-create --ws-index=index.html --ws-error=404.html s3://my-example-bucket + + The `--ws-index` and `--ws-error` flags specify which objects the bucket should use to serve the static site's index page and error page, respectively. + +1. You will need to separately upload the `index.html` and `404.html` files (or however you have named the index and error pages) to your bucket: + + echo 'Index page' > index.html + echo 'Error page' > 404.html + s3cmd put index.html 404.html s3://my-example-bucket + +1. Your static site is accessed from a different URL than the generic URL for your Object Storage bucket. Static sites are available at the `website-us-east-1` subdomain. Using `my-example-bucket` as an example, you would navigate to `http://my-example-bucket.website-us-east-1.linodeobjects.com`. + +For more information on hosting a static website with Object Storage, read our [Host a Static Site using Linode Object Storage](/docs/platform/object-storage/host-static-site-object-storage/) guide. + +### Other s3cmd Commands + +To upload an entire directory of files, you can use the the `sync` command, which will automatically sync all new or changed files. Navigate to the directory you would like to sync, then enter the following: + + s3cmd sync . s3://my-example-bucket -P + +This can be useful for uploading the contents of a static site to your bucket. + +{{< note >}} +The period in the above command instructs s3cmd to upload the current directory. If you do not want to first navigate to the directory you wish to upload, you can supply a path to the directory instead of the period. +{{}} + +## Cyberduck + +Cyberduck is a desktop application that facilitates file transfer over FTP, SFTP, and a number of other protocols, including S3. + +### Install and Configure Cyberduck + +1. Download Cyberduck by [visiting their website](https://cyberduck.io/). + +1. Once you have Cyberduck installed, open the program and click on **Open Connection**. + +1. At the top of the Open Connection dialog, select **Amazon S3** from the dropdown menu. + + ![Open Cyberduck and click on 'Open Connection' to open the connection menu.](object-storage-cyberduck-open-connection.png) + +1. For the Server address, enter `us-east-1.linodeobjects.com`. + +1. Enter your access key in the **Access Key ID** field, and your secret key in the **Secret Access Key** field. + +1. Click **Connect**. + +You are now ready to create a bucket in Object Storage. + +### Create a Bucket with Cyberduck + +To create a bucket in Cyberduck: + +1. Right click within the window frame, or click **Action**, then click **New Folder**: + + ![Right click or click 'Action', then click 'New Folder'](object-storage-cyberduck-create-bucket.png) + +1. Enter your bucket's name and then click **Create**. Bucket names need to be unique within the same cluster, including buckets on other Linode accounts. If the name of your bucket is already in use, you will have to choose a different name. + +To delete the bucket using Cyberduck, right click on the bucket and select **Delete**. + +### Upload, Download, and Delete an Object with Cyberduck + +1. To upload objects with Cyberduck, you can simply drag and drop the object, or directory of objects, to the bucket you'd like to upload them to, and Cyberduck will do the rest. Alternatively, you can click on the **Action** button and select **Upload** from the menu: + + ![Click on the 'Action' button to use the file upload dialog.](object-storage-cyberduck-upload-menu.png) + +1. To make your objects publicly accessible, meaning that you can access them from the object's URL, you need to set the proper READ permissions. Right click on the object and select **Info**. + +1. Click on the **Permissions** tab. + +1. Click the gear icon at the bottom of the window and select **Everyone**. + + ![Open the file permissions prompt by right clicking on the file and selecting.](object-storage-cyberduck-object-permissions.png) + +1. A new entry for *Everyone* will appear in the Access Control List. Next to *Everyone*, under *Permissions* column heading, select **READ** from the drop down menu. + + ![Set the permissions for 'Everyone' to READ.](object-storage-cyberduck-object-permissions2.png) + + Your object is now accessible via the internet, at the URL `http://my-example-bucket.us-east-1.linodeobjects.com/example.txt`, where `my-example-bucket` is the name of your bucket, and `example.txt` is the name of your object. + +1. To download an object, right click on the object and select **Download**, or click **Download As** if you'd like to specify the location of the download. + +1. To delete an object, right click the object name and select **Delete**. + +### Create a Static Site with Cyberduck + +To create a static site from your bucket: + +1. Select a bucket, then right click on that bucket or select the **Action** button at the top of the menu. + +1. Click on **Info**, and then select the **Distribution (CDN)** tab. + +1. Check the box that reads **Enable Website Configuration (HTTP) Distribution**: + + ![Check the box labeled 'Enable Website Configuration (HTTP) Distribution'](object-storage-cyberduck-enable-static-site.png) + +1. You will need to separately upload the `index.html` and `404.html` files (or however you have named the index and error pages) to your bucket. Follow the instructions from the [Upload, Download, and Delete an Object with Cyberduck](#upload-download-and-delete-an-object-with-cyberduck) section to upload these files. + +1. Your static site is accessed from a different URL than the generic URL for your Object Storage bucket. Static sites are available at the `website-us-east-1` subdomain. Using `my-example-bucket` as an example, you would navigate to `http://my-example-bucket.website-us-east-1.linodeobjects.com`. + + For more information on hosting a static website with Object Storage, read our [Host a Static Site using Linode Object Storage](/docs/platform/object-storage/host-static-site-object-storage/) guide. + +## Next Steps + +There are S3 bindings available for a number of programming languages, including the popular [Boto](https://github.com/boto/boto3) library for Python, that allow you to interact with Object Storage programmatically. diff --git a/docs/platform/object-storage/how-to-use-object-storage/object-storage-acces-keys.png b/docs/platform/object-storage/how-to-use-object-storage/object-storage-acces-keys.png new file mode 100644 index 00000000000..8131d6cfd28 Binary files /dev/null and b/docs/platform/object-storage/how-to-use-object-storage/object-storage-acces-keys.png differ diff --git a/docs/platform/object-storage/how-to-use-object-storage/object-storage-access-keys-tab.png b/docs/platform/object-storage/how-to-use-object-storage/object-storage-access-keys-tab.png new file mode 100644 index 00000000000..6c77096b2da Binary files /dev/null and b/docs/platform/object-storage/how-to-use-object-storage/object-storage-access-keys-tab.png differ diff --git a/docs/platform/object-storage/how-to-use-object-storage/object-storage-add-a-bucket.png b/docs/platform/object-storage/how-to-use-object-storage/object-storage-add-a-bucket.png new file mode 100644 index 00000000000..577a0484e8c Binary files /dev/null and b/docs/platform/object-storage/how-to-use-object-storage/object-storage-add-a-bucket.png differ diff --git a/docs/platform/object-storage/how-to-use-object-storage/object-storage-api-keys-tab.png b/docs/platform/object-storage/how-to-use-object-storage/object-storage-api-keys-tab.png new file mode 100644 index 00000000000..fc62e4b5952 Binary files /dev/null and b/docs/platform/object-storage/how-to-use-object-storage/object-storage-api-keys-tab.png differ diff --git a/docs/platform/object-storage/how-to-use-object-storage/object-storage-create-a-bucket.png b/docs/platform/object-storage/how-to-use-object-storage/object-storage-create-a-bucket.png new file mode 100644 index 00000000000..c92542a9bee Binary files /dev/null and b/docs/platform/object-storage/how-to-use-object-storage/object-storage-create-a-bucket.png differ diff --git a/docs/platform/object-storage/how-to-use-object-storage/object-storage-create-key.png b/docs/platform/object-storage/how-to-use-object-storage/object-storage-create-key.png new file mode 100644 index 00000000000..e333f488818 Binary files /dev/null and b/docs/platform/object-storage/how-to-use-object-storage/object-storage-create-key.png differ diff --git a/docs/platform/object-storage/how-to-use-object-storage/object-storage-cyberduck-create-bucket.png b/docs/platform/object-storage/how-to-use-object-storage/object-storage-cyberduck-create-bucket.png new file mode 100644 index 00000000000..2dcdeaff819 Binary files /dev/null and b/docs/platform/object-storage/how-to-use-object-storage/object-storage-cyberduck-create-bucket.png differ diff --git a/docs/platform/object-storage/how-to-use-object-storage/object-storage-cyberduck-enable-static-site.png b/docs/platform/object-storage/how-to-use-object-storage/object-storage-cyberduck-enable-static-site.png new file mode 100644 index 00000000000..1aaf0d3387d Binary files /dev/null and b/docs/platform/object-storage/how-to-use-object-storage/object-storage-cyberduck-enable-static-site.png differ diff --git a/docs/platform/object-storage/how-to-use-object-storage/object-storage-cyberduck-object-permissions.png b/docs/platform/object-storage/how-to-use-object-storage/object-storage-cyberduck-object-permissions.png new file mode 100644 index 00000000000..18224655d34 Binary files /dev/null and b/docs/platform/object-storage/how-to-use-object-storage/object-storage-cyberduck-object-permissions.png differ diff --git a/docs/platform/object-storage/how-to-use-object-storage/object-storage-cyberduck-object-permissions2.png b/docs/platform/object-storage/how-to-use-object-storage/object-storage-cyberduck-object-permissions2.png new file mode 100644 index 00000000000..588bb8b2881 Binary files /dev/null and b/docs/platform/object-storage/how-to-use-object-storage/object-storage-cyberduck-object-permissions2.png differ diff --git a/docs/platform/object-storage/how-to-use-object-storage/object-storage-cyberduck-open-connection.png b/docs/platform/object-storage/how-to-use-object-storage/object-storage-cyberduck-open-connection.png new file mode 100644 index 00000000000..b39a0b1cd38 Binary files /dev/null and b/docs/platform/object-storage/how-to-use-object-storage/object-storage-cyberduck-open-connection.png differ diff --git a/docs/platform/object-storage/how-to-use-object-storage/object-storage-cyberduck-upload-menu.png b/docs/platform/object-storage/how-to-use-object-storage/object-storage-cyberduck-upload-menu.png new file mode 100644 index 00000000000..4f202b6a8ed Binary files /dev/null and b/docs/platform/object-storage/how-to-use-object-storage/object-storage-cyberduck-upload-menu.png differ