Skip to content
This repository was archived by the owner on Jul 22, 2024. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
462c501
initial add of lab5 cos
remkohdev-ibm Nov 28, 2020
a3b6125
add updates to cos
remkohdev-ibm Dec 2, 2020
4810dee
add updates to cos
remkohdev-ibm Dec 2, 2020
1e3991f
add updates to cos
remkohdev-ibm Dec 2, 2020
e444257
add updates to cos
remkohdev-ibm Dec 2, 2020
aeccd0c
add updates to cos
remkohdev-ibm Dec 2, 2020
58db719
add updates to cos
remkohdev-ibm Dec 2, 2020
9182064
add updates to cos
remkohdev-ibm Dec 2, 2020
641fc8c
add updates to cos
remkohdev-ibm Dec 2, 2020
e8a70ca
add updates to cos
remkohdev-ibm Dec 2, 2020
0059216
add updates to cos
remkohdev-ibm Dec 2, 2020
c0a74fa
add updates to cos
remkohdev-ibm Dec 2, 2020
c5baf99
add updates to cos
remkohdev-ibm Dec 2, 2020
4958321
add updates to cos
remkohdev-ibm Dec 2, 2020
bc13bb4
add updates to cos
remkohdev-ibm Dec 2, 2020
b1b5c2a
add updates to cos
remkohdev-ibm Dec 2, 2020
0ed8370
add updates to cos
remkohdev-ibm Dec 2, 2020
5f9d207
add updates to cos
remkohdev-ibm Dec 2, 2020
c08427d
add updates to cos
remkohdev-ibm Dec 2, 2020
44a66ef
add updates to cos
remkohdev-ibm Dec 2, 2020
320c580
add updates to cos
remkohdev-ibm Dec 2, 2020
4e8c35d
add updates to cos
remkohdev-ibm Dec 2, 2020
f276429
add updates to cos
remkohdev-ibm Dec 2, 2020
036cf05
add updates to cos
remkohdev-ibm Dec 2, 2020
0d7b76e
add updates to cos
remkohdev-ibm Dec 2, 2020
6432cea
add updates to cos
remkohdev-ibm Dec 2, 2020
1e64a34
add updates to cos
remkohdev-ibm Dec 2, 2020
16e998f
add updates to cos
remkohdev-ibm Dec 2, 2020
9693189
add updates to cos
remkohdev-ibm Dec 2, 2020
d411d5d
add updates to cos
remkohdev-ibm Dec 2, 2020
75107bf
add updates to cos
remkohdev-ibm Dec 2, 2020
cf19b10
add updates to cos
remkohdev-ibm Dec 2, 2020
8f61fa4
add updates to cos
remkohdev-ibm Dec 2, 2020
b8baa9d
add updates to cos
remkohdev-ibm Dec 2, 2020
9e4198d
add updates to cos
remkohdev-ibm Dec 2, 2020
f635e96
add flexvolume background
remkohdev-ibm Dec 13, 2020
5af2a1a
updates to fuse
remkohdev-ibm Jan 5, 2021
4d693ef
fix lint issues
remkohdev-ibm Jan 5, 2021
cb63b1f
fix markdownlint issues
remkohdev-ibm Jan 6, 2021
43f55c2
fix markdownlint issues
remkohdev-ibm Jan 6, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added workshop/.gitbook/cognitiveclass/newTerminal.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added workshop/.gitbook/cognitiveclass/waiting.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added workshop/.gitbook/ibmcos-plugin-arch.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added workshop/.gitbook/register/create-account.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added workshop/.gitbook/register/create.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added workshop/.gitbook/register/dashboard.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added workshop/.gitbook/register/login.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added workshop/.gitbook/register/verify-email.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
74 changes: 66 additions & 8 deletions workshop/Lab5/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,69 @@
# Lab 1. Title
Let's investigate how Helm can help us focus on other things by letting a chart do the work for us. We'll first deploy an application to a Kubernetes cluster by using `kubectl` and then show how we can offload the work to a chart by deploying the same app with Helm.
# Object Storage with Kubernetes

The application is the [Guestbook App](https://github.com/IBM/guestbook), which is a sample multi-tier web application.
## About Object Storage

## Scenario 1: Deploy the application using `kubectl`
In `object storage` or `Object-based Storage Devices (OSD)`, data is organized into flexible-sized objects that abstract the physical blocks of data, in contrast to block-oriented interfaces that read and write fixed sized blocks of data, like `file storage` or `block storage`. Objects include data, a globally unique identifier and metadata for indexing and management.

```bash
git clone https://github.com/IBM/workshop-template
cd workshop-template
```
Object storage also provides programmatic interfaces (mostly RESTful APIs) to manipulate data for CRUD, versioning, replication, life-cycle management and data transfer. Applications don't need to go through an operating system's storage drivers to manipulate data, they simply send `get`, `put`, or `delete` requests to the storage system.

Object storage has the following benefits:

1. durable, built-in data integrity (e.g. in case of disk failure),
1. available, highly available via REST APIs at the manager layer,
1. scalable, in order of terabytes (TBs), petabytes (PBs), and greater, unavailable in file or block storage,
1. flexible, access from anywhere via REST APIs,
1. secure, encrypt at-rest and in-transit.

### Usage

Object storage is often used for handling large amounts of unstructured data, including email, video, photos, web pages, audio, sensor data and other types of media and web content, both textual and non-textual.

Use cases are:

1. Disaster recovery (DR) and backup (BC),
1. AI and analytics, as a data lake in combination with Spark and Tensorflow,
1. cloud native, startups combining cost-effectiveness of cloud native with flexibility of object storage,
1. data archive, e.g. media files.

### Standards

The International Committee for Information Technology Standards (INCITS) is an American standards organization for computer and communications standards. Its [T10 committee](http://www.t10.org) is devoted to Small Computer Systems Interface (SCSI) technology and this T10 committee has published 2 standards for Object-Based Storage Devices (OSD):

* Object-Based Storage Device Commands (OSD), INCITS 400-2004 (R2013), InterNational Committee for Information Technology Standards. Retrieved 8 November 2013.
* Object-Based Storage Devices - 2 (OSD-2), INCITS 458-2011 (R2016), InterNational Committee for Information Technology Standards. 15 March 2011. Retrieved 8 November 2013.

## About IBM Cloud Object Storage

The IBM Cloud Object Storage (COS) offers a few features that help secure your data. IBM Cloud Object Storage (COS) actively participates in several industry compliance programs and provides the following compliance, certifications, attestations, or reports as measure of proof:

* ISO 27001,
* PCI-DSS for Payment Card Industry (PCI) USA,
* HIPAA for Healthcare USA, (including administrative, physical, and technical safeguards required of Business Associates in 45 CFR Part 160 and Subparts A and C of Part 164),
* ISO 22301 Business Continuity Management,
* ISO 27017,
* ISO 27018,
* ISO 31000 Risk Management Principles,
* ISO 9001 Quality Management System,
* SOC1 Type 2 (SSAE 16), (System and Organization Controls 1),
* SOC2 Type 2 (SSAE 16), (System and Organization Controls 2),
* CSA STAR Level 1 (Self-Assessment),
* General Data Protection Regulation (GDPR) ready,
* Privacy shield certified.

At a high level, information on `IBM Cloud Object Storage (COS)` is encrypted, then dispersed across multiple geographic locations, and accessed over popular protocols like HTTP with a RESTful API.

`SecureSlice` distributes the data in slices across geo locations so that no full copy of data exists on any individual storage node, and automatically encrypts each segment of data before it is erasure coded and dispersed.

The content can only be re-assembled through IBM Cloud’s `Accesser` technology at the client’s primary data center, where the data was originally received, and decrypted again by `SecureSlice`.

`Data-in-place` or `data-at-rest` security is ensured when you persist database contents in IBM Cloud Object Storage.

You also have a choice to use integration capabilities with IBM Cloud Key Management Services like `IBM Key Protect` (using FIPS 140-2 Level 3 certified hardware security modules (HSMs)) and `Hyper Protect Crypto Services` (built on FIPS 140-2 Level 4-certified hardware) for enhanced security features and compliance.

## About IBM Cloud Object Storage Plugin

If you want to learn more about the IBM Cloud Object Storage plugin and s3fs-fuse, read [IBM Cloud Object Storage plugin](cos-with-s3fs/IBMC-S3FS.md)

## Next

[Lab 5: Add Object Storage to a Persistent Database](cos-with-s3fs/README.md)
119 changes: 119 additions & 0 deletions workshop/Lab5/cos-with-s3fs/CLUSTER.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# Configure your Cluster

You now have an Object Storage instance with a bucket, and have found the corresponding private endpoint for your Object Storage. Next, we can configure a Kubernetes cluster:

1. Create a New Namespace in your Cluster,
1. Create a Secret to Access the Object Storage,

## Create a New Namespace in your Cluster

1. Previously, you logged in to your personal account to create a free instance of IBM Cloud Object Storage (COS). If the cluster exists in a different account, make sure to to switch accounts and log in to the IBM Cloud where your cluster exists.

```console
ibmcloud login -u $IBM_ID
```

**Note:** if you use a single-sign-on provider, use the `-sso` flag.

1. If you needed to switch accounts, you will have logged in again, and when prompted to `Select an account`, this time, choose the account with your cluster. In the example below, I have to choose account number **2** from the list, `2. IBM Client Developer Advocacy (e65910fa61) <-> 1234567`,

```console
ibmcloud login -u b.newell2@remkoh.dev
API endpoint: https://cloud.ibm.com
Region: us-south

Password>
Authenticating...
OK

Select an account:
1. B Newell's Account (31296e3a285)
2. IBM Client Developer Advocacy (e65910fa61) <-> 1234567
Enter a number> **2**
Targeted account IBM Client Developer Advocacy (e65910fa61) <-> 1234567
```

1. Retrieve your cluster information.

```console
ibmcloud ks clusters
```

outputs,

```console
$ ibmcloud ks clusters

Name ID State Created Workers Location Version Resource Group Name Provider
<yourcluster> br78vuhd069a00er8s9g normal 1 day ago 1 Dallas 1.16.10_1533 default classic
```

1. Retrieve the name of your cluster, in this example, I set the name of the first cluster with index `0`,

```console
CLUSTER_NAME=$(ibmcloud ks clusters --output json | jq -r '.[0].name')
echo $CLUSTER_NAME
```

1. **In your browser:** get the login command for your cluster:

1. Go to the IBM Cloud resources page at [https://cloud.ibm.com/resources](https://cloud.ibm.com/resources),
Under `Clusters` find and select your cluster, and load the cluster overview page. There are two ways to retrieve the login command with token:
1. Click the `Actions` drop down next to the `OpenShift web console` button, and select `Connect via CLI`, in the pop-up window, click the `oauth token request page` link, or
1. Click `OpenShift web console` button, in the `OpenShift web console`, click your profile name, such as `IAM#name@email.com`, and then click `Copy Login Command`.

![Copy Login Command](../images/cluster/ocp_copylogincommand.png)

In the new page that opens for both options, click `Display Token`,

![Display Token](../images/cluster/ocp_displaytoken.png)

Copy the `oc login` command,

![Copy oc login](../images/cluster/ocp_copy_oclogin.png)

and paste the command into your terminal.

```console
$ oc login --token=HjXc6nNGyCB1imhqtc9csTmGQ5obrPcoe4SRJqTnnT8 --server=https://c100-e.us-south.containers.cloud.ibm.com:30712
Logged into "https://c100-e.us-south.containers.cloud.ibm.com:30712" as "IAM#b.newell2@remkoh.dev" using the token provided.

You have one project on this server: "<your-project>"

Using project "<your-project>".
Welcome! See 'oc help' to get started.
```

1. Create a new project `cos-with-s3fs`,

```console
oc new-project $NAMESPACE
```

1. Make sure you're still logged in to your cluster and namespace,

```console
oc project

Using project "cos-with-s3fs"
```

## Create a Secret to Access the Object Storage

Create a `Kubernetes Secret` to store the COS service credentials named `cos-write-access`.

```console
oc create secret generic cos-write-access --type=ibm/ibmc-s3fs --from-literal=api-key=$COS_APIKEY --from-literal=service-instance-id=$COS_GUID
```

outputs,

```console
$ oc create secret generic cos-write-access --type=ibm/ibmc-s3fs --from-literal=api-key=$COS_APIKEY --from-literal=service-instance-id=$COS_GUID

secret/cos-write-access created
```

## Next

[Setup the Cloud Object Storage plugin](COS-PLUGIN.md)
Loading