Skip to content

Commit

Permalink
start to add storage experiment setup
Browse files Browse the repository at this point in the history
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
  • Loading branch information
vsoch committed Sep 22, 2023
1 parent 2ce6f4e commit fbcbbab
Show file tree
Hide file tree
Showing 7 changed files with 965 additions and 0 deletions.
50 changes: 50 additions & 0 deletions google/kubecon/storage-fio/run2/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Additional Volume Setups

In addition to the already implemented described in [run1](../run1), I'd like to focus
on benchmarking single node for [provided storage solutions](https://cloud.google.com/kubernetes-engine/docs/concepts/storage-overview)
meaning we don't have to setup our own filesystem in addition.

## Contenders

### Block Storage Persistent Disk

- Includes filestore (already ready to go for experiment)
- [Cloud Volumes Service](https://netapp-trident.readthedocs.io/en/stable-v20.01/kubernetes/operations/tasks/backends/cvs_gcp.html) (requires external service, no)
- [Google Compute Engine Persistent Disk](https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/gce-pd-csi-driver) TODO try this.

## Choices

These are the ones I want to test. Filestore was already tested in [run0](run0) and
the yamls are included here in [crd](crd). We need to figure out the rest.

### Persistent Volumes

We are going to try a vanilla cluster as described in the [docs](https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/gce-pd-csi-driver) first:

#### 1. Create Cluster

```bash
gcloud container clusters create storage-cluster \
--addons=GcePersistentDiskCsiDriver \
--machine-type=c2d-standard-2
```

#### 2. Get Storageclass

Get the storage classes available. We will want to prepare files (storage classes) for each.

```bash
kubectl get sc
```
```console
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
premium-rwo pd.csi.storage.gke.io Delete WaitForFirstConsumer true 2m2s
standard kubernetes.io/gce-pd Delete Immediate true 2m1s
standard-rwo (default) pd.csi.storage.gke.io Delete WaitForFirstConsumer true 2m1s
```

Let's create additional ones we can ask for. Note that [we cannot use the c3 type](https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/gce-pd-csi-driver#unsupported_machine_types).


TODO I think I need to [create a persistent disk](https://cloud.google.com/compute/docs/disks/extreme-persistent-disk#provisioning_iops) too.
This means we can ask for any three of these volumes, and
22 changes: 22 additions & 0 deletions google/kubecon/storage-fio/run2/crd/metrics-filestore.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
apiVersion: flux-framework.org/v1alpha1
kind: MetricSet
metadata:
labels:
app.kubernetes.io/name: metricset
app.kubernetes.io/instance: metricset-sample
name: metricset-sample
spec:
storage:
volume:
# The claim name in pvc-filestore.yaml
claimName: data

# Mount Google filestore here in the container
path: /filestore

metrics:
- name: io-fio
options:
directory: /filestore
size: SUB_SIZE
blocksize: SUB_BLOCKSIZE
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: pd-balanced
provisioner: pd.csi.storage.gke.io
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
parameters:
type: pd-balanced
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: pd-extreme
provisioner: pd.csi.storage.gke.io
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
parameters:
type: pd-extreme
# IMPORTANT this needs to match the disk we make
provisioned-iops-on-create: '10000'
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: pd-ssd
provisioner: pd.csi.storage.gke.io
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
parameters:
type: pd-ssd
11 changes: 11 additions & 0 deletions google/kubecon/storage-fio/run2/crd/pvc-filestore.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: data
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Ti
storageClassName: standard-rwx

0 comments on commit fbcbbab

Please sign in to comment.