Skip to content
CSI Driver of Amazon FSx for Lustre https://aws.amazon.com/fsx/lustre/
Go Shell Makefile Dockerfile
Branch: master
Clone or download
Latest commit 7443ce9 Oct 17, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github
cmd Update makefile and copyright Sep 15, 2019
deploy/kubernetes
docs Update README Oct 17, 2019
examples/kubernetes Update README for IAM policy Oct 4, 2019
hack Update go mod and update mount library Oct 9, 2019
pkg Test framework initial checkin Oct 11, 2019
tester Reduce the feature flags Oct 16, 2019
tests Add e2e test for s3 data repository Oct 15, 2019
.dockerignore Add dockerignore to ignore vendor dir Aug 24, 2019
.gitignore Change static and dynamic example to use ReadWriteMany by default Mar 31, 2019
.travis.yml Switch to use prow job Sep 5, 2019
CHANGELOG-0.x.md Update README Oct 17, 2019
CONTRIBUTING.md Add Kubernetes code of conduct OWNER file Apr 3, 2019
Dockerfile Update makefile and copyright Sep 15, 2019
LICENSE Creating initial file from template Dec 15, 2018
Makefile Update README Oct 17, 2019
OWNERS Add OWNERS, SECURITY_CONTACTS and CoC files Apr 2, 2019
SECURITY_CONTACTS Add OWNERS, SECURITY_CONTACTS and CoC files Apr 2, 2019
THIRD-PARTY Support s3 data repository in dynamic provision (#33) Mar 9, 2019
code-of-conduct.md Add OWNERS, SECURITY_CONTACTS and CoC files Apr 2, 2019
go.mod Test framework initial checkin Oct 11, 2019
go.sum Add e2e test skeleton Oct 10, 2019

README.md

Build Status Coverage Status Go Report Card

NOTE: This driver is currently an ALPHA release. This means that there may potentially be backwards compatible breaking changes moving forward. Do NOT use this driver in a production environment in its current state.

Amazon FSx for Lustre CSI Driver

Overview

The Amazon FSx for Lustre Container Storage Interface (CSI) Driver implements CSI specification for container orchestrators (CO) to manage lifecycle of Amazon FSx for Lustre filesystems.

CSI Specification Compability Matrix

AWS FSx for Lustre CSI Driver \ CSI Version v0.3.0 v1.1.0
master branch no yes
v0.1.0 yes no

Features

The following CSI interfaces are implemented:

  • Controller Service: CreateVolume, DeleteVolume, ControllerGetCapabilities, ValidateVolumeCapabilities
  • Node Service: NodePublishVolume, NodeUnpublishVolume, NodeGetCapabilities, NodeGetInfo, NodeGetId
  • Identity Service: GetPluginInfo, GetPluginCapabilities, Probe

FSx for Lustre CSI Driver on Kubernetes

Following sections are Kubernetes specific. If you are Kubernetes user, use followings for driver features, installation steps and examples.

Kubernetes Version Compability Matrix

AWS FSx for Lustre CSI Driver \ Kubernetes Version v1.11 v1.12 v1.13 v1.14 v1.15
master branch no no no yes yes
v0.1.0 yes yes yes no no

Container Images

FSx CSI Driver Version Image
master branch amazon/aws-fsx-csi-driver:latest
v0.1.0 amazon/aws-fsx-csi-driver:v0.1.0

Features

  • Static provisioning - FSx for Lustre file system needs to be created manually first, then it could be mounted inside container as a volume using the Driver.
  • Dynamic provisioning - uses persistent volume claim (PVC) to let the Kuberenetes to create the FSx for Lustre filesystem for you and consumes the volume from inside container.
  • Mount options - mount options can be specified in storageclass to define how the volume should be mounted.

Notes:

  • For dynamically provisioned volumes, only one subnet is allowed inside storageclass's parameters.subnetId. This is a limitation that is enforced by FSx for Lustre.

Installation

Set up driver permission

The driver requires IAM permission to talk to Amazon FSx for Lustre service to create/delete the filesystem on user's behalf. There are several methods to grant driver IAM permission:

  • Using secret object - create an IAM user with proper permission, put that user's credentials in secret manifest then deploy the secret.
curl https://raw.githubusercontent.com/kubernetes-sigs/aws-fsx-csi-driver/master/deploy/kubernetes/secret.yaml > secret.yaml
# Edit the secret with user credentials
kubectl apply -f secret.yaml
  • Using worker node instance profile - grant all the worker nodes with proper permission by attach policy to the instance profile of the worker.
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "iam:CreateServiceLinkedRole",
        "iam:AttachRolePolicy",
        "iam:PutRolePolicy"
       ],
      "Resource": "arn:aws:iam::*:role/aws-service-role/s3.data-source.lustre.fsx.amazonaws.com/*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:*",
        "fsx:*"
      ],
      "Resource": ["*"]
    }
  ]
}

Deploy driver

kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-fsx-csi-driver/master/deploy/kubernetes/manifest.yaml

Examples

Before the example, you need to:

  • Get yourself familiar with how to setup Kubernetes on AWS and create FSx for Lustre filesystem if you are using static provisioning.
  • When creating FSx for Lustre file system, make sure its VPC is accessible from Kuberenetes cluster's VPC and network traffic is allowed by security group.
    • For FSx for Lustre VPC, you can either create FSx for lustre filesystem inside the same VPC as Kubernetes cluster or using VPC peering.
    • For security group, make sure port 988 is allowed for the security groups that are attached the lustre filesystem ENI.
  • Install FSx for Lustre CSI driver following the Installation steps.

Example links

Development

Please go through CSI Spec and General CSI driver development guideline to get some basic understanding of CSI driver before you start.

Requirements

  • Golang 1.12.7+

Dependency

Dependencies are managed through go module. To build the project, first turn on go mod using export GO111MODULE=on, to build the project run: make

Testing

  • To execute all unit tests, run: make test
  • To execute sanity tests, run: make test-sanity
  • To execute e2e tests, run: make test-e2e

License

This library is licensed under the Apache 2.0 License.

You can’t perform that action at this time.