Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add EKS tutorial with service account setup. #4669

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
191 changes: 191 additions & 0 deletions docs/deployment/kubernetes/eks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,191 @@
---
title: EKS + Helm
sidebar_position: 3
---

This guide will help you set up a Quickwit cluster on EKS with the correct S3 permissions.

## Prerequisites
- Running Elastic Kubernetes cluster (EKS)
- `kubectl`
- Permission to create the IAM role and Policies
- AWS CLI
- `eksctl` if you don't have an IAM OIDC provider for your cluster.

## Set up

Let's use the following environment variables:

```bash
export NAMESPACE=qw-tutorial
export EKS_CLUSTER=qw-cluster
export S3_BUCKET={your-bucket}
export SERVICE_ACCOUNT_NAME=qw-sa
export REGION={your-region}
export CLUSTER_ID={your-cluster-id}
```

Create the namespace for our playground:

```bash
kubectl create ns ${NAMESPACE}
```

And set this namespace as the default one:

```bash
kubectl config set-context --current --namespace=${NAMESPACE}
```


### Create IAM OIDC provider if you don't have one

To check if you have one provider for your EKS cluster, just run:

```bash
aws iam list-open-id-connect-providers
```

If you have one, you will get a response similar to this one:

```json
{
"OpenIDConnectProviderList": [
{
"Arn": "arn:aws:iam::(some-ID):oidc-provider/oidc.eks.{your-region}.amazonaws.com/id/{your-cluster-id}"
}
]
}
```

If you don't, run the following command:

```bash
eksctl utils associate-iam-oidc-provider --cluster ${EKS_CLUSTER} --approve
```

You can run again `aws iam list-open-id-connect-providers` to get the ARN of the provider.

### Create an IAM policy

You need to set the following policy to allow Quickwit to access your S3 bucket.

Then create the policy using the AWS CLI:

```bash
cat > s3-policy.json <<EOF
{
"Statement": [
{
"Action": [
"s3:AbortMultipartUpload",
"s3:DeleteObject",
"s3:GetObject",
"s3:ListMultipartUploadParts",
"s3:PutObject"
Comment on lines +81 to +85
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the object that you request doesn’t exist, the error that Amazon S3 returns depends on whether you also have the s3:ListBucket permission.
   - If you have the s3:ListBucket permission on the bucket, Amazon S3 returns an HTTP status code 404 Not Found error.
   - If you don’t have the s3:ListBucket permission, Amazon S3 returns an HTTP status code 403 Access Denied error.

(extract from https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html)
we probably need s3:ListBucket (or to handle errors differently), at least when initializing the metastore for the first time

it's a permission to have on arn:aws:s3:::${S3_BUCKET} directly, and not arn:aws:s3:::${S3_BUCKET}/*

],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::${S3_BUCKET}/*"
]
}
],
"Version": "2012-10-17"
}
EOF
```

```bash
aws iam create-policy --policy-name qw-s3-policy --policy-document file://s3-policy.json
```

### Create an IAM Role and attach the policy

```bash
cat > s3-role.json << EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::${IAM_ID}:oidc-provider/oidc.eks.${REGION}.amazonaws.com/id/${CLUSTER_ID}"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.${REGION}.amazonaws.com/id/${CLUSTER_ID}:aud": "sts.amazonaws.com",
"oidc.eks.${REGION}.amazonaws.com/id/${CLUSTER_ID}:sub": "system:serviceaccount:${S3_BUCKET}:${SERVICE_ACCOUNT_NAME}"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"oidc.eks.${REGION}.amazonaws.com/id/${CLUSTER_ID}:sub": "system:serviceaccount:${S3_BUCKET}:${SERVICE_ACCOUNT_NAME}"
"oidc.eks.${REGION}.amazonaws.com/id/${CLUSTER_ID}:sub": "system:serviceaccount:${NAMESPACE}:${SERVICE_ACCOUNT_NAME}"

Think you meant to use NAMESPACE here rather than bucket

}
}
}
]
}
EOF
```

```bash
aws iam create-role --role-name s3-role --assume-role-policy-document file://s3-role.json
```

And then attach the policy to the role:

```bash
aws iam attach-role-policy --role-name s3-role --policy-arn=arn:aws:iam::${IAM_ID}:policy/s3-policy
```

## Install Quickwit using Helm

We are now ready to install Quickwit on EKS. If you'd like to know more about Helm, consult our [comprehensive guide](./helm.md) for installing Quickwit on Kubernetes.

```bash
helm repo add quickwit https://helm.quickwit.io
helm repo update quickwit
```

Let's set Quickwit `values.yaml`:

```yaml
# We use the edge version here as we recently fixed
# a bug which prevents the metastore from running on GCS.
image:
repository: quickwit/quickwit
pullPolicy: Always

serviceAccount:
create: true
name: ${SERVICE_ACCOUNT_NAME}
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::${ARN_ID}:role/${SERVICE_ACCOUNT_NAME}

config:
default_index_root_uri: s3://${S3_BUCKET}/qw-indexes
metastore_uri: s3://${S3_BUCKET}/qw-indexes

```

We're ready to deploy:

```bash
helm install <deployment name> quickwit/quickwit -f values.yaml
```

## Check that Quickwit is running

It should take a few seconds for the cluster to start. During the startup process, individual pods might restart themselves several times.

To access the UI, you can run the following command and then open your browser at [http://localhost:7280](http://localhost:7280):

```
kubectl port-forward svc/{release-name}-quickwit-searcher 7280:7280
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
kubectl port-forward svc/{release-name}-quickwit-searcher 7280:7280
kubectl port-forward svc/quickwit-searcher 7280:7280

Kubetcl seems to not want the release name there

```

## Uninstall the deployment

Run the following Helm command to uninstall the deployment

```bash
helm uninstall <deployment name>
```

And don't forget to clean your bucket, Quickwit should have stored 3 files in `s3://${S3_BUCKET}/qw-indexes`.
16 changes: 10 additions & 6 deletions docs/deployment/kubernetes/gke.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
title: Google GKE
title: GKE + helm
sidebar_position: 2
---

This guide will help you set up a Quickwit cluster with the correct GCS permissions.
This guide will help you set up a Quickwit cluster on GKE with the correct GCS permissions.


## Set up
Expand All @@ -18,7 +18,6 @@ kubectl create ns ${NS}
Quickwit stores its index on an object storage. We will use GCS, which is natively supported since the 0.7 version (for versions < 0.7, you should use an S3 interoperability key).

The following steps create a GCP and a GKE service account and bind them together.
We are going to create them, set the right permissions and bind them.

```bash
export PROJECT_ID={your-project-id}
Expand Down Expand Up @@ -64,9 +63,15 @@ image:
pullPolicy: Always
tag: edge

serviceAccount:
create: true
name: ${GKE_SERVICE_ACCOUNT}
annotations:
iam.gke.io/gcp-service-account: ${GCP_SERVICE_ACCOUNT}@${PROJECT_ID}.iam.gserviceaccount.com

config:
default_index_root_uri: gs://{BUCKET}/qw-indexes
metastore_uri: gs://{BUCKET}/qw-indexes
default_index_root_uri: gs://${BUCKET}/qw-indexes
metastore_uri: gs://${BUCKET}/qw-indexes

```

Expand All @@ -86,7 +91,6 @@ To access the UI, you can run the following command and then open your browser a
kubectl port-forward svc/release-name-quickwit-searcher 7280:7280
```


## Uninstall the deployment

Run the following Helm command to uninstall the deployment
Expand Down
2 changes: 1 addition & 1 deletion docs/deployment/kubernetes/helm.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Kubernetes
title: Helm
sidebar_position: 1
---

Expand Down