From 589bfee0d11389ce5779f7f087ea19533df8a2c1 Mon Sep 17 00:00:00 2001 From: sudo-rgorai Date: Tue, 13 Jun 2023 17:21:45 +0530 Subject: [PATCH 01/10] Verify SSH fingerprint before connection --- trusted-fingerprint/README.md | 91 ++++++++++++++++++++++ trusted-fingerprint/deploy-lambda-layer.sh | 47 +++++++++++ trusted-fingerprint/lambda/server.py | 26 +++++++ trusted-fingerprint/requirements.txt | 1 + trusted-fingerprint/update-lambda.sh | 16 ++++ trusted-fingerprint/update-layer.sh | 34 ++++++++ trusted-fingerprint/verify-ssh-key.sh | 61 +++++++++++++++ 7 files changed, 276 insertions(+) create mode 100644 trusted-fingerprint/README.md create mode 100755 trusted-fingerprint/deploy-lambda-layer.sh create mode 100644 trusted-fingerprint/lambda/server.py create mode 100644 trusted-fingerprint/requirements.txt create mode 100755 trusted-fingerprint/update-lambda.sh create mode 100755 trusted-fingerprint/update-layer.sh create mode 100755 trusted-fingerprint/verify-ssh-key.sh diff --git a/trusted-fingerprint/README.md b/trusted-fingerprint/README.md new file mode 100644 index 0000000..1246a04 --- /dev/null +++ b/trusted-fingerprint/README.md @@ -0,0 +1,91 @@ +# SSH Key Retrieval + +This function, deployed as an AWS lambda function retrieves the SSH public key from a remote server using the Paramiko library. The function expects two parameters in the event payload: `url` and `keyType`. It establishes an SSH connection to the specified host using the provided URL and retrieves the public key using the specified key type. + +## Dependencies + +The function requires the Paramiko library to be installed. This can be included as a Lambda layer when creating the Lambda function. + +## Usage + +To set up as a Lambda function, follow these steps: + +1. Create a new IAM role with `AWSLambdaBasicExecutionRole` policy. +2. Copy the Role ARN and set it as `VERIFY_SSH_LAMBDA_ROLE_ARN` environment variable (Alternatively, you can pass it as an input parameter to `deploy-lambda-layer.sh`). +3. Run `deploy-lambda-layer.sh` to deploy the lambda function along with the layer. +4. To update the lambda function, edit `server.py` then run `update-lambda.sh`. +5. To update the layer, modify `requirements.txt` and then run `update-layer.sh`. +6. Configure an API via API Gateway as an event source to trigger the Lambda function with the required event payload. + +To set up an API using API Gateway, follow these steps: + +1. Click on `Add trigger` in lambda function homepage. +2. Select `API Gateway` as source. +3. Select `Create a new API` and choose `REST API`. +4. Select `API key` under security. +5. Go to the API's homepage and under `actions` select `Create Method` and create a `GET` method. +6. Choose `Integration type` as `Lambda Function` and add the name of the function in `Lambda function` field. Click on `Save`. +7. In the API homepage, click on `Method request`. Select `Validate query string parameters and headers` under `Request Validator` and set `API Key Required` to `true`. +8. Under `URL Query String Parameters` add parameters: `keyType` and `url`. Set them both to `Required`. +9. Go back to `Method Execution` and click on `Integration Request`. +10. Click on `Mapping Templates` and under `Request body passthrough` select `When there are no templates defined`. +11. Click on `Add mapping template` and enter `application/json` under `Content-Type` +12. Add the following template underneath it: +``` +{ + "url": "$input.params('url')", + "keyType": "$input.params('keyType')" +} +``` +13. Click on `save` +14. Go back to `Method Execution` and click on `Test` to test the API. + +To set up Github Actions Workflow, follow these steps: + +1. Create a new role in IAM dashboard. +2. Select `Web Identity` under `Trusted entity type`. +3. Click on `Create new` under `Identity provider`. +4. Select `OpenID Connect` as `Provider type`. +5. For the `Provider URL`: Use `https://token.actions.githubusercontent.com` +6. For the `Audience`: Use `sts.amazonaws.com`. +7. Create a role with this identity provider. +8. Add the following policy to the role under permission policy: +``` +{ + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "VisualEditor0", + "Effect": "Allow", + "Action": "lambda:UpdateFunctionCode", + "Resource": + } + ] +} +``` +9. Edit `Trusted entities` under `Trust Relationships` to add the `sub` field to the validation conditions. For example: +``` +"Condition": { + "StringEquals": { + "token.actions.githubusercontent.com:aud": "sts.amazonaws.com", + "token.actions.githubusercontent.com:sub": "repo:getfundwave/production:ref:refs/heads/master" + } +} +``` +10. Add the following workflow secrets: +- `AWS_REGION` - The AWS region in which the lambda is created +- `AWS_SSH_UPDATE_ROLE_ARN` - The ARN for the role created above + +### Event Payload + +The Lambda function expects the following parameters in the `event` object: + +- `url` (string): The URL or IP address of the remote server to connect to. +- `keyType` (string): The type of SSH key to retrieve (e.g., "ssh-rsa", "ssh-ed25519", etc.). + +### Return Value + +The Lambda function returns a JSON object with the following properties: + +- `statusCode` (integer): The HTTP status code of the response. +- `keyBody` (string): The base64-encoded string representation of the retrieved SSH public key. If an error occurs during the retrieval process, this property will be set to `null`. \ No newline at end of file diff --git a/trusted-fingerprint/deploy-lambda-layer.sh b/trusted-fingerprint/deploy-lambda-layer.sh new file mode 100755 index 0000000..40d37ae --- /dev/null +++ b/trusted-fingerprint/deploy-lambda-layer.sh @@ -0,0 +1,47 @@ +#!/bin/bash + +# Variables +FUNCTION_NAME=${1:-"VerifySSHKey"} +LAYER_NAME=${2:-"paramiko-layer"} +VERIFY_SSH_LAMBDA_ROLE_ARN=${3:-$VERIFY_SSH_LAMBDA_ROLE_ARN} + +LAMBDA_FUNCTION_ZIP_FILE="lambda-function.zip" +LAYER_ZIP_FILE="layer.zip" + +# Create a virtual environment and install dependencies +python3.8 -m venv env +source env/bin/activate +mkdir -p python/lib/python3.8/site-packages + +# As Paramiko has system level dependencies, we must package them in the same environment that lambda uses. +# Here we use the AWS Serverless Application Model (SAM) image to emulate the python3.8 AWS Lambda runtime. +# https://gallery.ecr.aws/sam/build-python3.8 +docker run -v "$PWD":/var/task "public.ecr.aws/sam/build-python3.8" /bin/sh -c "pip install --platform manylinux2010_x86_64 --implementation cp --python 3.8 --only-binary=:all: --upgrade -r requirements.txt -t python/lib/python3.8/site-packages; exit" + +# Package the Lambda function and the Paramiko layer +zip -r9 $LAMBDA_FUNCTION_ZIP_FILE server.py +zip -r9 $LAYER_ZIP_FILE python + +# Create the Lambda layer +LAYER_ARN=$(aws lambda publish-layer-version \ + --layer-name $LAYER_NAME \ + --description "Layer for Paramiko" \ + --compatible-runtimes python3.8 \ + --zip-file fileb://$LAYER_ZIP_FILE \ + --query 'LayerVersionArn' \ + --output text) + +# Create the Lambda function +aws lambda create-function \ + --function-name $FUNCTION_NAME \ + --runtime python3.8 \ + --handler server.lambda_handler \ + --zip-file fileb://$LAMBDA_FUNCTION_ZIP_FILE \ + --layers $LAYER_ARN \ + --role $VERIFY_SSH_LAMBDA_ROLE_ARN + +# Clean up +deactivate +sudo rm -r env python +sudo rm *.zip +docker rmi -f public.ecr.aws/sam/build-python3.8 \ No newline at end of file diff --git a/trusted-fingerprint/lambda/server.py b/trusted-fingerprint/lambda/server.py new file mode 100644 index 0000000..2bc83f3 --- /dev/null +++ b/trusted-fingerprint/lambda/server.py @@ -0,0 +1,26 @@ +import paramiko + +def lambda_handler(event, context): + + try: + host = event['url'] + key_type = event['keyType'] + + transport = paramiko.Transport(host) + transport.get_security_options().key_types = [key_type] + transport.connect() + + key = transport.get_remote_server_key() + key_body = key.get_base64() + + transport.close() + + return { + 'statusCode': 200, + 'keyBody': key_body + } + except: + return { + 'statusCode': 500, + 'keyBody': None + } diff --git a/trusted-fingerprint/requirements.txt b/trusted-fingerprint/requirements.txt new file mode 100644 index 0000000..fd8d419 --- /dev/null +++ b/trusted-fingerprint/requirements.txt @@ -0,0 +1 @@ +Paramiko \ No newline at end of file diff --git a/trusted-fingerprint/update-lambda.sh b/trusted-fingerprint/update-lambda.sh new file mode 100755 index 0000000..03c5e7a --- /dev/null +++ b/trusted-fingerprint/update-lambda.sh @@ -0,0 +1,16 @@ +#!/bin/bash + +# Variables +FUNCTION_NAME=${1:-"VerifySSHKey"} +ZIP_FILE="lambda-function.zip" + +# Package the updated Lambda function code +zip -r9 $ZIP_FILE server.py + +# Update the Lambda function code +aws lambda update-function-code \ + --function-name $FUNCTION_NAME \ + --zip-file fileb://$ZIP_FILE + +# Clean up +rm $ZIP_FILE diff --git a/trusted-fingerprint/update-layer.sh b/trusted-fingerprint/update-layer.sh new file mode 100755 index 0000000..233e234 --- /dev/null +++ b/trusted-fingerprint/update-layer.sh @@ -0,0 +1,34 @@ +#!/bin/bash + +# Variables +LAYER_NAME=${1:-"paramiko-layer"} +FUNCTION_NAME=${2:-"VerifySSHKey"} +ZIP_FILE="layer.zip" + +# Create a virtual environment and install dependencies +python3.8 -m venv env +source env/bin/activate +mkdir -p python/lib/python3.8/site-packages + +docker run -v "$PWD":/var/task "public.ecr.aws/sam/build-python3.8" /bin/sh -c "pip install --platform manylinux2010_x86_64 --implementation cp --python 3.8 --only-binary=:all: --upgrade -r requirements.txt -t python/lib/python3.8/site-packages; exit" + +# Package the updated layer code +zip -r9 $ZIP_FILE python + +# Publish a new version of the Lambda layer +LAYER_ARN=$(aws lambda publish-layer-version \ + --layer-name $LAYER_NAME \ + --description "My Layer" \ + --compatible-runtimes python3.8 \ + --zip-file fileb://$ZIP_FILE \ + --query 'LayerVersionArn' \ + --output text) + +# Update the Lambda function(s) that use the layer +aws lambda update-function-configuration \ + --function-name $FUNCTION_NAME \ + --layers $LAYER_ARN + +# Clean up +sudo rm -r env python +rm $ZIP_FILE diff --git a/trusted-fingerprint/verify-ssh-key.sh b/trusted-fingerprint/verify-ssh-key.sh new file mode 100755 index 0000000..65fc9f1 --- /dev/null +++ b/trusted-fingerprint/verify-ssh-key.sh @@ -0,0 +1,61 @@ +#!/bin/bash + +HOST=$1 +USER_KEY=$2 +VERIFY_SSH_LAMBDA_URL=${3:-$VERIFY_SSH_LAMBDA_URL} +VERIFY_SSH_LAMBDA_API_KEY=${4:-$VERIFY_SSH_LAMBDA_API_KEY} + +USER_KEY_TYPE=$(echo $USER_KEY | cut -d " " -f 1) +KNOWN_HOSTS="/home/$USER/.ssh/known_hosts" + +if [[ $USER_KEY_TYPE == "ssh-ed25519" ]]; then + host_key=$(ssh-keyscan -t ed25519 $HOST | awk '{print $3}') +elif [[ $USER_KEY_TYPE == "ssh-rsa" ]]; then + host_key=$(ssh-keyscan -t rsa $HOST | awk '{print $3}') +elif [[ $USER_KEY_TYPE == "ecdsa-sha2-nistp256" ]]; then + host_key=$(ssh-keyscan -t ecdsa $HOST | awk '{print $3}') +fi + +# Check if the key is empty +if [[ -z "$host_key" ]]; then + echo "The corresponding $USER_KEY_TYPE key could not be found on the host" + exit 1 +fi + +hashed_hostname=$(echo -n "$HOST" | sha256sum | cut -d " " -f 1 | awk '{ print $1 }') + +# Check if the hostname (hashed or unhashed) exists in known hosts +if [[ $(grep -q "$HOST" "$KNOWN_HOSTS"; echo $?) -eq 0 || $(grep -q "$hashed_hostname" "$KNOWN_HOSTS"; echo $?) -eq 0 ]]; then + echo "Key found in known_hosts." + exit 0 +else + echo "Key not found in known_hosts." + echo "Attempting key verification..." + + lambda_response=$(curl --location --request GET "$VERIFY_SSH_LAMBDA_URL?url=$HOST&keyType=$USER_KEY_TYPE" --header "x-api-key: $VERIFY_SSH_LAMBDA_API_KEY"| awk '{print}') + lambda_response_status=$(echo $lambda_response | cut -d " " -f 2 | cut -d '"' -f 2 | cut -d '}' -f 1| cut -d ',' -f 1) + lambda_response_key=$(echo $lambda_response | cut -d " " -f 4 | cut -d '"' -f 2 | cut -d '}' -f 1) + + echo $lambda_response_status + + if [[ $lambda_response_status != "200" ]]; then + echo "Encountered server error" + exit 1 + fi + + if [[ + $host_key == $lambda_response_key + ]]; then + + echo "Key verified." + echo "Adding keys to known hosts" + echo "$HOST $USER_KEY_TYPE $lambda_response_key" >> $KNOWN_HOSTS + exit 0 + + else + echo "Key could not be verified" + echo "Host Key ($host_key) does not match with lambda response ($lambda_response_key)" + exit 1 + fi + +fi \ No newline at end of file From 5243aa0030dcf38d5f0075e50da6a318d66b66e7 Mon Sep 17 00:00:00 2001 From: sudo-rgorai Date: Tue, 13 Jun 2023 17:27:53 +0530 Subject: [PATCH 02/10] Change file locations --- trusted-fingerprint/{ => lambda}/README.md | 0 trusted-fingerprint/{ => lambda}/deploy-lambda-layer.sh | 0 trusted-fingerprint/{ => lambda}/requirements.txt | 0 trusted-fingerprint/{ => lambda}/update-lambda.sh | 0 trusted-fingerprint/{ => lambda}/update-layer.sh | 0 5 files changed, 0 insertions(+), 0 deletions(-) rename trusted-fingerprint/{ => lambda}/README.md (100%) rename trusted-fingerprint/{ => lambda}/deploy-lambda-layer.sh (100%) rename trusted-fingerprint/{ => lambda}/requirements.txt (100%) rename trusted-fingerprint/{ => lambda}/update-lambda.sh (100%) rename trusted-fingerprint/{ => lambda}/update-layer.sh (100%) diff --git a/trusted-fingerprint/README.md b/trusted-fingerprint/lambda/README.md similarity index 100% rename from trusted-fingerprint/README.md rename to trusted-fingerprint/lambda/README.md diff --git a/trusted-fingerprint/deploy-lambda-layer.sh b/trusted-fingerprint/lambda/deploy-lambda-layer.sh similarity index 100% rename from trusted-fingerprint/deploy-lambda-layer.sh rename to trusted-fingerprint/lambda/deploy-lambda-layer.sh diff --git a/trusted-fingerprint/requirements.txt b/trusted-fingerprint/lambda/requirements.txt similarity index 100% rename from trusted-fingerprint/requirements.txt rename to trusted-fingerprint/lambda/requirements.txt diff --git a/trusted-fingerprint/update-lambda.sh b/trusted-fingerprint/lambda/update-lambda.sh similarity index 100% rename from trusted-fingerprint/update-lambda.sh rename to trusted-fingerprint/lambda/update-lambda.sh diff --git a/trusted-fingerprint/update-layer.sh b/trusted-fingerprint/lambda/update-layer.sh similarity index 100% rename from trusted-fingerprint/update-layer.sh rename to trusted-fingerprint/lambda/update-layer.sh From 725342cfac02abc590b4ec4a5a6e5931f62ae7fd Mon Sep 17 00:00:00 2001 From: sudo-rgorai Date: Wed, 14 Jun 2023 10:59:34 +0530 Subject: [PATCH 03/10] Move python env inside docker container --- trusted-fingerprint/lambda/deploy-lambda-layer.sh | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/trusted-fingerprint/lambda/deploy-lambda-layer.sh b/trusted-fingerprint/lambda/deploy-lambda-layer.sh index 40d37ae..27f0dda 100755 --- a/trusted-fingerprint/lambda/deploy-lambda-layer.sh +++ b/trusted-fingerprint/lambda/deploy-lambda-layer.sh @@ -9,14 +9,12 @@ LAMBDA_FUNCTION_ZIP_FILE="lambda-function.zip" LAYER_ZIP_FILE="layer.zip" # Create a virtual environment and install dependencies -python3.8 -m venv env -source env/bin/activate mkdir -p python/lib/python3.8/site-packages # As Paramiko has system level dependencies, we must package them in the same environment that lambda uses. # Here we use the AWS Serverless Application Model (SAM) image to emulate the python3.8 AWS Lambda runtime. # https://gallery.ecr.aws/sam/build-python3.8 -docker run -v "$PWD":/var/task "public.ecr.aws/sam/build-python3.8" /bin/sh -c "pip install --platform manylinux2010_x86_64 --implementation cp --python 3.8 --only-binary=:all: --upgrade -r requirements.txt -t python/lib/python3.8/site-packages; exit" +docker run -v "$PWD":/var/task "public.ecr.aws/sam/build-python3.8" /bin/sh -c "python3.8 -m venv env && source env/bin/activate && pip install --platform manylinux2010_x86_64 --implementation cp --python 3.8 --only-binary=:all: --upgrade -r requirements.txt -t python/lib/python3.8/site-packages; exit" # Package the Lambda function and the Paramiko layer zip -r9 $LAMBDA_FUNCTION_ZIP_FILE server.py @@ -41,7 +39,6 @@ aws lambda create-function \ --role $VERIFY_SSH_LAMBDA_ROLE_ARN # Clean up -deactivate sudo rm -r env python sudo rm *.zip docker rmi -f public.ecr.aws/sam/build-python3.8 \ No newline at end of file From 7ba67e51ffacb2324f48681d8ca0e9733b8b7550 Mon Sep 17 00:00:00 2001 From: sudo-rgorai Date: Thu, 6 Jul 2023 18:19:54 +0530 Subject: [PATCH 04/10] Return server error body, remove unnecessary sudo and mkdir, accept user ssh directory as param --- trusted-fingerprint/lambda/deploy-lambda-layer.sh | 4 +--- trusted-fingerprint/lambda/server.py | 2 +- trusted-fingerprint/lambda/update-layer.sh | 1 - trusted-fingerprint/verify-ssh-key.sh | 2 +- 4 files changed, 3 insertions(+), 6 deletions(-) diff --git a/trusted-fingerprint/lambda/deploy-lambda-layer.sh b/trusted-fingerprint/lambda/deploy-lambda-layer.sh index 27f0dda..9941a35 100755 --- a/trusted-fingerprint/lambda/deploy-lambda-layer.sh +++ b/trusted-fingerprint/lambda/deploy-lambda-layer.sh @@ -9,8 +9,6 @@ LAMBDA_FUNCTION_ZIP_FILE="lambda-function.zip" LAYER_ZIP_FILE="layer.zip" # Create a virtual environment and install dependencies -mkdir -p python/lib/python3.8/site-packages - # As Paramiko has system level dependencies, we must package them in the same environment that lambda uses. # Here we use the AWS Serverless Application Model (SAM) image to emulate the python3.8 AWS Lambda runtime. # https://gallery.ecr.aws/sam/build-python3.8 @@ -40,5 +38,5 @@ aws lambda create-function \ # Clean up sudo rm -r env python -sudo rm *.zip +rm lambda-function.zip layer.zip docker rmi -f public.ecr.aws/sam/build-python3.8 \ No newline at end of file diff --git a/trusted-fingerprint/lambda/server.py b/trusted-fingerprint/lambda/server.py index 2bc83f3..77e8d19 100644 --- a/trusted-fingerprint/lambda/server.py +++ b/trusted-fingerprint/lambda/server.py @@ -22,5 +22,5 @@ def lambda_handler(event, context): except: return { 'statusCode': 500, - 'keyBody': None + 'body': 'Internal Server Error' } diff --git a/trusted-fingerprint/lambda/update-layer.sh b/trusted-fingerprint/lambda/update-layer.sh index 233e234..9020e5c 100755 --- a/trusted-fingerprint/lambda/update-layer.sh +++ b/trusted-fingerprint/lambda/update-layer.sh @@ -8,7 +8,6 @@ ZIP_FILE="layer.zip" # Create a virtual environment and install dependencies python3.8 -m venv env source env/bin/activate -mkdir -p python/lib/python3.8/site-packages docker run -v "$PWD":/var/task "public.ecr.aws/sam/build-python3.8" /bin/sh -c "pip install --platform manylinux2010_x86_64 --implementation cp --python 3.8 --only-binary=:all: --upgrade -r requirements.txt -t python/lib/python3.8/site-packages; exit" diff --git a/trusted-fingerprint/verify-ssh-key.sh b/trusted-fingerprint/verify-ssh-key.sh index 65fc9f1..4ee2497 100755 --- a/trusted-fingerprint/verify-ssh-key.sh +++ b/trusted-fingerprint/verify-ssh-key.sh @@ -4,9 +4,9 @@ HOST=$1 USER_KEY=$2 VERIFY_SSH_LAMBDA_URL=${3:-$VERIFY_SSH_LAMBDA_URL} VERIFY_SSH_LAMBDA_API_KEY=${4:-$VERIFY_SSH_LAMBDA_API_KEY} +KNOWN_HOSTS=${5:-"/home/$USER/.ssh/known_hosts"} USER_KEY_TYPE=$(echo $USER_KEY | cut -d " " -f 1) -KNOWN_HOSTS="/home/$USER/.ssh/known_hosts" if [[ $USER_KEY_TYPE == "ssh-ed25519" ]]; then host_key=$(ssh-keyscan -t ed25519 $HOST | awk '{print $3}') From ec01eef59624422f2f25ff99210319b164d10734 Mon Sep 17 00:00:00 2001 From: sudo-rgorai Date: Fri, 7 Jul 2023 11:30:13 +0530 Subject: [PATCH 05/10] Remove sudo for deleting env/ and python/ --- trusted-fingerprint/lambda/deploy-lambda-layer.sh | 2 +- trusted-fingerprint/lambda/update-layer.sh | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/trusted-fingerprint/lambda/deploy-lambda-layer.sh b/trusted-fingerprint/lambda/deploy-lambda-layer.sh index 9941a35..bdd9a47 100755 --- a/trusted-fingerprint/lambda/deploy-lambda-layer.sh +++ b/trusted-fingerprint/lambda/deploy-lambda-layer.sh @@ -37,6 +37,6 @@ aws lambda create-function \ --role $VERIFY_SSH_LAMBDA_ROLE_ARN # Clean up -sudo rm -r env python +rm -r env python rm lambda-function.zip layer.zip docker rmi -f public.ecr.aws/sam/build-python3.8 \ No newline at end of file diff --git a/trusted-fingerprint/lambda/update-layer.sh b/trusted-fingerprint/lambda/update-layer.sh index 9020e5c..c5144bb 100755 --- a/trusted-fingerprint/lambda/update-layer.sh +++ b/trusted-fingerprint/lambda/update-layer.sh @@ -29,5 +29,5 @@ aws lambda update-function-configuration \ --layers $LAYER_ARN # Clean up -sudo rm -r env python +rm -r env python rm $ZIP_FILE From b04aff2e4abde72cb2acbf7a506dd45adcc4156b Mon Sep 17 00:00:00 2001 From: sudo-rgorai Date: Tue, 11 Jul 2023 18:28:29 +0530 Subject: [PATCH 06/10] Use HTTP instead of REST API Gateway --- trusted-fingerprint/lambda/README.md | 46 +++++++++++++--------------- trusted-fingerprint/lambda/server.py | 28 ++++++++++++++--- 2 files changed, 45 insertions(+), 29 deletions(-) diff --git a/trusted-fingerprint/lambda/README.md b/trusted-fingerprint/lambda/README.md index 1246a04..85ac9d8 100644 --- a/trusted-fingerprint/lambda/README.md +++ b/trusted-fingerprint/lambda/README.md @@ -1,6 +1,6 @@ # SSH Key Retrieval -This function, deployed as an AWS lambda function retrieves the SSH public key from a remote server using the Paramiko library. The function expects two parameters in the event payload: `url` and `keyType`. It establishes an SSH connection to the specified host using the provided URL and retrieves the public key using the specified key type. +This function, deployed as an AWS lambda function retrieves the SSH public key from a remote server using the Paramiko library. The function expects three parameters in the event payload: `Host`, `KeyType` and `Authorization`. It establishes an SSH connection to the specified host using the provided URL and retrieves the public key using the specified key type. ## Dependencies @@ -21,24 +21,8 @@ To set up an API using API Gateway, follow these steps: 1. Click on `Add trigger` in lambda function homepage. 2. Select `API Gateway` as source. -3. Select `Create a new API` and choose `REST API`. -4. Select `API key` under security. -5. Go to the API's homepage and under `actions` select `Create Method` and create a `GET` method. -6. Choose `Integration type` as `Lambda Function` and add the name of the function in `Lambda function` field. Click on `Save`. -7. In the API homepage, click on `Method request`. Select `Validate query string parameters and headers` under `Request Validator` and set `API Key Required` to `true`. -8. Under `URL Query String Parameters` add parameters: `keyType` and `url`. Set them both to `Required`. -9. Go back to `Method Execution` and click on `Integration Request`. -10. Click on `Mapping Templates` and under `Request body passthrough` select `When there are no templates defined`. -11. Click on `Add mapping template` and enter `application/json` under `Content-Type` -12. Add the following template underneath it: -``` -{ - "url": "$input.params('url')", - "keyType": "$input.params('keyType')" -} -``` -13. Click on `save` -14. Go back to `Method Execution` and click on `Test` to test the API. +3. Select `Create a new API` and choose `HTTP API`. +4. Select `Open` under security and click on `Add`. To set up Github Actions Workflow, follow these steps: @@ -50,7 +34,7 @@ To set up Github Actions Workflow, follow these steps: 6. For the `Audience`: Use `sts.amazonaws.com`. 7. Create a role with this identity provider. 8. Add the following policy to the role under permission policy: -``` +```json { "Version": "2012-10-17", "Statement": [ @@ -64,7 +48,7 @@ To set up Github Actions Workflow, follow these steps: } ``` 9. Edit `Trusted entities` under `Trust Relationships` to add the `sub` field to the validation conditions. For example: -``` +```json "Condition": { "StringEquals": { "token.actions.githubusercontent.com:aud": "sts.amazonaws.com", @@ -80,12 +64,26 @@ To set up Github Actions Workflow, follow these steps: The Lambda function expects the following parameters in the `event` object: -- `url` (string): The URL or IP address of the remote server to connect to. -- `keyType` (string): The type of SSH key to retrieve (e.g., "ssh-rsa", "ssh-ed25519", etc.). +- `Host` (string): The URL or IP address of the remote server to connect to. +- `KeyType` (string): The type of SSH key to retrieve (e.g., "ssh-rsa", "ssh-ed25519", etc.). +- `Authorization` (string): The authorization token with the form `Bearer `. + +### Usage Sample + +```bash +curl -X 'POST' \ + -H 'Content-Type: application/json' \ + -d '{ + "Host": "ravenclaw.fundwave.com", + "KeyType": "ssh-rsa", + "Authorization": "Bearer " + }' \ + +``` ### Return Value The Lambda function returns a JSON object with the following properties: - `statusCode` (integer): The HTTP status code of the response. -- `keyBody` (string): The base64-encoded string representation of the retrieved SSH public key. If an error occurs during the retrieval process, this property will be set to `null`. \ No newline at end of file +- `body` (string): The base64-encoded string representation of the retrieved SSH public key (in case there are no errors). \ No newline at end of file diff --git a/trusted-fingerprint/lambda/server.py b/trusted-fingerprint/lambda/server.py index 77e8d19..a90c488 100644 --- a/trusted-fingerprint/lambda/server.py +++ b/trusted-fingerprint/lambda/server.py @@ -1,10 +1,28 @@ import paramiko +import json +import os def lambda_handler(event, context): - try: - host = event['url'] - key_type = event['keyType'] + event_body = json.loads(event['body']) + + if 'Authorization' not in event_body: + return { + 'statusCode': 403, + 'body': 'Forbidden' + } + + auth = event_body['Authorization'].split() + host = event_body['Host'] + key_type = event_body['KeyType'] + + secret_token = os.environ['SECRET_TOKEN'] + + if len(auth) != 2 or auth[0] != "Bearer" or auth[1] != secret_token: + return { + 'statusCode': 403, + 'body': 'Forbidden' + } transport = paramiko.Transport(host) transport.get_security_options().key_types = [key_type] @@ -17,10 +35,10 @@ def lambda_handler(event, context): return { 'statusCode': 200, - 'keyBody': key_body + 'body': key_body } except: return { 'statusCode': 500, - 'body': 'Internal Server Error' + 'body': 'Internal Server Error ' } From 80647a118d749a6bc259c39d5a7136a60512b491 Mon Sep 17 00:00:00 2001 From: sudo-rgorai Date: Wed, 12 Jul 2023 15:58:35 +0530 Subject: [PATCH 07/10] Update response from curl --- .talismanrc | 1 + .../lambda/deploy-lambda-layer.sh | 8 ++++++- trusted-fingerprint/lambda/update-lambda.sh | 6 ++++- trusted-fingerprint/lambda/update-layer.sh | 10 ++++++-- trusted-fingerprint/verify-ssh-key.sh | 24 +++++++++++++------ 5 files changed, 38 insertions(+), 11 deletions(-) create mode 100644 .talismanrc diff --git a/.talismanrc b/.talismanrc new file mode 100644 index 0000000..afb4a06 --- /dev/null +++ b/.talismanrc @@ -0,0 +1 @@ +threshold: medium \ No newline at end of file diff --git a/trusted-fingerprint/lambda/deploy-lambda-layer.sh b/trusted-fingerprint/lambda/deploy-lambda-layer.sh index bdd9a47..1e342b4 100755 --- a/trusted-fingerprint/lambda/deploy-lambda-layer.sh +++ b/trusted-fingerprint/lambda/deploy-lambda-layer.sh @@ -4,6 +4,8 @@ FUNCTION_NAME=${1:-"VerifySSHKey"} LAYER_NAME=${2:-"paramiko-layer"} VERIFY_SSH_LAMBDA_ROLE_ARN=${3:-$VERIFY_SSH_LAMBDA_ROLE_ARN} +AWS_REGION=${4:-"ap-south-1"} +AWS_PROFILE=${5:-"default"} LAMBDA_FUNCTION_ZIP_FILE="lambda-function.zip" LAYER_ZIP_FILE="layer.zip" @@ -25,6 +27,8 @@ LAYER_ARN=$(aws lambda publish-layer-version \ --compatible-runtimes python3.8 \ --zip-file fileb://$LAYER_ZIP_FILE \ --query 'LayerVersionArn' \ + --region "${AWS_REGION}" \ + --profile "${AWS_PROFILE}" \ --output text) # Create the Lambda function @@ -34,7 +38,9 @@ aws lambda create-function \ --handler server.lambda_handler \ --zip-file fileb://$LAMBDA_FUNCTION_ZIP_FILE \ --layers $LAYER_ARN \ - --role $VERIFY_SSH_LAMBDA_ROLE_ARN + --role $VERIFY_SSH_LAMBDA_ROLE_ARN \ + --region "${AWS_REGION}" \ + --profile "${AWS_PROFILE}" # Clean up rm -r env python diff --git a/trusted-fingerprint/lambda/update-lambda.sh b/trusted-fingerprint/lambda/update-lambda.sh index 03c5e7a..025f755 100755 --- a/trusted-fingerprint/lambda/update-lambda.sh +++ b/trusted-fingerprint/lambda/update-lambda.sh @@ -2,6 +2,8 @@ # Variables FUNCTION_NAME=${1:-"VerifySSHKey"} +AWS_REGION=${2:-"ap-south-1"} +AWS_PROFILE=${3:-"default"} ZIP_FILE="lambda-function.zip" # Package the updated Lambda function code @@ -10,7 +12,9 @@ zip -r9 $ZIP_FILE server.py # Update the Lambda function code aws lambda update-function-code \ --function-name $FUNCTION_NAME \ - --zip-file fileb://$ZIP_FILE + --zip-file fileb://$ZIP_FILE \ + --region "${AWS_REGION}" \ + --profile "${AWS_PROFILE}" # Clean up rm $ZIP_FILE diff --git a/trusted-fingerprint/lambda/update-layer.sh b/trusted-fingerprint/lambda/update-layer.sh index c5144bb..87de7ea 100755 --- a/trusted-fingerprint/lambda/update-layer.sh +++ b/trusted-fingerprint/lambda/update-layer.sh @@ -3,6 +3,8 @@ # Variables LAYER_NAME=${1:-"paramiko-layer"} FUNCTION_NAME=${2:-"VerifySSHKey"} +AWS_REGION=${3:-"ap-south-1"} +AWS_PROFILE=${4:-"default"} ZIP_FILE="layer.zip" # Create a virtual environment and install dependencies @@ -21,12 +23,16 @@ LAYER_ARN=$(aws lambda publish-layer-version \ --compatible-runtimes python3.8 \ --zip-file fileb://$ZIP_FILE \ --query 'LayerVersionArn' \ - --output text) + --output text\ + --region "${AWS_REGION}" \ + --profile "${AWS_PROFILE}") # Update the Lambda function(s) that use the layer aws lambda update-function-configuration \ --function-name $FUNCTION_NAME \ - --layers $LAYER_ARN + --layers $LAYER_ARN \ + --region "${AWS_REGION}" \ + --profile "${AWS_PROFILE}" # Clean up rm -r env python diff --git a/trusted-fingerprint/verify-ssh-key.sh b/trusted-fingerprint/verify-ssh-key.sh index 4ee2497..e6779d8 100755 --- a/trusted-fingerprint/verify-ssh-key.sh +++ b/trusted-fingerprint/verify-ssh-key.sh @@ -3,7 +3,7 @@ HOST=$1 USER_KEY=$2 VERIFY_SSH_LAMBDA_URL=${3:-$VERIFY_SSH_LAMBDA_URL} -VERIFY_SSH_LAMBDA_API_KEY=${4:-$VERIFY_SSH_LAMBDA_API_KEY} +VERIFY_SSH_LAMBDA_TOKEN=${4:-$VERIFY_SSH_LAMBDA_TOKEN} KNOWN_HOSTS=${5:-"/home/$USER/.ssh/known_hosts"} USER_KEY_TYPE=$(echo $USER_KEY | cut -d " " -f 1) @@ -25,24 +25,34 @@ fi hashed_hostname=$(echo -n "$HOST" | sha256sum | cut -d " " -f 1 | awk '{ print $1 }') # Check if the hostname (hashed or unhashed) exists in known hosts -if [[ $(grep -q "$HOST" "$KNOWN_HOSTS"; echo $?) -eq 0 || $(grep -q "$hashed_hostname" "$KNOWN_HOSTS"; echo $?) -eq 0 ]]; then +if [[ $(grep -q "$HOST $USER_KEY_TYPE" "$KNOWN_HOSTS"; echo $?) -eq 0 || + $(grep -q "$hashed_hostname $USER_KEY_TYPE" "$KNOWN_HOSTS"; echo $?) -eq 0 + ]]; then echo "Key found in known_hosts." exit 0 else echo "Key not found in known_hosts." echo "Attempting key verification..." - lambda_response=$(curl --location --request GET "$VERIFY_SSH_LAMBDA_URL?url=$HOST&keyType=$USER_KEY_TYPE" --header "x-api-key: $VERIFY_SSH_LAMBDA_API_KEY"| awk '{print}') - lambda_response_status=$(echo $lambda_response | cut -d " " -f 2 | cut -d '"' -f 2 | cut -d '}' -f 1| cut -d ',' -f 1) - lambda_response_key=$(echo $lambda_response | cut -d " " -f 4 | cut -d '"' -f 2 | cut -d '}' -f 1) - - echo $lambda_response_status + lambda_response_status=$(curl -sw '%{http_code}' \ + -o key.txt \ + -X 'POST' \ + -H 'Content-Type: application/json' \ + -d '{ + "Host": '"\"${HOST}\""', + "KeyType": '"\"${USER_KEY_TYPE}\""', + "Authorization": '"\"Bearer ${VERIFY_SSH_LAMBDA_TOKEN}\""' + }' \ + $VERIFY_SSH_LAMBDA_URL) if [[ $lambda_response_status != "200" ]]; then echo "Encountered server error" exit 1 fi + lambda_response_key=$(cat key.txt) + rm key.txt + if [[ $host_key == $lambda_response_key ]]; then From 8f2baa142625d371ae90583c738a65cedf56c2a5 Mon Sep 17 00:00:00 2001 From: sudo-rgorai Date: Fri, 14 Jul 2023 11:33:55 +0530 Subject: [PATCH 08/10] Add instruction for adding secret token --- trusted-fingerprint/lambda/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/trusted-fingerprint/lambda/README.md b/trusted-fingerprint/lambda/README.md index 85ac9d8..430ba6e 100644 --- a/trusted-fingerprint/lambda/README.md +++ b/trusted-fingerprint/lambda/README.md @@ -15,6 +15,7 @@ To set up as a Lambda function, follow these steps: 3. Run `deploy-lambda-layer.sh` to deploy the lambda function along with the layer. 4. To update the lambda function, edit `server.py` then run `update-lambda.sh`. 5. To update the layer, modify `requirements.txt` and then run `update-layer.sh`. +6. Add an environment variable called `SECRET_TOKEN` and set its value to a token to be used in Authorization header while invoking the lambda. 6. Configure an API via API Gateway as an event source to trigger the Lambda function with the required event payload. To set up an API using API Gateway, follow these steps: From 64ae731c982b4d578b52ac466ecdf74392843ef9 Mon Sep 17 00:00:00 2001 From: sudo-rgorai Date: Fri, 14 Jul 2023 11:48:31 +0530 Subject: [PATCH 09/10] Add github actions workflow for updating lambda --- .../update-ssh-verification-lambda.yml | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+) create mode 100644 .github/workflows/update-ssh-verification-lambda.yml diff --git a/.github/workflows/update-ssh-verification-lambda.yml b/.github/workflows/update-ssh-verification-lambda.yml new file mode 100644 index 0000000..5d5aed6 --- /dev/null +++ b/.github/workflows/update-ssh-verification-lambda.yml @@ -0,0 +1,31 @@ +name: Update SSH Verification Lambda on file change + +on: + push: + paths: + - trusted-fingerprint/lambda/server.py + - .github/workflows/update-ssh-verification-lambda.yml + +permissions: + id-token: write + contents: read + +jobs: + update-lambda: + runs-on: "ubuntu-22.04" + + steps: + - name: Checkout code + uses: actions/checkout@v3 + + - name: Configure AWS Credentials + uses: aws-actions/configure-aws-credentials@v2 + with: + aws-region: ${{ secrets.AWS_REGION }} + role-to-assume: ${{ secrets.AWS_SSH_UPDATE_ROLE_ARN }} + role-session-name: UpdateSSHLambda + + - name: Run update-lambda.sh + run: | + cd trusted-fingerprint/lambda/ + bash update-lambda.sh From 45eaeb012232dd773c9da93f53d8bd902e164005 Mon Sep 17 00:00:00 2001 From: Mohit Garg Date: Mon, 31 Jul 2023 17:48:28 +0530 Subject: [PATCH 10/10] update readme and defaults --- .../{ => client}/verify-ssh-key.sh | 0 trusted-fingerprint/lambda/README.md | 71 ++++++++++--------- ...a-layer.sh => deploy-lambda-with-layer.sh} | 6 +- 3 files changed, 40 insertions(+), 37 deletions(-) rename trusted-fingerprint/{ => client}/verify-ssh-key.sh (100%) rename trusted-fingerprint/lambda/{deploy-lambda-layer.sh => deploy-lambda-with-layer.sh} (94%) diff --git a/trusted-fingerprint/verify-ssh-key.sh b/trusted-fingerprint/client/verify-ssh-key.sh similarity index 100% rename from trusted-fingerprint/verify-ssh-key.sh rename to trusted-fingerprint/client/verify-ssh-key.sh diff --git a/trusted-fingerprint/lambda/README.md b/trusted-fingerprint/lambda/README.md index 430ba6e..faf3492 100644 --- a/trusted-fingerprint/lambda/README.md +++ b/trusted-fingerprint/lambda/README.md @@ -4,27 +4,58 @@ This function, deployed as an AWS lambda function retrieves the SSH public key f ## Dependencies -The function requires the Paramiko library to be installed. This can be included as a Lambda layer when creating the Lambda function. +The function requires the Paramiko library to be installed. This is included as a layer when creating the Lambda function. ## Usage -To set up as a Lambda function, follow these steps: +### To set up as a Lambda function, follow these steps: 1. Create a new IAM role with `AWSLambdaBasicExecutionRole` policy. -2. Copy the Role ARN and set it as `VERIFY_SSH_LAMBDA_ROLE_ARN` environment variable (Alternatively, you can pass it as an input parameter to `deploy-lambda-layer.sh`). -3. Run `deploy-lambda-layer.sh` to deploy the lambda function along with the layer. +2. Copy the Role ARN and set it as `VERIFY_SSH_LAMBDA_ROLE_ARN` environment variable (Alternatively, you can pass it as an input parameter to `deploy-lambda-with-layer.sh`). +3. Run `deploy-lambda-with-layer.sh` to deploy the lambda function along with the layer. 4. To update the lambda function, edit `server.py` then run `update-lambda.sh`. 5. To update the layer, modify `requirements.txt` and then run `update-layer.sh`. 6. Add an environment variable called `SECRET_TOKEN` and set its value to a token to be used in Authorization header while invoking the lambda. 6. Configure an API via API Gateway as an event source to trigger the Lambda function with the required event payload. -To set up an API using API Gateway, follow these steps: +### To set up an API using API Gateway, follow these steps: 1. Click on `Add trigger` in lambda function homepage. 2. Select `API Gateway` as source. 3. Select `Create a new API` and choose `HTTP API`. 4. Select `Open` under security and click on `Add`. +### Event Payload + +The Lambda function expects the following parameters in the `event` object: + +- `Host` (string): The URL or IP address of the remote server to connect to. +- `KeyType` (string): The type of SSH key to retrieve (e.g., "ssh-rsa", "ssh-ed25519", etc.). +- `Authorization` (string): The authorization token with the form `Bearer `. + +### Usage Sample + +```bash +curl -X 'POST' \ + -H 'Content-Type: application/json' \ + -d '{ + "Host": "", + "KeyType": "ssh-rsa", + "Authorization": "Bearer " + }' \ + +``` + +### Return Value + +The Lambda function returns a JSON object with the following properties: + +- `statusCode` (integer): The HTTP status code of the response. +- `body` (string): The base64-encoded string representation of the retrieved SSH public key (in case there are no errors). + + +## Actions workflow + To set up Github Actions Workflow, follow these steps: 1. Create a new role in IAM dashboard. @@ -53,38 +84,10 @@ To set up Github Actions Workflow, follow these steps: "Condition": { "StringEquals": { "token.actions.githubusercontent.com:aud": "sts.amazonaws.com", - "token.actions.githubusercontent.com:sub": "repo:getfundwave/production:ref:refs/heads/master" + "token.actions.githubusercontent.com:sub": "repo:/:ref:refs/heads/master" } } ``` 10. Add the following workflow secrets: - `AWS_REGION` - The AWS region in which the lambda is created - `AWS_SSH_UPDATE_ROLE_ARN` - The ARN for the role created above - -### Event Payload - -The Lambda function expects the following parameters in the `event` object: - -- `Host` (string): The URL or IP address of the remote server to connect to. -- `KeyType` (string): The type of SSH key to retrieve (e.g., "ssh-rsa", "ssh-ed25519", etc.). -- `Authorization` (string): The authorization token with the form `Bearer `. - -### Usage Sample - -```bash -curl -X 'POST' \ - -H 'Content-Type: application/json' \ - -d '{ - "Host": "ravenclaw.fundwave.com", - "KeyType": "ssh-rsa", - "Authorization": "Bearer " - }' \ - -``` - -### Return Value - -The Lambda function returns a JSON object with the following properties: - -- `statusCode` (integer): The HTTP status code of the response. -- `body` (string): The base64-encoded string representation of the retrieved SSH public key (in case there are no errors). \ No newline at end of file diff --git a/trusted-fingerprint/lambda/deploy-lambda-layer.sh b/trusted-fingerprint/lambda/deploy-lambda-with-layer.sh similarity index 94% rename from trusted-fingerprint/lambda/deploy-lambda-layer.sh rename to trusted-fingerprint/lambda/deploy-lambda-with-layer.sh index 1e342b4..74da419 100755 --- a/trusted-fingerprint/lambda/deploy-lambda-layer.sh +++ b/trusted-fingerprint/lambda/deploy-lambda-with-layer.sh @@ -1,10 +1,10 @@ #!/bin/bash # Variables -FUNCTION_NAME=${1:-"VerifySSHKey"} +FUNCTION_NAME=${1:-"trusted-fingerprint"} LAYER_NAME=${2:-"paramiko-layer"} VERIFY_SSH_LAMBDA_ROLE_ARN=${3:-$VERIFY_SSH_LAMBDA_ROLE_ARN} -AWS_REGION=${4:-"ap-south-1"} +AWS_REGION=${4:-"ap-southeast-1"} AWS_PROFILE=${5:-"default"} LAMBDA_FUNCTION_ZIP_FILE="lambda-function.zip" @@ -43,6 +43,6 @@ aws lambda create-function \ --profile "${AWS_PROFILE}" # Clean up -rm -r env python +sudo rm -r env python rm lambda-function.zip layer.zip docker rmi -f public.ecr.aws/sam/build-python3.8 \ No newline at end of file