Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 18 additions & 16 deletions content/en/ninja-workshops/6-lambda-kinesis/1-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ weight: 1
## Prerequisites

### Observability Workshop Instance
The Observability Workshop is most often completed on a Splunk-issued and preconfigured EC2 instance running Ubuntu.
The Observability Workshop uses the `Splunk4Ninjas - Observability` workshop template in Splunk Show,
which provides a pre-configured EC2 instance running Ubuntu.

Your workshop instructor will provide you with the credentials to your assigned workshop instance.

Expand Down Expand Up @@ -66,19 +67,14 @@ Your Splunk-issued workshop instance should already have **terraform** installed
sudo apt update && sudo apt install terraform
```

### Workshop Directory (o11y-lambda-workshop)
The Workshop Directory `o11y-lambda-workshop` is a repository that contains all the configuration files and scripts to complete both the auto-instrumentation and manual instrumentation of the example Lambda-based application we will be using today.
### Workshop Directory (lambda)
The Workshop Directory `lambda` is a repository that contains all the configuration files and scripts to complete both the auto-instrumentation and manual instrumentation of the example Lambda-based application we will be using today.

- Confirm you have the workshop directory in your home directory:
```bash
cd && ls
```
- _The expected output would include **o11y-lambda-workshop**_

- If the **o11y-lambda-workshop** directory is not in your home directory, clone it with the following command:
```bash
git clone https://github.com/gkono-splunk/o11y-lambda-workshop.git
cd ~/workshop && ls
```
- _The expected output would include **lambda**_

### AWS & Terraform Variables

Expand All @@ -92,7 +88,7 @@ The AWS CLI requires that you have credentials to be able to access and manage r
- _This command should provide a prompt similar to the one below:_
```bash
AWS Access Key ID [None]: XXXXXXXXXXXXXXXX
AWS Secret Acces Key [None]: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
AWS Secret Access Key [None]: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Default region name [None]: us-east-1
Default outoput format [None]:
```
Expand All @@ -102,6 +98,11 @@ The AWS CLI requires that you have credentials to be able to access and manage r
aws configure
```

> Note to the workshop instructor: create a new user in the target AWS account called `lambda-workshop-user`.
> Ensure it has full permissions to perform the required actions via Terraform. Create an access token for the `lambda-workshop-user`
> user and share the Access Key ID and Secret Access Key with the workshop participants. Delete the user
> when the workshop is complete.

#### Terraform
Terraform supports the passing of variables to ensure sensitive or dynamic data is not hard-coded in your .tf configuration files, as well as to make those values reusable throughout your resource definitions.

Expand All @@ -118,7 +119,7 @@ We will be using a combination of _**variables.tf**_ and _**terraform.tfvars**_

- Using either **vi** or **nano**, open the _**terraform.tfvars**_ file in either the **auto** or **manual** directory
```bash
vi ~/o11y-lambda-workshop/auto/terraform.tfvars
vi ~/workshop/lambda/auto/terraform.tfvars
```
- Set the variables with their values. Replace the **CHANGEME** placeholders with those provided by your instructor.
```bash
Expand All @@ -128,22 +129,23 @@ We will be using a combination of _**variables.tf**_ and _**terraform.tfvars**_
prefix = "CHANGEME"
```
- _Ensure you change only the placeholders, leaving the quotes and brackets intact, where applicable._
- _For the **otel_lambda_layer**, use the value for **us-east-1** found [here](https://github.com/signalfx/lambda-layer-versions/blob/main/splunk-apm/splunk-apm.md)
- _The _**prefix**_ is a unique identifier you can choose for yourself, to make your resources distinct from other participants' resources. We suggest using a short form of your name, for example._
- _Also, please only lowercase letters for the **prefix**. Certain resouces in AWS, such as S3, would through an error if you use uppercase letters._
- _Also, please only lowercase letters for the **prefix**. Certain resources in AWS, such as S3, would through an error if you use uppercase letters._
- Save your file and exit the editor.
- Finally, copy the _**terraform.tfvars**_ file you just edited to the other directory.
```bash
cp ~/o11y-lambda-workshop/auto/terraform.tfvars ~/o11y-lambda-workshop/manual
cp ~/workshop/lambda/auto/terraform.tfvars ~/workshop/lambda/manual
```
- _We do this as we will be using the same values for both the autoinstrumentation and manual instrumentation protions of the workshop_

### File Permissions

While all other files are fine as they are, the **send_message.py** script in both the `auto` and `manual` will have to be executed as part of our workshop. As a result, it needs to have the appropriate permissions to run as expected. Follow these instructions to set them.

- First, ensure you are in the `o11y-lambda-workshop` directory:
- First, ensure you are in the `lambda` directory:
```bash
cd ~/o11y-lambda-workshop
cd ~/workshop/lambda
```

- Next, run the following command to set executable permissions on the `send_message.py` script:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,12 @@ weight: 2
The first part of our workshop will demonstrate how auto-instrumentation with OpenTelemetry allows the OpenTelemetry Collector to auto-detect what language your function is written in, and start capturing traces for those functions.

### The Auto-Instrumentation Workshop Directory & Contents
First, let us take a look at the `o11y-lambda-workshop/auto` directory, and some of its files. This is where all the content for the auto-instrumentation portion of our workshop resides.
First, let us take a look at the `workshop/lambda/auto` directory, and some of its files. This is where all the content for the auto-instrumentation portion of our workshop resides.

#### The `auto` Directory
- Run the following command to get into the **o11y-lambda-workshop/auto** directory:
- Run the following command to get into the **workshop/lambda/auto** directory:
```bash
cd ~/o11y-lambda-workshop/auto
cd ~/workshop/lambda/auto
```

- Inspect the contents of this directory:
Expand Down Expand Up @@ -114,11 +114,11 @@ In order to deploy the resources defined in the `main.tf` file, you first need t
```bash
pwd
```
- _The expected output would be **~/o11y-lambda-workshop/auto**_
- _The expected output would be **~/workshop/lambda/auto**_

- If you are not in the `auto` directory, run the following command:
```bash
cd ~/o11y-lambda-workshop/auto
cd ~/workshop/lambda/auto
```

- Run the following command to initialize Terraform in this directory
Expand Down Expand Up @@ -172,11 +172,11 @@ To start getting some traces from our deployed Lambda functions, we would need t
```bash
pwd
```
- _The expected output would be **~/o11y-lambda-workshop/auto**_
- _The expected output would be **~/workshop/lambda/auto**_

- If you are not in the `auto` directory, run the following command
```bash
cd ~/o11y-lambda-workshop/auto
cd ~/workshop/lambda/auto
```

The `send_message.py` script is a Python script that will take input at the command line, add it to a JSON dictionary, and send it to your `producer-lambda` function's endpoint repeatedly, as part of a while loop.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -86,12 +86,12 @@ Please follow these steps to destroy your resources:
pwd
```

- _The expected output would be **~/o11y-lambda-workshop/auto**_
- _The expected output would be **~/workshop/lambda/auto**_

- If you are not in the `auto` directory, run the following command:

```bash
cd ~/o11y-lambda-workshop/auto
cd ~/workshop/lambda/auto
```

- Destroy the Lambda functions and other AWS resources you deployed earlier:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,14 @@ The second part of our workshop will focus on demonstrating how manual instrumen

### The Manual Instrumentation Workshop Directory & Contents

Once again, we will first start by taking a look at our operating directory, and some of its files. This time, it will be `o11y-lambda-workshop/manual` directory. This is where all the content for the manual instrumentation portion of our workshop resides.
Once again, we will first start by taking a look at our operating directory, and some of its files. This time, it will be `workshop/lambda/manual` directory. This is where all the content for the manual instrumentation portion of our workshop resides.

#### The `manual` directory

- Run the following command to get into the `o11y-lambda-workshop/manual` directory:
- Run the following command to get into the `workshop/lambda/manual` directory:

```bash
cd ~/o11y-lambda-workshop/manual
cd ~/workshop/lambda/manual
```

- Inspect the contents of this directory with the `ls` command:
Expand All @@ -42,23 +42,23 @@ Let's make sure that all these files that LOOK the same, are actually the same.
- Compare the `main.tf` files in the `auto` and `manual` directories:

```bash
diff ~/o11y-lambda-workshop/auto/main.tf ~/o11y-lambda-workshop/manual/main.tf
diff ~/workshop/lambda/auto/main.tf ~/workshop/lambda/manual/main.tf
```

- There is no difference! _(Well, there shouldn't be. Ask your workshop facilitator to assist you if there is)_

- Now, let's compare the `producer.mjs` files:

```bash
diff ~/o11y-lambda-workshop/auto/handler/producer.mjs ~/o11y-lambda-workshop/manual/handler/producer.mjs
diff ~/workshop/lambda/auto/handler/producer.mjs ~/workshop/lambda/manual/handler/producer.mjs
```

- There's quite a few differences here!

- You may wish to view the entire file and examine its content

```bash
cat ~/o11y-lambda-workshop/handler/producer.mjs
cat ~/workshop/lambda/manual/handler/producer.mjs
```

- Notice how we are now importing some OpenTelemetry objects directly into our function to handle some of the manual instrumentation tasks we require.
Expand All @@ -75,7 +75,7 @@ Let's make sure that all these files that LOOK the same, are actually the same.
- Finally, compare the `consumer.mjs` files:

```bash
diff ~/o11y-lambda-workshop/auto/handler/consumer.mjs ~/o11y-lambda-workshop/manual/handler/consumer.mjs
diff ~/workshop/lambda/auto/handler/consumer.mjs ~/workshop/lambda/manual/handler/consumer.mjs
```

- Here also, there are a few differences of note. Let's take a closer look
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,12 @@ Seeing as we're in a new directory, we will need to initialize Terraform here on
pwd
```

- _The expected output would be **~/o11y-lambda-workshop/manual**_
- _The expected output would be **~/workshop/lambda/manual**_

- If you are not in the `manual` directory, run the following command:

```bash
cd ~/o11y-lambda-workshop/manual
cd ~/workshop/lambda/manual
```

- Run the following command to initialize Terraform in this directory
Expand Down Expand Up @@ -72,12 +72,12 @@ Once more, we will send our `name` and `superpower` as a message to our endpoint
pwd
```

- _The expected output would be **~/o11y-lambda-workshop/manual**_
- _The expected output would be **~/workshop/lambda/manual**_

- If you are not in the `manual` directory, run the following command:

```bash
cd ~/o11y-lambda-workshop/manual
cd ~/workshop/lambda/manual
```

- Run the `send_message.py` script as a background process:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -93,12 +93,12 @@ Please follow these steps to destroy your resources:
pwd
```

- _The expected output would be **~/o11y-lambda-workshop/manual**_
- _The expected output would be **~/workshop/lambda/manual**_

- If you are not in the `manual` directory, run the following command:

```bash
cd ~/o11y-lambda-workshop/manual
cd ~/workshop/lambda/manual
```

- Destroy the Lambda functions and other AWS resources you deployed earlier:
Expand Down
1 change: 1 addition & 0 deletions workshop/lambda/auto/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
!terraform.tfvars
23 changes: 23 additions & 0 deletions workshop/lambda/auto/handler/consumer.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
// Lambda Consumer App for Kinesis Stream

// Lambda Consumer
const consumer = async( event ) => {
try{
for( const record of event.Records ) {
const payload = record.kinesis;
const message = Buffer.from(payload.data, 'base64').toString();

console.log(
`Kinesis Message:
partition key: ${payload.partitionKey}
sequence number: ${payload.sequenceNumber}
kinesis schema version: ${payload.kinesisSchemaVersion}
data: ${message}`
);
}
} catch ( error ) {
console.log(error);
}
};

export { consumer };
47 changes: 47 additions & 0 deletions workshop/lambda/auto/handler/producer.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
// Lambda Producer App for Kinesis Stream
import { KinesisClient, PutRecordCommand } from "@aws-sdk/client-kinesis";


// Lambda Producer
const producer = async( event ) => {
const kinesis = new KinesisClient({});

let statusCode = 200;
let message;

if (!event.body) {
return {
statusCode: 400,
body: JSON.stringify({
message: "No message body found",
}),
};
}

const streamName = process.env.KINESIS_STREAM;

try {
await kinesis.send(
new PutRecordCommand({
StreamName: streamName,
PartitionKey: "1234",
Data: event.body,
}),

message = `Message placed in the Event Stream: ${streamName}`
)
} catch ( error ) {
console.log(error);
message = error;
statusCode = 500;
}

return {
statusCode,
body: JSON.stringify({
message,
}),
};
};

export { producer };
Loading