Skip to content

Commit

Permalink
Rebase to original repo (#1)
Browse files Browse the repository at this point in the history 
* Create codeql-analysis.yml (DataDog#100)

* Create codeql-analysis.yml

* Update codeql-analysis.yml

* Update run_integration_tests.sh

* Do not show error messages even if neither DD_API_KEY nor DD_KMS_API_KEY is set when Lambda Extension is running (DataDog#102)

* Bump version to 1.4.0

* Bump go + fasthttp + lint (DataDog#104)

* Consolidate serverless configurations into one place (DataDog#105)

* Update README.md

* Update README.md

* Bump dd-trace-go to latest version to address some vulnerabilities (DataDog#109)

* Bump dd-trace-go to latest version to address some vulnaribilities
* update go.sum with `go mod tidy`

* Bump version to 1.6.0

* bump codeql (DataDog#112)

* Bump dd-trace-go to v1.41 (DataDog#115)

* Bump version to 1.7.0

* [SLS-2330] Add support for universal instrumentation with the extension (DataDog#116)

add option to use universal instrumentation

* [EEP-444] include error in failed metric send log (DataDog#118)

Co-authored-by: Corey Griffin <CoreyGriffin@users.noreply.github.com>

* [SLS-2492] Upgrade aws sdk v2 (DataDog#113)

upgrade sdk

* Bump version to 1.8.0

* Use new account in integration tests (DataDog#119)

* set the architecture explicitely (DataDog#122)

* mask init runtime logs (DataDog#123)

* Update libs (DataDog#121)

* bump go 1.18 (DataDog#125)

* Retry sending trace payloads on failure. (DataDog#128)

* Bump version to 1.9.0

* Update DD Trace to  v1.51.0(DataDog#133)

* Bump go version to 1.20 (DataDog#140)

Bump go version to 1.20

* Upgrade version of dd-trace-go to v1.54.1 (DataDog#141)

* Bump version to 1.10.0

* Propagate trace context from SQS events (DataDog#142)

* Default parent id to be trace id if not found elsewhere.

* Look for trace context in context object as well as headers.

* Apply trace context before starting the function execution span.

* Update signature in tests.

* Add spanid of execution span to context.

* Do not ignore priority "-128".

* Test that default parent id set to trace id.

* Test span id added to context.

* Test uses trace context from context object.

* Bump version to 1.11.0

* feat: automate AppSec enablement setup (e.g: `AWS_LAMBDA_RUNTIME_API`) (DataDog#143)

* feat: honor AWS_LAMBDA_EXEC_WRAPPER when AWS Lambda does not

In order to simplify onboarding & make it more uniform across languages,
inspect the value of the `AWS_LAMBDA_EXEC_WRAPPER` environment variable
and apply select environment variable changes it perofrms upon
decorating a handler.

This is necessary/useful because that environment variable is not
honored by custom runtimes (`provided`, `provided.al2`) as well as the
`go1.x` runtime (which is a glorified provided runtime). The datadog
Lambda wrapper starts a proxy to inject ASM functionality directly on
the Lambda runtime API instead of having to manually instrument each and
every lambda handler/application, and modifies `AWS_LAMBDA_RUNTIME_API`
to instruct Lambda language runtime client libraries to go through it
instead of directly interacting with the Lambda control plane.

APPSEC-11534

* pivot to a different, cheaper strategy

* typo fix

* PR feedback

* minor fixups

* add warning in go1.x runtime if lambda.norpc build tag was not enabled

* Bump version to 1.12.0

* Re-add configs after upstream rebase

* Bump packages

* Remove deprecated `io/ioutil` calls

---------

Co-authored-by: Tian Chu <tian.chu@datadoghq.com>
Co-authored-by: Soshi Katsuta <skatsuta@users.noreply.github.com>
Co-authored-by: Maxime David <maxime.david@datadoghq.com>
Co-authored-by: kimi <47579703+kimi-p@users.noreply.github.com>
Co-authored-by: Kimi Wu <kimi.wu@datadoghq.com>
Co-authored-by: Dylan Yang <dylan.yang@datadoghq.com>
Co-authored-by: Corey Griffin <15809365+CoreyGriffin@users.noreply.github.com>
Co-authored-by: Corey Griffin <CoreyGriffin@users.noreply.github.com>
Co-authored-by: Marcin Rabenda <xrn.design@gmail.com>
Co-authored-by: Rey Abolofia <purple4reina@gmail.com>
Co-authored-by: Rey Abolofia <rey.abolofia@datadoghq.com>
Co-authored-by: Andrew Rodriguez <49878080+zARODz11z@users.noreply.github.com>
Co-authored-by: Ivan Topolcic <IvanTopolcic@users.noreply.github.com>
Co-authored-by: Romain Marcadier <romain.muller@telecomnancy.net>
  • Loading branch information
@peterdeme @tianchu
@skatsuta @maxday @kimi-p @DylanLovesCoffee @CoreyGriffin @xrn @purple4reina @zARODz11z @IvanTopolcic @RomainMuller
15 people committed Dec 4, 2023
1 parent 0e69246 commit 52bd050
Show file tree
Hide file tree
Showing 38 changed files with 1,909 additions and 614 deletions.
104 changes: 49 additions & 55 deletions .github/workflows/build.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,74 +8,68 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v4

- name: Set up Go
uses: actions/setup-go@v2
uses: actions/setup-go@v4
with:
go-version: 1.13

- name: Install Golint
run: go install golang.org/x/lint/golint

- name: Lint
run: golint
go-version: '1.21'

- name: golangci-lint
uses: golangci/golangci-lint-action@v3

unit-test:
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v4

- name: Set up Go
uses: actions/setup-go@v1
uses: actions/setup-go@v4
with:
go-version: 1.13
go-version: '1.21'

- name: Run tests
run: go test -race -coverprofile=coverage.txt -covermode=atomic ./...

- name: Upload code coverage report
run: bash <(curl -s https://codecov.io/bash)

integration-test:
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v2

- name: Set up Node 14
uses: actions/setup-node@v1
with:
node-version: 14

- name: Set up Go
uses: actions/setup-go@v1
with:
go-version: 1.13

- name: Cache Node modules
id: cache-node-modules
uses: actions/cache@v2
with:
path: "**/node_modules"
key: ${{ runner.os }}-modules-${{ hashFiles('**/yarn.lock') }}

- name: Install Serverless Framework
run: sudo yarn global add serverless --prefix /usr/local

- name: Install dependencies
if: steps.cache-node-modules.outputs.cache-hit != 'true'
working-directory: tests/integration_tests/
run: yarn install

- name: Run tests
env:
BUILD_LAYERS: true
DD_API_KEY: ${{ secrets.DD_API_KEY }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
working-directory: tests/integration_tests/
run: ./run_integration_tests.sh
# integration-test:
# runs-on: ubuntu-latest

# steps:
# - name: Checkout
# uses: actions/checkout@v4

# - name: Set up Node 14
# uses: actions/setup-node@v3
# with:
# node-version: 14

# - name: Set up Go
# uses: actions/setup-go@v4
# with:
# go-version: '1.21'

# - name: Cache Node modules
# id: cache-node-modules
# uses: actions/cache@v3
# with:
# path: "**/node_modules"
# key: ${{ runner.os }}-modules-${{ hashFiles('**/yarn.lock') }}

# - name: Install Serverless Framework
# run: sudo yarn global add serverless --prefix /usr/local

# - name: Install dependencies
# if: steps.cache-node-modules.outputs.cache-hit != 'true'
# working-directory: tests/integration_tests/
# run: yarn install

# - name: Run tests
# env:
# BUILD_LAYERS: true
# DD_API_KEY: ${{ secrets.DD_API_KEY }}
# AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
# AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
# working-directory: tests/integration_tests/
# run: ./run_integration_tests.sh
70 changes: 70 additions & 0 deletions .github/workflows/codeql-analysis.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL"

on:
push:
branches: [ main, master ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ main ]
schedule:
- cron: '33 8 * * 1'

jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write

strategy:
fail-fast: false
matrix:
language: [ 'go' ]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Learn more about CodeQL language support at https://git.io/codeql-language-support

steps:
- name: Checkout repository
uses: actions/checkout@v4

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# queries: ./path/to/local/query, your-org/your-repo/queries@main

# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v2

# ℹ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl

# ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
# and modify them (or add more) to build your code if your project
# uses a compiled language

#- run: |
# make bootstrap
# make release

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
99 changes: 3 additions & 96 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,104 +10,11 @@ Datadog Lambda Library for Go enables enhanced Lambda metrics, distributed traci

## Installation

Follow the installation instructions [here](https://docs.datadoghq.com/serverless/installation/go/).
Follow the [installation instructions](https://docs.datadoghq.com/serverless/installation/go/), and view your function's enhanced metrics, traces and logs in Datadog.

## Enhanced Metrics
## Configurations

Once [installed](#installation), you should be able to view enhanced metrics for your Lambda function in Datadog.

Check out the official documentation on [Datadog Lambda enhanced metrics](https://docs.datadoghq.com/integrations/amazon_lambda/?tab=go#real-time-enhanced-lambda-metrics).

## Custom Metrics

Once [installed](#installation), you should be able to submit custom metrics from your Lambda function.

Check out the instructions for [submitting custom metrics from AWS Lambda functions](https://docs.datadoghq.com/integrations/amazon_lambda/?tab=go#custom-metrics).

## Tracing

Set the `DD_TRACE_ENABLED` environment variable to `true` to enable Datadog tracing. When Datadog tracing is enabled, the library will inject a span representing the Lambda's execution into the context object. You can then use the included `dd-trace-go` package to create additional spans from the context or pass the context to other services. For more information, see the [dd-trace-go documentation](https://godoc.org/gopkg.in/DataDog/dd-trace-go.v1/ddtrace).

```go
import (
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"
httptrace "gopkg.in/DataDog/dd-trace-go.v1/contrib/net/http"
)

func handleRequest(ctx context.Context, ev events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
// Trace an HTTP request
req, _ := http.NewRequestWithContext(ctx, "GET", "https://www.datadoghq.com", nil)
client := http.Client{}
client = *httptrace.WrapClient(&client)
client.Do(req)

// Create a custom span
s, _ := tracer.StartSpanFromContext(ctx, "child.span")
time.Sleep(100 * time.Millisecond)
s.Finish()
}
```

You can also use the injected span to [connect your logs and traces](https://docs.datadoghq.com/tracing/connect_logs_and_traces/go/).

```go
func handleRequest(ctx context.Context, ev events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
currentSpan, _ := tracer.SpanFromContext(ctx)
log.Printf("my log message %v", currentSpan)
}
```

If you are also using AWS X-Ray to trace your Lambda functions, you can set the `DD_MERGE_XRAY_TRACES` environment variable to `true`, and Datadog will merge your Datadog and X-Ray traces into a single, unified trace.

### Trace Context Extraction

To link your distributed traces, datadog-lambda-go looks for the `x-datadog-trace-id`, `x-datadog-parent-id` and `x-datadog-sampling-priority` trace `headers` in the Lambda event payload.
If the headers are found it will set the parent trace to the trace context extracted from the headers.

It is possible to configure your own trace context extractor function if the default extractor does not support your event.

```go
myExtractorFunc := func(ctx context.Context, ev json.RawMessage) map[string]string {
// extract x-datadog-trace-id, x-datadog-parent-id and x-datadog-sampling-priority.
}

cfg := &ddlambda.Config{
TraceContextExtractor: myExtractorFunc,
}
ddlambda.WrapFunction(handler, cfg)
```

A more complete example can be found in the `ddlambda_example_test.go` file.

## Environment Variables

### DD_FLUSH_TO_LOG

Set to `true` (recommended) to send custom metrics asynchronously (with no added latency to your Lambda function executions) through CloudWatch Logs with the help of [Datadog Forwarder](https://github.com/DataDog/datadog-serverless-functions/tree/master/aws/logs_monitoring). Defaults to `false`. If set to `false`, you also need to set `DD_API_KEY` and `DD_SITE`.

### DD_API_KEY

If `DD_FLUSH_TO_LOG` is set to `false` (not recommended), the Datadog API Key must be defined.

### DD_SITE

If `DD_FLUSH_TO_LOG` is set to `false` (not recommended), you must set `DD_SITE`. Possible values are `datadoghq.com`, `datadoghq.eu`, `us3.datadoghq.com`, `us5.datadoghq.com`, and `ddog-gov.com`. The default is `datadoghq.com`.

### DD_LOG_LEVEL

Set to `debug` enable debug logs from the Datadog Lambda Library. Defaults to `info`.

### DD_ENHANCED_METRICS

Generate enhanced Datadog Lambda integration metrics, such as, `aws.lambda.enhanced.invocations` and `aws.lambda.enhanced.errors`. Defaults to `true`.

### DD_TRACE_ENABLED

Initialize the Datadog tracer when set to `true`. Defaults to `false`.

### DD_MERGE_XRAY_TRACES

If you are using both X-Ray and Datadog tracing, set this to `true` to merge the X-Ray and Datadog traces. Defaults to `false`.
See the [advanced configuration options](https://docs.datadoghq.com/serverless/configuration) to tag your telemetry, capture request/response payloads, filter or scrub sensitive information from logs or traces, and more.

## Opening Issues

Expand Down
14 changes: 14 additions & 0 deletions awslambdanorpc.go
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
//go:build lambda.norpc
// +build lambda.norpc

/*
* Unless explicitly stated otherwise all files in this repository are licensed
* under the Apache License Version 2.0.
*
* This product includes software developed at Datadog (https://www.datadoghq.com/).
* Copyright 2021 Datadog, Inc.
*/

package ddlambda

const awsLambdaRpcSupport = false
14 changes: 14 additions & 0 deletions awslambdawithrpc.go
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
//go:build !lambda.norpc
// +build !lambda.norpc

/*
* Unless explicitly stated otherwise all files in this repository are licensed
* under the Apache License Version 2.0.
*
* This product includes software developed at Datadog (https://www.datadoghq.com/).
* Copyright 2021 Datadog, Inc.
*/

package ddlambda

const awsLambdaRpcSupport = true
Loading

0 comments on commit 52bd050

Please sign in to comment.