-
Notifications
You must be signed in to change notification settings - Fork 0
kuka-serverless tests failing #165
Comments
Still fixing this. Builds still failing. But basically got everything to work. I'm just deploying straight to Lambda to run CI tests. Some tests failed, because the table already existed and had some entries. Gotta make it |
Can only add one GSI to cloudformation: https://cloudkatha.com/solved-cannot-perform-more-than-one-gsi-creation-or-deletion-in-a-single-update/ |
Solution: |
https://stackoverflow.com/questions/36918408/unable-to-add-gsi-to-dynamodb-table-using-cloudformation Do it programatically: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/getting-started-step-6.html |
Thinking of something like this to create the table and first GSI: resources: # CloudFormation template syntax
Resources:
usersTable:
Type: AWS::DynamoDB::Table
Properties:
TableName: ${env:TABLE_NAME}-${env:STAGE}
AttributeDefinitions:
- AttributeName: "PK"
AttributeType: S
- AttributeName: "SK"
AttributeType: S
KeySchema:
- AttributeName: "PK"
KeyType: "HASH"
- AttributeName: "SK"
KeyType: "RANGE"
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
GlobalSecondaryIndexes:
IndexName: "sk-pk-index"
KeySchema:
- AttributeName: "SK"
KeyType: "HASH"
- AttributeName: "PK"
KeyType: "RANGE"
Projection:
ProjectionType: "ALL" |
Gotta still make script to add second GSI and delete table after tests (even if tests fail). |
Table deletion could look something like this:
const { DynamoDB } = require("@aws-sdk/client-dynamodb")
;(async () => {
const client = new DynamoDB({ region: process.env.REGION })
const params = {
TableName: process.env.TABLE_NAME + "-" + process.env.STAGE,
}
await client.deleteTable(params)
})() |
To add GSI to existing table: const { DynamoDB } = require("@aws-sdk/client-dynamodb")
;(async () => {
const client = new DynamoDB({ region: process.env.REGION })
const params = {
TableName: process.env.TABLE_NAME + "-" + process.env.STAGE,
GlobalSecondaryIndexUpdates: {
Create: {
IndexName: "email-pk-index",
KeySchema: [
{ AttributeName: "email", KeyType: "HASH" },
{ AttributeName: "pk", KeyType: "RANGE" },
],
Projection: "ALL",
ProvisionedThroughput: { ReadCapacityUnits: 1, WriteCapacityUnits: 1 },
},
},
}
await client.updateTable(params)
})() |
my #!/usr/bin/env bash
set -e
npm config set @kuka-js:registry http://registry.npmjs.org
npm config set //registry.npmjs.org/:_authToken $NPM_TOKEN
npm whoami
npm ci --also=dev
./node_modules/serverless/bin/serverless.js config credentials --provider aws --key $AWS_KEY --secret $AWS_SECRET
./node_modules/serverless/bin/serverless.js deploy
# Need to create new GSI, because during deployment only one GSI can be added. https://github.com/kuka-js/kuka/issues/165
./scripts/create-GSI.js
CI_URL="https://$(node ./get-api-id.js).execute-api.eu-north-1.amazonaws.com/ci/"
sed -i -e "s|URL_PREFIX|$CI_URL|g" ./.newman/postman_environment.json
npm test
./scripts/delete-table.js
# Gotta disable semantic release because I'm trying to fix CICD currently
#npm run semantic-release Still gotta add endpoint deletion there though (deletion of CI related apigw and lambda, and maybe something else) |
finally managed to programatically create a GSI to an existing table const {
DynamoDBClient,
UpdateTableCommand,
} = require("@aws-sdk/client-dynamodb")
;(async () => {
const config = {
region: process.env.REGION,
credentials: {
accessKeyId: process.env.AWS_KEY,
secretAccessKey: process.env.AWS_SECRET,
},
}
console.log(config)
const client = new DynamoDBClient(config)
const params = {
TableName: process.env.TABLE_NAME + "-" + process.env.STAGE,
AttributeDefinitions: [
{ AttributeName: "email", AttributeType: "S" },
{ AttributeName: "pk", AttributeType: "S" },
],
GlobalSecondaryIndexUpdates: [
{
Create: {
IndexName: "email-pk-index",
KeySchema: [
{ AttributeName: "email", KeyType: "HASH" },
{ AttributeName: "pk", KeyType: "RANGE" },
],
Projection: { ProjectionType: "ALL" },
ProvisionedThroughput: {
ReadCapacityUnits: 1,
WriteCapacityUnits: 1,
},
},
},
],
}
const command = new UpdateTableCommand(params)
try {
await client.send(command)
} catch (e) {
console.error(e)
}
console.log(JSON.stringify(command))
})() |
However creating table is not instantaneous. Script ends but GSI not created. So we need to poll the creation and when it is done continue with the CI/CD pipeline. |
In fact why do we even have this |
oh my god, after so much work finally I think I'm almost done... finally |
Holy crap. Finally done. https://github.com/kuka-js/kuka/runs/3269743728?check_suite_focus=true There is very ugly git history now. But I do not care. I'm not gonna start commit squashing and rebasing, because then I might have to do "release not found release branch after git push --force" fix described here all over again. And there is no way in hell I'm doing that again. Ticket closed. Finally. Did I say finally often enough? |
They used to work with spinning up a serverless-offline in the CI. Flat out not working anymore. Tried to fix it in a couple of ways already.
This does not seem like stable solution. I might genuinely just deploy to AWS Lambda dev on CI and then test against that dev.
There have been some articles making arguments for testing against actual AWS Lambda and not serverless-offline.
The text was updated successfully, but these errors were encountered: