Skip to content

Commit

Permalink
Add CLI e2e test for transaction resource (#2757)
Browse files Browse the repository at this point in the history
* Adding example of transactions

* Adding run test for transactions

* Adding Apply test for Transactions

* Adding delete test

* Adding list test

* Fixing OpenAPI def that was causing our JSON marshaller to skip transaction id

* Adding Makefile commands into the same shell session to recognize env vars

* Fixing run transaction scenario

* Update transaction test
  • Loading branch information
danielbdias committed Jun 19, 2023
1 parent bbb642c commit cf4c251
Show file tree
Hide file tree
Showing 24 changed files with 804 additions and 140 deletions.
1 change: 0 additions & 1 deletion api/transactions.yaml
Expand Up @@ -29,7 +29,6 @@ components:
properties:
id:
type: string
readOnly: true
name:
type: string
description:
Expand Down
4 changes: 3 additions & 1 deletion cli/openapi/model_transaction.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 8 additions & 4 deletions testing/cli-e2etest/Makefile
Expand Up @@ -9,9 +9,13 @@ help: Makefile ## show list of commands
@awk 'BEGIN {FS = ":.*?## "} /[a-zA-Z_-]+:.*?## / {sub("\\\\n",sprintf("\n%22c"," "), $$2);printf "\033[36m%-40s\033[0m %s\n", $$1, $$2}' $(MAKEFILE_LIST) | sort

test: # run tests for this application
go clean -testcache
go test -timeout 300s -p 1 ./...
export TRACETEST_CLI=$(TRACETEST_CLI); \
export TEST_ENVIRONMENT=$(TEST_ENVIRONMENT); \
export TAG=$(TAG); \
export ENABLE_CLI_DEBUG=$(ENABLE_CLI_DEBUG); \
go clean -testcache; \
go test -timeout 300s -p 1 ./...;

test/debug: # run tests for this application with debug mode enabled
ENABLE_CLI_DEBUG="true"
make test
export ENABLE_CLI_DEBUG="true"; \
make test;
34 changes: 17 additions & 17 deletions testing/cli-e2etest/README.md
Expand Up @@ -31,10 +31,10 @@ The main idea is to test every CLI command against the Tracetest server with dif
| CLI Command | Test scenarios |
| ------------------------------------------------------------------ | -------------- |
| `test list` | |
| `test run -d [test-definition]` | [TestRunTestWithGrpcTrigger](./testscenarios/test/run_test_with_grpc_trigger_test.go) |
| `test run -d [test-definition] -e [environment-id]` | [TestRunTestWithHttpTriggerAndEnvironmentFile](./testscenarios/test/run_test_with_http_trigger_and_environment_file_test.go) |
| `test run -d [test-definition] -e [environment-definition]` | [TestRunTestWithHttpTriggerAndEnvironmentFile](./testscenarios/test/run_test_with_http_trigger_and_environment_file_test.go) |
| `test run -d [transaction-definition]` | |
| `test run -d [test-definition]` | [RunTestWithGrpcTrigger](./testscenarios/test/run_test_with_grpc_trigger_test.go) |
| `test run -d [test-definition] -e [environment-id]` | [RunTestWithHttpTriggerAndEnvironmentFile](./testscenarios/test/run_test_with_http_trigger_and_environment_file_test.go) |
| `test run -d [test-definition] -e [environment-definition]` | [RunTestWithHttpTriggerAndEnvironmentFile](./testscenarios/test/run_test_with_http_trigger_and_environment_file_test.go) |
| `test run -d [transaction-definition]` | [RunTransaction](./testscenarios/transaction//run_transaction_test.go) |
| `test run -d [transaction-definition] -e [environment-id]` | |
| `test run -d [transaction-definition] -e [environment-definition]` | |

Expand Down Expand Up @@ -117,19 +117,19 @@ The main idea is to test every CLI command against the Tracetest server with dif

| CLI Command | Test scenarios |
| ----------------------------------------------------------- | -------------- |
| `apply transaction -f [new-transaction-file]` | |
| `apply transaction -f [existing-transaction-file]` | |
| `delete transaction --id [existing-id]` | |
| `delete transaction --id [non-existing-id]` | |
| `get transaction --id [non-existing-id]` | |
| `get transaction --id [existing-id] --output pretty` | |
| `get transaction --id [existing-id] --output json` | |
| `get transaction --id [existing-id] --output yaml` | |
| `list transaction --output pretty` | |
| `list transaction --output json` | |
| `list transaction --output yaml` | |
| `list transaction --skip 1 --take 2` | |
| `list transaction --sortBy name --sortDirection asc` | |
| `apply transaction -f [new-transaction-file]` | [ApplyTransaction](./testscenarios/transaction/apply_transaction_test.go) |
| `apply transaction -f [existing-transaction-file]` | [ApplyTransaction](./testscenarios/transaction/apply_transaction_test.go) |
| `delete transaction --id [existing-id]` | [DeleteTransaction](./testscenarios/transaction/delete_transaction_test.go) |
| `delete transaction --id [non-existing-id]` | [DeleteTransaction](./testscenarios/transaction/delete_transaction_test.go) |
| `get transaction --id [non-existing-id]` | [GetTransaction](./testscenarios/transaction/get_transaction_test.go), [DeleteTransaction](./testscenarios/transaction/delete_transaction_test.go) |
| `get transaction --id [existing-id] --output pretty` | [GetTransaction](./testscenarios/transaction/get_transaction_test.go) |
| `get transaction --id [existing-id] --output json` | [GetTransaction](./testscenarios/transaction/get_transaction_test.go) |
| `get transaction --id [existing-id] --output yaml` | [GetTransaction](./testscenarios/transaction/get_transaction_test.go) |
| `list transaction --output pretty` | [ListTransaction](./testscenarios/transaction/list_transactions_test.go) |
| `list transaction --output json` | [ListTransaction](./testscenarios/transaction/list_transactions_test.go) |
| `list transaction --output yaml` | [ListTransaction](./testscenarios/transaction/list_transactions_test.go) |
| `list transaction --skip 1 --take 2` | [ListTransaction](./testscenarios/transaction/list_transactions_test.go) |
| `list transaction --sortBy name --sortDirection asc` | [ListTransaction](./testscenarios/transaction/list_transactions_test.go) |

### Resources: Tests

Expand Down
@@ -0,0 +1,88 @@
package transaction

import (
"fmt"
"testing"

"github.com/kubeshop/tracetest/cli-e2etest/environment"
"github.com/kubeshop/tracetest/cli-e2etest/helpers"
"github.com/kubeshop/tracetest/cli-e2etest/testscenarios/types"
"github.com/kubeshop/tracetest/cli-e2etest/tracetestcli"
"github.com/stretchr/testify/require"
)

func TestApplyTransaction(t *testing.T) {
// instantiate require with testing helper
require := require.New(t)

// setup isolated e2e environment
env := environment.CreateAndStart(t)
defer env.Close(t)

cliConfig := env.GetCLIConfigPath(t)

// Given I am a Tracetest CLI user
// And I have my server recently created

// When I try to set up a new transaction
// Then it should be applied with success
newTransactionPath := env.GetTestResourcePath(t, "new-transaction")

result := tracetestcli.Exec(t, fmt.Sprintf("apply transaction --file %s", newTransactionPath), tracetestcli.WithCLIConfig(cliConfig))
helpers.RequireExitCodeEqual(t, result, 0)

transaction := helpers.UnmarshalYAML[types.TransactionResource](t, result.StdOut)

require.Equal("Transaction", transaction.Type)
require.Equal("Qti5R3_VR", transaction.Spec.ID)
require.Equal("New Transaction", transaction.Spec.Name)
require.Equal("a transaction", transaction.Spec.Description)
require.Len(transaction.Spec.Steps, 2)
require.Equal("9wtAH2_Vg", transaction.Spec.Steps[0])
require.Equal("ajksdkasjbd", transaction.Spec.Steps[1])

// When I try to get the transaction applied on the last step
// Then it should return it
result = tracetestcli.Exec(t, "get transaction --id Qti5R3_VR --output yaml", tracetestcli.WithCLIConfig(cliConfig))
helpers.RequireExitCodeEqual(t, result, 0)

require.Equal("Transaction", transaction.Type)
require.Equal("Qti5R3_VR", transaction.Spec.ID)
require.Equal("New Transaction", transaction.Spec.Name)
require.Equal("a transaction", transaction.Spec.Description)
require.Len(transaction.Spec.Steps, 2)
require.Equal("9wtAH2_Vg", transaction.Spec.Steps[0])
require.Equal("ajksdkasjbd", transaction.Spec.Steps[1])

// When I try to update the last transaction
// Then it should be applied with success
updatedNewTransactionPath := env.GetTestResourcePath(t, "updated-new-transaction")

result = tracetestcli.Exec(t, fmt.Sprintf("apply transaction --file %s", updatedNewTransactionPath), tracetestcli.WithCLIConfig(cliConfig))
helpers.RequireExitCodeEqual(t, result, 0)

updatedTransaction := helpers.UnmarshalYAML[types.TransactionResource](t, result.StdOut)
require.Equal("Transaction", updatedTransaction.Type)
require.Equal("Qti5R3_VR", updatedTransaction.Spec.ID)
require.Equal("Updated Transaction", updatedTransaction.Spec.Name)
require.Equal("an updated transaction", updatedTransaction.Spec.Description)
require.Len(updatedTransaction.Spec.Steps, 3)
require.Equal("9wtAH2_Vg", updatedTransaction.Spec.Steps[0])
require.Equal("ajksdkasjbd", updatedTransaction.Spec.Steps[1])
require.Equal("ajksdkasjbd", updatedTransaction.Spec.Steps[2])

// When I try to get the transaction applied on the last step
// Then it should return it
result = tracetestcli.Exec(t, "get transaction --id Qti5R3_VR --output yaml", tracetestcli.WithCLIConfig(cliConfig))
helpers.RequireExitCodeEqual(t, result, 0)

updatedTransaction = helpers.UnmarshalYAML[types.TransactionResource](t, result.StdOut)
require.Equal("Transaction", updatedTransaction.Type)
require.Equal("Qti5R3_VR", updatedTransaction.Spec.ID)
require.Equal("Updated Transaction", updatedTransaction.Spec.Name)
require.Equal("an updated transaction", updatedTransaction.Spec.Description)
require.Len(updatedTransaction.Spec.Steps, 3)
require.Equal("9wtAH2_Vg", updatedTransaction.Spec.Steps[0])
require.Equal("ajksdkasjbd", updatedTransaction.Spec.Steps[1])
require.Equal("ajksdkasjbd", updatedTransaction.Spec.Steps[2])
}
@@ -0,0 +1,50 @@
package transaction

import (
"fmt"
"testing"

"github.com/kubeshop/tracetest/cli-e2etest/environment"
"github.com/kubeshop/tracetest/cli-e2etest/helpers"
"github.com/kubeshop/tracetest/cli-e2etest/tracetestcli"
"github.com/stretchr/testify/require"
)

func TestDeleteTransaction(t *testing.T) {
// instantiate require with testing helper
require := require.New(t)

// setup isolated e2e environment
env := environment.CreateAndStart(t)
defer env.Close(t)

cliConfig := env.GetCLIConfigPath(t)

// Given I am a Tracetest CLI user
// And I have my server recently created

// When I try to delete a transaction that don't exist
// Then it should return an error and say that this resource does not exist
result := tracetestcli.Exec(t, "delete transaction --id dont-exist", tracetestcli.WithCLIConfig(cliConfig))
helpers.RequireExitCodeEqual(t, result, 1)
require.Contains(result.StdErr, "Resource transactions with ID dont-exist not found") // TODO: update this message to singular

// When I try to set up a new transaction
// Then it should be applied with success
newTransactionPath := env.GetTestResourcePath(t, "new-transaction")

result = tracetestcli.Exec(t, fmt.Sprintf("apply transaction --file %s", newTransactionPath), tracetestcli.WithCLIConfig(cliConfig))
helpers.RequireExitCodeEqual(t, result, 0)

// When I try to delete the transaction
// Then it should delete with success
result = tracetestcli.Exec(t, "delete transaction --id Qti5R3_VR", tracetestcli.WithCLIConfig(cliConfig))
helpers.RequireExitCodeEqual(t, result, 0)
require.Contains(result.StdOut, "✔ Transaction successfully deleted")

// When I try to get a transaction again
// Then it should return a message saying that the transaction was not found
result = tracetestcli.Exec(t, "delete transaction --id Qti5R3_VR", tracetestcli.WithCLIConfig(cliConfig))
helpers.RequireExitCodeEqual(t, result, 1)
require.Contains(result.StdErr, "Resource transactions with ID Qti5R3_VR not found")
}
108 changes: 108 additions & 0 deletions testing/cli-e2etest/testscenarios/transaction/get_transaction_test.go
@@ -0,0 +1,108 @@
package transaction

import (
"fmt"
"strings"
"testing"

"github.com/kubeshop/tracetest/cli-e2etest/environment"
"github.com/kubeshop/tracetest/cli-e2etest/helpers"
"github.com/kubeshop/tracetest/cli-e2etest/testscenarios/types"
"github.com/kubeshop/tracetest/cli-e2etest/tracetestcli"
"github.com/stretchr/testify/require"
)

func addGetTransactionPreReqs(t *testing.T, env environment.Manager) {
cliConfig := env.GetCLIConfigPath(t)

// Given I am a Tracetest CLI user
// And I have my server recently created

// When I try to set up a new transaction
// Then it should be applied with success
newTransactionPath := env.GetTestResourcePath(t, "new-transaction")

result := tracetestcli.Exec(t, fmt.Sprintf("apply transaction --file %s", newTransactionPath), tracetestcli.WithCLIConfig(cliConfig))
helpers.RequireExitCodeEqual(t, result, 0)
}

func TestGetTransaction(t *testing.T) {
// instantiate require with testing helper
require := require.New(t)

env := environment.CreateAndStart(t)
defer env.Close(t)

cliConfig := env.GetCLIConfigPath(t)

t.Run("get with no transaction initialized", func(t *testing.T) {
// Given I am a Tracetest CLI user
// And I have my server recently created
// And no transaction registered

// When I try to get a transaction on yaml mode
// Then it should return a error message
result := tracetestcli.Exec(t, "get transaction --id no-id --output yaml", tracetestcli.WithCLIConfig(cliConfig))
helpers.RequireExitCodeEqual(t, result, 0)
require.Contains(result.StdOut, "Resource transaction with ID no-id not found")
})

addGetTransactionPreReqs(t, env)

t.Run("get with YAML format", func(t *testing.T) {
// Given I am a Tracetest CLI user
// And I have my server recently created
// And I have a transaction already set

// When I try to get a transaction on yaml mode
// Then it should print a YAML
result := tracetestcli.Exec(t, "get transaction --id Qti5R3_VR --output yaml", tracetestcli.WithCLIConfig(cliConfig))
helpers.RequireExitCodeEqual(t, result, 0)

transaction := helpers.UnmarshalYAML[types.TransactionResource](t, result.StdOut)

require.Equal("Transaction", transaction.Type)
require.Equal("Qti5R3_VR", transaction.Spec.ID)
require.Equal("New Transaction", transaction.Spec.Name)
require.Equal("a transaction", transaction.Spec.Description)
require.Len(transaction.Spec.Steps, 2)
require.Equal("9wtAH2_Vg", transaction.Spec.Steps[0])
require.Equal("ajksdkasjbd", transaction.Spec.Steps[1])
})

t.Run("get with JSON format", func(t *testing.T) {
// Given I am a Tracetest CLI user
// And I have my server recently created
// And I have a transaction already set

// When I try to get a transaction on json mode
// Then it should print a json
result := tracetestcli.Exec(t, "get transaction --id Qti5R3_VR --output json", tracetestcli.WithCLIConfig(cliConfig))
helpers.RequireExitCodeEqual(t, result, 0)

transaction := helpers.UnmarshalJSON[types.TransactionResource](t, result.StdOut)

require.Equal("Transaction", transaction.Type)
require.Equal("Qti5R3_VR", transaction.Spec.ID)
require.Equal("New Transaction", transaction.Spec.Name)
require.Equal("a transaction", transaction.Spec.Description)
require.Len(transaction.Spec.Steps, 2)
require.Equal("9wtAH2_Vg", transaction.Spec.Steps[0])
require.Equal("ajksdkasjbd", transaction.Spec.Steps[1])
})

t.Run("get with pretty format", func(t *testing.T) {
// Given I am a Tracetest CLI user
// And I have my server recently created
// And I have a transaction already set

// When I try to get a transaction on pretty mode
// Then it should print a table with 4 lines printed: header, separator, transaction item and empty line
result := tracetestcli.Exec(t, "get transaction --id Qti5R3_VR --output pretty", tracetestcli.WithCLIConfig(cliConfig))
helpers.RequireExitCodeEqual(t, result, 0)
require.Contains(result.StdOut, "New Transaction")

lines := strings.Split(result.StdOut, "\n")
require.Len(lines, 4)
})
}

0 comments on commit cf4c251

Please sign in to comment.