test: comprehensive E2E test suite#158
Merged
Merged
Conversation
Add tests/test_e2e.py with 9 tests covering 23 progressive steps:
- Project CRUD (add/list/status/edit/remove)
- Storage CRUD (bucket/table create, upload CSV, download round-trip, unload, delete)
- Config operations (list/detail/search/update --set/--dry-run/--configuration/delete)
- Component discovery (list with --type filter, detail via AI Service)
- File lifecycle (upload/list/detail/download/tag/delete with dry-run)
- Encrypt values (KBC::ProjectSecure verification)
- Branch lifecycle (list/create/use/reset/delete)
- Sharing list + lineage show (read-only)
- Error handling (invalid token, nonexistent resources, correct exit codes)
- JSON consistency (all read commands return valid JSON, token never leaked)
Verbose output shows exact CLI commands and abbreviated JSON responses.
Resources use e2e-{timestamp} prefix with guaranteed cleanup on failure.
Also: add make test-e2e target, register @pytest.mark.e2e, fix help regex
to support digits in target names, exclude e2e from default make test.
Expand TestFullE2E from 23 to 36 progressive steps: - Add init command test (create .kbagent/ in sub-directory) - Add storage upload-table --incremental (verify 8 rows after appending 3) - Add storage load-file (upload CSV as file, load into table) - Add config update --merge (deep-merge partial JSON, verify preservation) - Add config new scaffold generation (keboola.ex-http) - Add full workspace lifecycle (create/list/detail/password/load/query/delete) - Add branch merge command (verify URL response, then delete) - Add permissions list/show/check commands - Add job list --component-id filter and job detail Add TestE2ESyncWorkflow: sync init/pull/status/diff/push --dry-run in temp git repo Add TestE2EToolCommands: tool list + tool call get_buckets (skip if no MCP) Update CLAUDE.md: add convention #16 requiring E2E tests for new commands, add test_e2e.py to project structure. Total: 12 tests, ~2.5 min runtime against real API.
Documents all 12 E2E tests (36 main steps) with exact commands, assertions, and coverage. Includes table of commands NOT covered with reasons (manage token, interactive prompts, side effects).
Full transformation lifecycle: - Create output bucket + Snowflake SQL transformation config via API - SQL: SELECT id, name, CAST(value AS INT), CAST(value AS INT)*2 AS doubled_value - Run transformation with job run --wait --timeout 300 - Verify job detail: status=success, isFinished=true - Download output table, verify doubled_value == value * 2 for all rows - Clean up transformation config and output bucket via CLI Brings main E2E flow from 36 to 41 steps. All 12 tests pass (~3.5 min).
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tests/test_e2e.py— 9 tests covering 23 progressive steps against a real Keboola projectmake test-e2etarget, register@pytest.mark.e2e, exclude from defaultmake testTest plan
make test-e2epasses (9/9, ~90s)make teststill works (excludes e2e)make checkpasses (lint + format + existing tests)