feat: add integration test infrastructure for CLI#2
Merged
andrzejchm merged 9 commits intomainfrom Mar 23, 2026
Merged
Conversation
- Add .env.test to .gitignore - Add test:integration npm script - Create vitest.config.integration.ts for serial integration tests - Create docs/testing-setup.md with Notion workspace setup guide - Add .env.test.example template with all 8 fixture env vars - Add dotenv-cli devDependency for local test runs
- Create fixture duplication helper (page + database support) - Create CLI runner helper (shells out to dist/cli.js) - Add env var validation setup file for 8 required vars - Update vitest config for serial execution (forks, single worker)
- Add read tests (rich page, empty page, JSON output, invalid ID) - Add append tests (append + read-back verification) - Add search test (find fixture by title) - Add db query tests (unfiltered, filtered, empty database) - Add db schema test (verify column names) - Add comments test (list page comments) - Add comment-add tests (add + verify comment) - Add ls test (list workspace content)
- Add integration job to ci.yml (depends on ci, PR-only) - Build CLI and run integration tests with 8 Notion secrets - Add 10-minute timeout for safety - Document required GitHub Actions secrets in testing-setup.md
- Create testing-notion-cli skill for on-demand exploratory testing - Add 4 initial scenarios: read, append, db query, search - Document fixture-to-env-var mapping and placeholder substitution - Include scenario discovery and authoring guidance
- Add exclude patterns to vitest.config.ts for tests/integration/**, tests/e2e/**, node_modules/**, .opencode/** - Remove stale passWithNoTests note from testing-setup.md
duplicatePage was creating title-only copies, causing read.test.ts to fail with empty stdout. Now fetches markdown via retrieveMarkdown and passes it to pages.create so the duplicate includes all blocks.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds integration tests that run the built CLI binary against a real Notion workspace. Also adds an LLM agent test skill for on-demand exploratory testing. Unit tests are unchanged.
Changes
Integration Test Infrastructure
vitest.config.integration.ts- serial execution, 30s timeoutstest:integrationnpm script +dotenv-clidevDependencyvitest.config.ts- exclude patterns so unit tests and integration tests stay separateTest Helpers (
tests/integration/helpers/)fixture.ts- duplicates Notion pages/databases per test, cleans up aftercli-runner.ts- shells out todist/cli.jswithNOTION_API_TOKENvalidate-env.ts- fails fast when env vars are missingIntegration Tests - 15 test cases across 8 files
readappendsearchdb querydb schemacommentscommentlsCI Workflow
integrationjob inci.yml- depends onci, PR-only, passes 8 Notion secretsLLM Agent Testing
.agents/skills/testing-notion-cli/SKILL.md- on-demand exploratory test skilltests/e2e/scenarios.yaml- 4 initial scenariosDocumentation
docs/testing-setup.md- Notion workspace setup guide + CI secrets config.env.test.example- template with all 8 env varsSetup Required
To run integration tests, set up GitHub Actions secrets listed in
docs/testing-setup.md:NOTION_TEST_TOKEN,NOTION_TEST_ROOT_PAGE_ID,NOTION_FIXTURE_RICH_PAGE_ID,NOTION_FIXTURE_SIMPLE_PAGE_ID,NOTION_FIXTURE_EMPTY_PAGE_ID,NOTION_FIXTURE_TASK_DB_ID,NOTION_FIXTURE_EMPTY_DB_ID,NOTION_FIXTURE_COMMENTS_PAGE_IDVerification
npm test- 77 unit tests pass (unchanged)npm run typecheck- cleannpm run check- 70 files, no issues