Skip to content

test: remove unnecessary mocks, use real filesystem#1156

Merged
jariy17 merged 6 commits intoaws:mainfrom
Hweinstock:test/remove-unnecessary-mocks
May 8, 2026
Merged

test: remove unnecessary mocks, use real filesystem#1156
jariy17 merged 6 commits intoaws:mainfrom
Hweinstock:test/remove-unnecessary-mocks

Conversation

@Hweinstock
Copy link
Copy Markdown
Contributor

@Hweinstock Hweinstock commented May 7, 2026

Description

Remove unnecessary mocks from 4 test files, replacing them with real filesystem operations using temp directories. This improves test reliability by verifying actual behavior rather than mock call sequences.

This change refactors a few tests to no longer mock the file system (there are many others to go), and adds to the reviewer system prompt to flag this in future PRs.

Related Issue

N/A — test quality improvement

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Other (please describe): Test quality improvement — removing unnecessary mocks

Testing

How have you tested the change?

  • I ran npm run test:unit and npm run test:integ
  • I ran npm run typecheck
  • I ran npm run lint
  • If I modified src/assets/, I ran npm run test:update-snapshots and committed the updated snapshots

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the
terms of your choice.

@github-actions github-actions Bot added size/m PR size: M agentcore-harness-reviewing AgentCore Harness review in progress labels May 7, 2026
Copy link
Copy Markdown

@agentcore-cli-automation agentcore-cli-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice cleanup overall — swapping in real tmp dirs makes these tests exercise more real behavior. I have a few concerns that I think are worth addressing before merging:

  1. The succeeds even when cache write fails test no longer reliably triggers a write failure (chmod on the directory doesn't prevent overwriting an existing file on Linux, is bypassed by root, and is a no-op on Windows). As written, the test passes even when the cache write silently succeeds, so it no longer validates the error-handling path.
  2. resolve-ui-dist-dir.test.ts dropped the returns null when no candidate has index.html and returns the first candidate that has index.html tests. The new replacement for the "skips AGENT_INSPECTOR_PATH..." case only asserts result !== tmpDir, which is a significantly weaker guarantee than toBeNull() and will pass for unrelated reasons (e.g., if any bundled candidate is present in the test environment).

Details inline.

Comment thread src/cli/__tests__/update-notifier.test.ts
Comment thread src/cli/operations/dev/web-ui/__tests__/resolve-ui-dist-dir.test.ts
Comment thread src/cli/__tests__/update-notifier.test.ts Outdated
@github-actions github-actions Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label May 7, 2026
@Hweinstock Hweinstock force-pushed the test/remove-unnecessary-mocks branch from 9190b1c to f50a436 Compare May 7, 2026 17:35
@github-actions github-actions Bot added size/m PR size: M and removed size/m PR size: M labels May 7, 2026
@Hweinstock Hweinstock force-pushed the test/remove-unnecessary-mocks branch from f50a436 to 66c3764 Compare May 7, 2026 17:37
@github-actions github-actions Bot added size/m PR size: M and removed size/m PR size: M labels May 7, 2026
@Hweinstock Hweinstock force-pushed the test/remove-unnecessary-mocks branch from 66c3764 to b8ca898 Compare May 7, 2026 17:40
@github-actions github-actions Bot added size/m PR size: M and removed size/m PR size: M labels May 7, 2026
@Hweinstock Hweinstock force-pushed the test/remove-unnecessary-mocks branch from b8ca898 to d719e53 Compare May 7, 2026 17:42
@github-actions github-actions Bot added size/m PR size: M and removed size/m PR size: M labels May 7, 2026
@Hweinstock Hweinstock force-pushed the test/remove-unnecessary-mocks branch from d719e53 to 50834f9 Compare May 7, 2026 17:46
@github-actions github-actions Bot added size/m PR size: M and removed size/m PR size: M labels May 7, 2026
@Hweinstock Hweinstock force-pushed the test/remove-unnecessary-mocks branch from 50834f9 to 5c53fb6 Compare May 7, 2026 17:52
@github-actions github-actions Bot added size/m PR size: M and removed size/m PR size: M labels May 7, 2026
@Hweinstock Hweinstock force-pushed the test/remove-unnecessary-mocks branch from 5c53fb6 to 611346f Compare May 7, 2026 17:55
@github-actions github-actions Bot added size/m PR size: M and removed size/m PR size: M labels May 7, 2026
@Hweinstock Hweinstock force-pushed the test/remove-unnecessary-mocks branch from 611346f to 62d9307 Compare May 7, 2026 17:57
@github-actions github-actions Bot added size/m PR size: M and removed size/m PR size: M labels May 7, 2026
@Hweinstock Hweinstock force-pushed the test/remove-unnecessary-mocks branch from 62d9307 to 354c9c3 Compare May 7, 2026 17:59
@github-actions github-actions Bot removed the size/m PR size: M label May 7, 2026
@github-actions github-actions Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label May 7, 2026
@Hweinstock Hweinstock force-pushed the test/remove-unnecessary-mocks branch from eec4585 to 022c09b Compare May 7, 2026 18:42
@github-actions github-actions Bot added size/m PR size: M and removed size/m PR size: M labels May 7, 2026
@Hweinstock Hweinstock closed this May 7, 2026
@Hweinstock Hweinstock reopened this May 7, 2026
@github-actions github-actions Bot added size/m PR size: M agentcore-harness-reviewing AgentCore Harness review in progress and removed size/m PR size: M labels May 7, 2026
Copy link
Copy Markdown

@agentcore-cli-automation agentcore-cli-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took a pass on the latest commit (022c09b). The two substantive concerns from prior reviews — the cache-write-failure test not reliably triggering the error path, and the missing coverage for resolveUIDistDir returning null — have both been addressed:

  • succeeds even when cache write fails now points AGENTCORE_CONFIG_DIR at a regular file, so mkdir/writeFile reliably fail cross-platform without depending on permission bits or uid.
  • resolve-ui-dist-dir.test.ts restores the returns null when no candidate has index.html and skips AGENT_INSPECTOR_PATH when env var is set but dir lacks index.html cases with a narrow fs.existsSync spy, which is consistent with the "mock only at true I/O boundaries" guidance added to review.md.

The refactor is a clear improvement — real tmp dirs exercise actual fs behavior, and dropping the compareVersions mock means the cache-path tests now verify real version comparison end-to-end. Nothing blocking from my side; approving.

@github-actions github-actions Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label May 7, 2026
@Hweinstock Hweinstock marked this pull request as ready for review May 7, 2026 18:51
@Hweinstock Hweinstock requested a review from a team May 7, 2026 18:51
tejaskash
tejaskash previously approved these changes May 7, 2026
Copy link
Copy Markdown
Contributor

@tejaskash tejaskash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Hweinstock added 5 commits May 7, 2026 19:08
…filesystem

- Replace fs/promises mock with real temp directory via GLOBAL_CONFIG_DIR
- Remove compareVersions mock, let real pure function run
- Refactor source to use existing GLOBAL_CONFIG_DIR (supports AGENTCORE_CONFIG_DIR env var)
- Only mock fetchLatestVersion (network) and PACKAGE_VERSION (constant)
- Remove redundant CACHE_DIR alias, use tmpDir directly
- Add force:true to afterAll rmSync for robustness
- Wrap chmod test in try/finally to prevent leaked read-only state
- Remove vacuous conditional assertion in resolve-ui-dist-dir
- Switch get-trace to beforeEach/afterEach for test isolation consistency
@Hweinstock Hweinstock force-pushed the test/remove-unnecessary-mocks branch from 022c09b to 86e2850 Compare May 7, 2026 19:09
@github-actions github-actions Bot added size/m PR size: M and removed size/m PR size: M labels May 7, 2026
Comment thread .github/harness/prompts/review.md Outdated
Comment thread src/lib/time-constants.ts
@github-actions github-actions Bot added size/m PR size: M and removed size/m PR size: M labels May 8, 2026
@Hweinstock
Copy link
Copy Markdown
Contributor Author

security audit failure is fixed in #1184 since its an unrelated issue.

@jariy17 jariy17 merged commit 9063a77 into aws:main May 8, 2026
23 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m PR size: M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants