Skip to content

fix: Make tests more robust & fix path-handling bugs on macOS #183

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 15, 2025

Conversation

varungandhi-src
Copy link
Contributor

@varungandhi-src varungandhi-src commented Aug 14, 2025

So it looks like basically there was a bug where the indexer was
generating 0 documents on macOS (related to the recent bug fix in
https://github.com/sourcegraph/scip-python/pull/177/files where we
started normalizing the file paths passed to setTrackedFiles). 😬

However, the tests were passing on macOS not because they were
correct, but because the zero documents case bypassed the
updating/diffing process. 😢

This patch changes the test diffing logic to be more robust, in handling
various scenarios such as 0 documents. It also introduces a couple
of flags, one for filtering tests, and one for running tests with fail-fast
behavior.

On fixing the tests, I discovered some bugs due to improper handling
of case-sensitivity, so I fixed those (which is why the snapshots are unchanged
and the tests are passing on macOS). For some of these, I'm not super
sure about Pyright behavior, so I added a little assertions library
inspired by Antithesis so that we can test the coverage of different
cases by our test suite on macOS.

See also: https://antithesis.com/docs/best_practices/sometimes_assertions/

  • Self-review of changes, since Amp did some of the more low-level stuff.

@varungandhi-src varungandhi-src changed the title fix: Make tests more robust & fix path bug fix: Make tests more robust & fix path bug on macOS Aug 14, 2025
@varungandhi-src varungandhi-src force-pushed the vg/run-all-tests branch 23 times, most recently from 4da1c41 to 321e78f Compare August 14, 2025 13:23
@varungandhi-src varungandhi-src requested a review from jupblb August 15, 2025 04:36
@varungandhi-src varungandhi-src changed the title fix: Make tests more robust & fix path bug on macOS fix: Make tests more robust & fix path-handling bugs on macOS Aug 15, 2025
Copy link
Contributor Author

varungandhi-src commented Aug 15, 2025

@varungandhi-src varungandhi-src merged commit 235614c into scip Aug 15, 2025
4 checks passed
@varungandhi-src varungandhi-src deleted the vg/run-all-tests branch August 15, 2025 12:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant