-
Notifications
You must be signed in to change notification settings - Fork 395
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: mark unreliable langchain_community tests #9490
Conversation
Datadog ReportBranch report: ✅ 0 Failed, 310 Passed, 140 Skipped, 8m 16.7s Total duration (2m 12.28s time saved) |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #9490 +/- ##
===========================================
- Coverage 69.41% 10.29% -59.13%
===========================================
Files 1315 1285 -30
Lines 124706 122866 -1840
===========================================
- Hits 86567 12645 -73922
- Misses 38139 110221 +72082 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've done some investigating and it looks like the underlying issue is a version issue with vcrpy (reading cassette files). I'm good to mark these tests as flaky for now, but will be pinning vcrpy in a future PR which should hopefully remove the flakiness.
BenchmarksBenchmark execution time: 2024-06-10 15:33:32 Comparing candidate commit 96a67f4 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 96 metrics, 0 unstable metrics. |
This change marks some unreliable tests in the langchain_community suite. Some of these had been recently unmarked in ecc56cf, but it seems like they still have some underlying unreliability (example).
Checklist
changelog/no-changelog
is set@DataDog/apm-tees
.Reviewer Checklist