Skip to content

Fix flaky fail-self-referencing-metaschema test#778

Merged
jviotti merged 1 commit intomainfrom
flaky-test
Mar 25, 2026
Merged

Fix flaky fail-self-referencing-metaschema test#778
jviotti merged 1 commit intomainfrom
flaky-test

Conversation

@jviotti
Copy link
Member

@jviotti jviotti commented Mar 25, 2026

Signed-off-by: Juan Cruz Viotti jv@jviotti.com

Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>
@augmentcode
Copy link

augmentcode bot commented Mar 25, 2026

🤖 Augment PR Summary

Summary: Makes the flaky fail-self-referencing-metaschema CLI test tolerant to either schema-detection order.
Changes: Generates two expected stderr transcripts and treats the test as passing if the actual output matches either variant.

🤖 Was this summary useful? React with 👍 or 👎

Copy link

@augmentcode augmentcode bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed. 1 suggestion posted.

Fix All in Augment

Comment augment review to trigger a new review at any time.

Did you forget to register a schema with such URI in the one?
EOF

diff "$TMP/output.txt" "$TMP/expected1.txt" || diff "$TMP/output.txt" "$TMP/expected2.txt"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

diff ... || diff ... will still print the first diff’s output when the output matches expected2.txt, so this test can be “passing” but noisy in CI logs. Consider ensuring the first mismatch doesn’t emit diff output when the fallback succeeds.

Severity: low

Fix This in Augment

🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.

Copy link

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

@jviotti jviotti merged commit 8671575 into main Mar 25, 2026
6 checks passed
@jviotti jviotti deleted the flaky-test branch March 25, 2026 18:17
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmark Index (community)

Details
Benchmark suite Current: 155d09b Previous: a52e00f Ratio
Add one schema (0 existing) 21 ms 20 ms 1.05
Add one schema (100 existing) 25 ms 26 ms 0.96
Add one schema (1000 existing) 75 ms 70 ms 1.07
Add one schema (10000 existing) 618 ms 586 ms 1.05
Update one schema (1 existing) 18 ms 21 ms 0.86
Update one schema (101 existing) 28 ms 31 ms 0.90
Update one schema (1001 existing) 78 ms 75 ms 1.04
Update one schema (10001 existing) 637 ms 600 ms 1.06
Cached rebuild (1 existing) 12 ms 10 ms 1.20
Cached rebuild (101 existing) 13 ms 12 ms 1.08
Cached rebuild (1001 existing) 27 ms 26 ms 1.04
Cached rebuild (10001 existing) 195 ms 186 ms 1.05
Index 100 schemas 129 ms 121 ms 1.07
Index 1000 schemas 966 ms 1046 ms 0.92
Index 10000 schemas 13591 ms 14704 ms 0.92

This comment was automatically generated by workflow using github-action-benchmark.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmark Index (enterprise)

Details
Benchmark suite Current: 155d09b Previous: a52e00f Ratio
Add one schema (0 existing) 27 ms 29 ms 0.93
Add one schema (100 existing) 27 ms 32 ms 0.84
Add one schema (1000 existing) 80 ms 92 ms 0.87
Add one schema (10000 existing) 639 ms 779 ms 0.82
Update one schema (1 existing) 23 ms 23 ms 1
Update one schema (101 existing) 28 ms 33 ms 0.85
Update one schema (1001 existing) 80 ms 93 ms 0.86
Update one schema (10001 existing) 631 ms 775 ms 0.81
Cached rebuild (1 existing) 12 ms 13 ms 0.92
Cached rebuild (101 existing) 17 ms 15 ms 1.13
Cached rebuild (1001 existing) 30 ms 34 ms 0.88
Cached rebuild (10001 existing) 203 ms 262 ms 0.77
Index 100 schemas 128 ms 135 ms 0.95
Index 1000 schemas 1079 ms 1301 ms 0.83
Index 10000 schemas 14119 ms 16504 ms 0.86

This comment was automatically generated by workflow using github-action-benchmark.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant