Skip to content

Conversation

@LeahMarieBush
Copy link
Contributor

@LeahMarieBush LeahMarieBush commented May 5, 2025

Description

This PR adds support for the nav-data endpoints to the compare-api-responses script. It also refactors a few parts of code to clean things up. I've added PR comments for more details on that refactoring.

Testing

First checkout this branch

Run the following commands to test the script:

One product:

  • npm run compare-api-responses -- -n https://web-unified-docs-hashicorp.vercel.app/ -o https://content.hashicorp.com -a nav-data -p terraform-cdk
  • Output:
> compare-api-responses
> node scripts/compare-api-responses/index.mjs -n https://web-unified-docs-hashicorp.vercel.app/ -o https://content.hashicorp.com -a nav-data -p terraform-cdk

Testing API URL:
https://web-unified-docs-hashicorp.vercel.app//api/content/terraform-cdk/nav-data/v0.20.x/cdktf
Compared values have no visual difference.
✅ No visual difference found.

Testing API URL:
https://web-unified-docs-hashicorp.vercel.app//api/content/terraform-cdk/nav-data/v0.19.x/cdktf
Compared values have no visual difference.
✅ No visual difference found.

Testing API URL:
https://web-unified-docs-hashicorp.vercel.app//api/content/terraform-cdk/nav-data/v0.18.x/cdktf
Compared values have no visual difference.
✅ No visual difference found.

Testing API URL:
https://web-unified-docs-hashicorp.vercel.app//api/content/terraform-cdk/nav-data/v0.17.x/cdktf
Compared values have no visual difference.
✅ No visual difference found.

Testing API URL:
https://web-unified-docs-hashicorp.vercel.app//api/content/terraform-cdk/nav-data/v0.16.x/cdktf
Compared values have no visual difference.
✅ No visual difference found.

Testing API URL:
https://web-unified-docs-hashicorp.vercel.app//api/content/terraform-cdk/nav-data/v0.15.x/cdktf
Compared values have no visual difference.
✅ No visual difference found.

Testing API URL:
https://web-unified-docs-hashicorp.vercel.app//api/content/terraform-cdk/nav-data/v0.14.x/cdktf
Compared values have no visual difference.
✅ No visual difference found.

Testing API URL:
https://web-unified-docs-hashicorp.vercel.app//api/content/terraform-cdk/nav-data/v0.13.x/cdktf
Compared values have no visual difference.
✅ No visual difference found.

Testing API URL:
https://web-unified-docs-hashicorp.vercel.app//api/content/terraform-cdk/nav-data/v0.12.x/cdktf
Compared values have no visual difference.
✅ No visual difference found.

Testing API URL:
https://web-unified-docs-hashicorp.vercel.app//api/content/terraform-cdk/nav-data/v0.11.x/cdktf
Compared values have no visual difference.
✅ No visual difference found.

Testing API URL:
https://web-unified-docs-hashicorp.vercel.app//api/content/terraform-cdk/nav-data/v0.10.x/cdktf
Compared values have no visual difference.
✅ No visual difference found.

Testing API URL:
https://web-unified-docs-hashicorp.vercel.app//api/content/terraform-cdk/nav-data/v0.9.x/cdktf
Compared values have no visual difference.
✅ No visual difference found.

Testing API URL:
https://web-unified-docs-hashicorp.vercel.app//api/content/terraform-cdk/nav-data/v0.8.x/cdktf
Compared values have no visual difference.
✅ No visual difference found.

Tests passed: 13 of 13 🎉

One version for one product:

  • npm run compare-api-responses -- -n https://web-unified-docs-hashicorp.vercel.app/ -o https://content.hashicorp.com -a nav-data -p terraform-cdk -v v0.1 0.x
  • Output:
> node scripts/compare-api-responses/index.mjs -n https://web-unified-docs-hashicorp.vercel.app/ -o https://content.hashicorp.com -a nav-data -p terraform-cdk -v v0.10.x

Testing API URL:
https://web-unified-docs-hashicorp.vercel.app//api/content/terraform-cdk/nav-data/v0.10.x/cdktf
Compared values have no visual difference.
✅ No visual difference found.

Tests passed: 1 of 1 🎉

All products:

  • npm run compare-api-responses -- -n https://web-unified-docs-hashicorp.vercel.app/ -o https://content.hashicorp.com -a nav-data
  • Output: Too long to include, there will be a few failures for this one because the content api has different nav-data for a few of the edge case versions that we fixed for UDR

@github-actions
Copy link
Contributor

github-actions bot commented May 5, 2025

Vercel Previews Deployed

Name Status Preview Updated (UTC)
Dev Portal ✅ Ready (Inspect) Visit Preview Tue May 6 15:40:08 UTC 2025
Unified Docs API ✅ Ready (Inspect) Visit Preview Tue May 6 15:35:33 UTC 2025

}
}

async function fetchApiTypeVersionAndNav(options, product, versionMetadata) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since I split up version-metadata and nav-data options and nav-data uses a different response value than what is returned here, it made sense to me to pull this out and put it within each case


const testsPassed = []
const testsFailed = []
const totalTests = []
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added totalTests here since options.numOfTests may not be defined for every test run. This way the test results do not print 'Tests passed: 14 of undefined'

totalTests.push(i + 1)
saveTestOutputIfSelected(outputString, newApiURL)
}
} else if (options.api === 'content-versions') {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'content-versions' got moved out of the for loops for versionmetadata and product because it does not use either thing in the loops. Since I removed the break statements for the loops, this would get run a bunch of times with the same output.

Comment on lines -306 to -309
break
if (difference.includes('Compared values have no visual difference.')) {
testsPassed.push(newApiURL)
console.log('✅ No visual difference found.\n')
} else {
testsFailed.push(newApiURL)
console.log(`${difference}\n`)
}

break
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These break statements are removed because they are causing the tests to only run on the first version of the first product. Now, all products and versions get tests run on them

Comment on lines +333 to +334
`Tests passed: ${testsPassed.length} of ${options.numOfTests ?? totalTests.length} ${
testsPassed.length === (options.numOfTests ?? totalTests.length)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to comment above about totalTests. The options.numOfTests is not always defined so we need a fallback value

@LeahMarieBush LeahMarieBush marked this pull request as ready for review May 5, 2025 19:52
@LeahMarieBush LeahMarieBush requested a review from a team as a code owner May 5, 2025 19:52
@LeahMarieBush LeahMarieBush requested a review from rmainwork May 5, 2025 19:52
@LeahMarieBush LeahMarieBush merged commit f92a63f into main May 7, 2025
10 checks passed
@LeahMarieBush LeahMarieBush deleted the leah/feat/api-compare-nav-data branch August 5, 2025 15:35
hashibot-web added a commit that referenced this pull request Sep 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants