Skip to content

add ci pipeline for api.md generation#46329

Open
swathipil wants to merge 35 commits intomainfrom
swathipil/add-ci-pipeline-for-api-md
Open

add ci pipeline for api.md generation#46329
swathipil wants to merge 35 commits intomainfrom
swathipil/add-ci-pipeline-for-api-md

Conversation

@swathipil
Copy link
Copy Markdown
Member

fixes: #45851

@swathipil
Copy link
Copy Markdown
Member Author

@copilot - address this error in the CI tools step, and any other foreseeable ones:
/usr/bin/pwsh -NoLogo -NoProfile -NonInteractive -Command . '/mnt/vss/_work/_temp/dd1b6084-9e73-4f66-8413-fb288fd7ac3f.ps1'
Using Python 3.10.19 environment at: venv-apiview-md
error: Request failed after 3 retries in 4.5s
Caused by: Failed to fetch: [https://pypi.org/simple/certifi/](https://pypi.org/simple/certifi/%60)
Caused by: error sending request for url (https://pypi.org/simple/certifi/)
Caused by: client error (Connect)
Caused by: tcp connect error
Caused by: Operation not permitted (os error 1)
NativeCommandExitException: /mnt/vss/_work/_temp/dd1b6084-9e73-4f66-8413-fb288fd7ac3f.ps1:5
Line |
5 | uv pip install -r eng/ci_tools.txt
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Program "uv" ended with non-zero exit code: 2.

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 15, 2026

@copilot - address this error in the CI tools step, and any other foreseeable ones:
/usr/bin/pwsh -NoLogo -NoProfile -NonInteractive -Command . '/mnt/vss/_work/_temp/dd1b6084-9e73-4f66-8413-fb288fd7a...

Updated in commit ee86057. I changed the CI tools install step in the api.md regeneration job to use python -m pip install -r eng/ci_tools.txt instead of uv pip, which avoids the blocked direct PyPI fetch path seen in the failure while keeping the rest of the job behavior intact.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • scanning-api.github.com
    • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

@swathipil swathipil force-pushed the swathipil/add-ci-pipeline-for-api-md branch from ee86057 to 8e22811 Compare April 15, 2026 19:47
Comment thread eng/pipelines/templates/jobs/apiview-md.yml Outdated
Copy link
Copy Markdown
Member

@scbedd scbedd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Triggers are safe. You're using the app. Looks g2g to me.

@swathipil swathipil requested review from benbp and weshaggard May 6, 2026 00:30
@swathipil swathipil marked this pull request as ready for review May 6, 2026 00:31
Copilot AI review requested due to automatic review settings May 6, 2026 00:31
@swathipil swathipil requested a review from mccoyp as a code owner May 6, 2026 00:31
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new Azure Pipelines workflow to mass-regenerate api.md files with azpysdk apistub --md, plus the tooling changes needed for in-place markdown generation and package filtering. This fits into the repo’s engineering automation by turning APIView markdown refreshes into a repeatable CI-driven maintenance flow.

Changes:

  • Added a new regenerate-apiview-md pipeline and job template to run APIView markdown generation and open a draft PR when changes are detected.
  • Updated azpysdk apistub and dispatch_checks.py to support --md, in-place api.md output, package exclusions, and management-only filtering.
  • Extended apistub tests to cover the new markdown output-path behavior.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
eng/tools/azure-sdk-tools/tests/test_apistub.py Updates unit tests for new apistub --md output behavior.
eng/tools/azure-sdk-tools/ci_tools/functions.py Adds an Only_management package filter.
eng/tools/azure-sdk-tools/azpysdk/apistub.py Changes apistub wheel lookup and markdown generation flow.
eng/scripts/dispatch_checks.py Threads new --md and exclusion options through concurrent check dispatch.
eng/pipelines/templates/jobs/apiview-md.yml Adds the job that regenerates api.md, stages changes, and creates a draft PR.
eng/pipelines/regenerate-apiview-md.yml Adds the top-level pipeline entrypoint and trigger configuration.

variables:
- template: /eng/pipelines/templates/variables/globals.yml
- name: ExcludedPackages
value: 'nspkg,azure-storage-extensions,azure-mgmt-app,azure-mgmt-videoanalyzer,azure-mgmt-changeanalysis,azure-mgmt-apimanagement'
# generated token JSON. The package should rename one of these types to fix this.
- task: PythonScript@0
displayName: 'Regenerate api.md for all packages'
continueOnError: true
Comment thread eng/pipelines/templates/jobs/apiview-md.yml
Comment on lines 199 to +203
logger.info(f"Generating api.md for {package_name}")
# When no --dest-dir is given, write api.md directly into the package
# directory so it is tracked by git. When --dest-dir is provided, keep
# the existing behaviour of writing into <dest_dir>/<package_name>/.
md_output_path = package_dir if not dest_dir else out_token_path
Comment on lines +36 to +40
# Check staging directory first (wheel built by create_package_and_install)
if staging_dir:
found_whl = find_whl(staging_dir, pkg_details.name, pkg_details.version)
if found_whl:
return os.path.join(staging_dir, found_whl)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add CI pipeline to mass‑regenerate api.md using azpysdk apiview --md

4 participants