Skip to content

[python] add back api view sphinx#10323

Merged
iscai-msft merged 45 commits into
microsoft:mainfrom
iscai-msft:python/addBackApiViewSphinx
Apr 21, 2026
Merged

[python] add back api view sphinx#10323
iscai-msft merged 45 commits into
microsoft:mainfrom
iscai-msft:python/addBackApiViewSphinx

Conversation

@iscai-msft
Copy link
Copy Markdown
Member

No description provided.

@microsoft-github-policy-service microsoft-github-policy-service Bot added the emitter:client:python Issue for the Python client emitter: @typespec/http-client-python label Apr 9, 2026
@pkg-pr-new
Copy link
Copy Markdown

pkg-pr-new Bot commented Apr 9, 2026

Open in StackBlitz

npm i https://pkg.pr.new/@typespec/http-client-python@10323

commit: 25cdefe

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 9, 2026

All changed packages have been documented.

  • @typespec/http-client-python
Show changes

@typespec/http-client-python - internal ✏️

Add apiview and sphinx to ci

@azure-sdk
Copy link
Copy Markdown
Collaborator

azure-sdk commented Apr 9, 2026

You can try these changes here

🛝 Playground 🌐 Website 🛝 VSCode Extension

Comment thread packages/http-client-python/eng/scripts/ci/regenerate.ts Fixed
iscai-msft and others added 11 commits April 15, 2026 14:48
…nation

When azure and unbranded tox envs run in parallel, both build packages
with the same name+version (e.g. typetest-array==1.0.0b1) from different
source directories. Without --no-cache, uv's shared build cache can serve
an unbranded wheel to the azure env (or vice versa), causing 'No module
named corehttp' errors in azure sphinx builds.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Key changes:
- Merge lint + mypy + pyright into single 'check-{flavor}' tox env,
  cutting redundant package installs from 10 to 6 (each builds ~100+
  wheels from source)
- Remove separate lint:generated and typecheck:generated steps from
  Test-Packages.ps1 — now covered by check-{flavor} in npm run ci
- Use per-flavor uv cache dir instead of --no-cache to get cache
  benefits within a flavor while preventing cross-flavor contamination
- Increase parallelism in CI check scripts from cpu_count/2 to cpu_count
- Increase Python regeneration jobs from jobs/2 to full jobs

Before: 10 tox envs × ~100 wheel builds each = ~1000 builds
After:  6 tox envs × ~100 wheel builds each = ~600 builds

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Move black formatting out of per-spec worker processes into a single
  sequential pass after all codegen completes. This avoids duplicating
  black's import/init cost across N workers and lets codegen workers
  finish faster.
- Increase install_packages.py batch size from 20 to 50 (paths are
  ~60 chars each, well under Windows 32KB cmd limit). Reduces number
  of uv pip install invocations from 6 batches to 3.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Major CI performance improvements:

1. Pre-build wheels once per flavor before starting tox envs. Each tox
   env then installs from pre-built wheels via --find-links (instant,
   no compilation). Falls back to source install if wheels unavailable.

2. Split docs into separate apiview and sphinx tox envs so they run
   in parallel instead of sequentially within a single env.

Before: 6 tox envs (test×2, check×2, docs×2), docs sequential
After:  8 tox envs (test×2, check×2, apiview×2, sphinx×2), all parallel
        + wheels pre-built so each env installs in <5s instead of ~2min

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Generated SDK packages are regenerated in CI and don't need to be
checked in. Only the 4 custom _patch.py files in authentication-api-key
(which add patch_added_operation) are preserved via git add -f.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sphinx needs azure-core (azure) or corehttp (unbranded) to import
generated packages for docstring extraction. Without these deps,
autodoc fails with 'No module named azure.core/corehttp'.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
check-azure was 1213s (20 min) running pylint→mypy→pyright sequentially.
Split back into separate envs so all 3 run in parallel. Pre-built wheels
make the extra package installs cheap (~5s each).

Also reduce apiview timeout from 60s→30s and retries from 3→2 to avoid
slow retries on genuinely broken packages.

Before: check-azure ~1213s (sequential lint+mypy+pyright)
After:  lint-azure ~400s, mypy-azure ~400s, pyright-azure ~400s (parallel)
Expected wall clock: ~400s instead of ~1213s

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
iscai-msft and others added 5 commits April 16, 2026 10:30
uv pip install --no-index --find-links expects package names or wheel
paths, not source directory paths. The source dirs weren't matching
any wheels, causing every install to fall back to building from source.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…lint

The Azure DevOps feed redirects downloads to internal URLs that require
auth, causing 401 errors with uv pip. Switch to pip install with --no-deps
for Azure-feed-only packages, and install PyPI deps separately.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The azure-pylint-guidelines-checker package provides the
pylint_guidelines_checker module. Using --no-deps prevented the module
from being properly registered, causing pylint to fail with
'Plugin pylint_guidelines_checker is impossible to load'.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
tox-uv creates venvs using uv which doesn't include pip. Commands using
bare 'pip install' either fail silently or install to wrong location.
Switch all tox commands to 'uv pip install' to match tox-uv managed envs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@iscai-msft iscai-msft enabled auto-merge April 20, 2026 16:18
@iscai-msft iscai-msft disabled auto-merge April 20, 2026 16:22
@iscai-msft iscai-msft added this pull request to the merge queue Apr 20, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Apr 20, 2026
@iscai-msft iscai-msft added this pull request to the merge queue Apr 21, 2026
Merged via the queue into microsoft:main with commit f1750fa Apr 21, 2026
41 of 42 checks passed
@iscai-msft iscai-msft deleted the python/addBackApiViewSphinx branch April 21, 2026 16:31
msyyc added a commit to Azure/autorest.python that referenced this pull request Apr 28, 2026
* sync structure diff

* add changelog

* fix ci error

* update

* update
self.write_file(
root_dir / Path("__init__.py"),
general_serializer.serialize_pkgutil_init_file(),
)
Copy link
Copy Markdown
Contributor

@msyyc msyyc Apr 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic will overwrite customized code outside generation-subdir like https://github.com/Azure/autorest.python/pull/3465/changes#diff-055573a6877b8aaf078962b6c0f48fccb187f821100505f094e505296bc0d8f2 which is not expected behavior.

Fortunately, we have test case https://github.com/Azure/autorest.python/blob/main/packages/typespec-python/tests/mock_api/shared/test_generation_subdir_for_customized_code.py in typespec-python to cover this scenario.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

emitter:client:python Issue for the Python client emitter: @typespec/http-client-python eng

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants