Skip to content

Support fetching config via git clone#78

Merged
Kilo59 merged 8 commits intomainfrom
gitclones
Mar 8, 2026
Merged

Support fetching config via git clone#78
Kilo59 merged 8 commits intomainfrom
gitclones

Conversation

@Kilo59
Copy link
Copy Markdown
Owner

@Kilo59 Kilo59 commented Mar 8, 2026

Summary by Sourcery

Add support for fetching upstream configuration via git clone for SSH/git URLs while preserving existing HTTP-based fetching.

New Features:

  • Allow ruff-sync to fetch pyproject.toml from git repositories when the upstream URL uses SSH or git-based schemes, using a shallow, blob-filtered clone to retrieve only the configuration file.

Enhancements:

  • Bypass raw URL conversion for git/SSH URLs and route resolution through a unified upstream config fetch helper shared by pull and check commands.

Documentation:

  • Document git-based cloning usage and behavior in the README, including supported URL schemes and shallow clone strategy.

Tests:

  • Add tests covering git-based upstream fetching, path handling within repositories, git failures, and resolution behavior for git/SSH URLs.

CI Improvements:

  • Added a pre-publish job that builds the package and verifies it installs correctly via uv tool install in a clean environment before publishing to PyPI.

Script Capabilities:

  • Renamed the dogfooding scripts for better clarity (pull_dogfood.sh and check_dogfood.sh).
  • Added a new gitclone_dogfood.sh script to exercise the tool's new git-fetching behavior locally.

@sourcery-ai
Copy link
Copy Markdown

sourcery-ai Bot commented Mar 8, 2026

Reviewer's Guide

Adds git/SSH-based upstream configuration fetching via a new fetch_upstream_config helper that performs an efficient shallow git clone for non-HTTP URLs, wires it into the check and pull commands, ensures raw URL resolution bypasses git/SSH URLs, updates CLI help and README usage docs, and adds tests covering git clone behavior and URL handling.

Sequence diagram for the new fetch_upstream_config retrieval flow

sequenceDiagram
  actor User
  participant RuffSyncCLI
  participant fetch_upstream_config
  participant GitSubprocess
  participant GitRemote
  participant HttpServer

  User->>RuffSyncCLI: run check/pull with upstream URL
  RuffSyncCLI->>fetch_upstream_config: url, client, branch, path
  alt URL is ssh/git/git+ssh or starts with git@
    fetch_upstream_config->>GitSubprocess: git clone --depth 1 --filter=blob:none --no-checkout --branch branch url
    GitSubprocess->>GitRemote: shallow clone metadata
    GitRemote-->>GitSubprocess: repo metadata
    GitSubprocess-->>fetch_upstream_config: clone completed
    fetch_upstream_config->>GitSubprocess: git -C temp_dir restore --source branch target_path
    GitSubprocess->>GitRemote: fetch pyproject.toml blob
    GitRemote-->>GitSubprocess: pyproject.toml contents
    GitSubprocess-->>fetch_upstream_config: file materialized in temp_dir
    fetch_upstream_config-->>RuffSyncCLI: StringIO(pyproject.toml from git)
  else URL is http/https
    fetch_upstream_config->>HttpServer: HTTP GET resolved raw URL
    HttpServer-->>fetch_upstream_config: pyproject.toml response text
    fetch_upstream_config-->>RuffSyncCLI: StringIO(pyproject.toml from http)
  end
  RuffSyncCLI->>RuffSyncCLI: get_ruff_tool_table and merge/check
  RuffSyncCLI-->>User: result of pull/check
Loading

File-Level Changes

Change Details Files
Introduce git/SSH-based upstream configuration fetching via shallow git clone and integrate it into existing commands.
  • Add fetch_upstream_config async helper that chooses between git clone for git/SSH URLs and HTTP download for others.
  • Implement a temp-directory-based git clone flow using --depth 1, --filter=blob:none, and --no-checkout, followed by git restore of only pyproject.toml or a path-prefixed variant.
  • Use asyncio.to_thread to offload the blocking subprocess-based git operations from the async context.
  • Wire fetch_upstream_config into check and pull to replace direct calls to download, preserving existing HTTP behavior while enabling git/SSH URLs.
ruff_sync.py
Ensure raw URL resolution does not rewrite git/SSH URLs so they are handled by the git-based path.
  • Update resolve_raw_url to early-return for ssh, git, git+ssh, and git@-style URLs instead of converting them to raw content URLs.
  • Add tests to assert git/SSH URLs are returned unchanged by resolve_raw_url.
ruff_sync.py
tests/test_git_fetch.py
Extend documentation and CLI help to describe git/SSH usage and capabilities.
  • Add git/SSH URL examples to the CLI help text to illustrate git-based pull usage.
  • Update README usage examples to show git clone style URLs and clarify default branch behavior and shallow clone semantics.
  • Document git clone support and broader host support (including private SSH servers) in the feature list.
ruff_sync.py
README.md
Add tests validating git-backed configuration fetching behavior and error handling.
  • Add async tests that monkeypatch subprocess.run to simulate successful git clone and restore flows, asserting command arguments and resulting file content for both root and subdirectory paths.
  • Add a failure-path test that ensures subprocess.CalledProcessError from git propagates out of fetch_upstream_config.
  • Verify that resolve_raw_url leaves git/SSH URLs unchanged via dedicated tests.
tests/test_git_fetch.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 8, 2026

Codecov Report

❌ Patch coverage is 88.09524% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.13%. Comparing base (7a6395c) to head (03bcd41).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
ruff_sync.py 88.09% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #78      +/-   ##
==========================================
- Coverage   92.83%   92.13%   -0.71%     
==========================================
  Files           1        1              
  Lines         321      356      +35     
==========================================
+ Hits          298      328      +30     
- Misses         23       28       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Kilo59 Kilo59 marked this pull request as ready for review March 8, 2026 20:41
@Kilo59 Kilo59 self-assigned this Mar 8, 2026
@Kilo59 Kilo59 linked an issue Mar 8, 2026 that may be closed by this pull request
Copy link
Copy Markdown

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue, and left some high level feedback:

  • The logic for detecting git/SSH URLs (url.scheme checks plus str(url).startswith('git@')) is duplicated in both resolve_raw_url and fetch_upstream_config; consider extracting this into a small helper (e.g., is_git_url(url)) to keep the behavior consistent and easier to update.
  • Relying on git restore assumes a relatively recent git version; if you need broader compatibility, consider falling back to git checkout or similar when restore is not available.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The logic for detecting git/SSH URLs (`url.scheme` checks plus `str(url).startswith('git@')`) is duplicated in both `resolve_raw_url` and `fetch_upstream_config`; consider extracting this into a small helper (e.g., `is_git_url(url)`) to keep the behavior consistent and easier to update.
- Relying on `git restore` assumes a relatively recent git version; if you need broader compatibility, consider falling back to `git checkout` or similar when `restore` is not available.

## Individual Comments

### Comment 1
<location path="tests/test_git_fetch.py" line_range="12-21" />
<code_context>
+        assert restore_args[6] == "sub/dir/pyproject.toml"
+
+
+@pytest.mark.asyncio
+async def test_fetch_upstream_config_git_failure(monkeypatch: pytest.MonkeyPatch):
+    url = URL("git@github.com:Kilo59/ruff-sync.git")
+
</code_context>
<issue_to_address>
**suggestion (testing):** Add a test for the FileNotFoundError path when pyproject.toml is missing after a successful git operation.

The code now explicitly raises `FileNotFoundError` when git operations succeed but `pyproject.toml` is missing. Currently, only the `CalledProcessError` path is tested. Please add a test that mocks successful `subprocess.run` calls (clone and restore both returning zero) without creating the config file, and asserts that `fetch_upstream_config` raises `FileNotFoundError` to cover this error path.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread tests/test_git_fetch.py
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 8, 2026

╒═══════════════╤═════════════════╤════════════╤══════════════╤═════════════════╕
│ File          │ Lines of Code   │ Unique     │ Cyclomatic   │ Maintainabili   │
│               │                 │ Operands   │ Complexity   │ ty Index        │
╞═══════════════╪═════════════════╪════════════╪══════════════╪═════════════════╡
│ ruff_sync.py  │ 660 -> 777      │ 12 -> 12   │ 98 -> 105    │ 35.4133 ->      │
│               │                 │            │              │ 33.3087         │
├───────────────┼─────────────────┼────────────┼──────────────┼─────────────────┤
│ tests/test_gi │ - -> 155        │ - -> 3     │ - -> 5       │ - -> 45.51265   │
│ t_fetch.py    │                 │            │              │ 435982624       │
╘═══════════════╧═════════════════╧════════════╧══════════════╧═════════════════╛

@Kilo59 Kilo59 changed the title Git clone based download Support fetching config via git clone Mar 8, 2026
@Kilo59 Kilo59 merged commit b406d3b into main Mar 8, 2026
12 of 14 checks passed
@Kilo59 Kilo59 deleted the gitclones branch March 8, 2026 21:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for pulling configuration from non-public repositories

1 participant