Skip to content

Add support for pulling configuration from non-public repositories #75

@Kilo59

Description

@Kilo59

Motivation

ruff-sync currently only works with publicly‑available pyproject.toml files. Many teams host their shared Ruff configuration in private repositories (e.g., internal GitHub orgs, self‑hosted Git servers, or GitHub Enterprise). Being able to pull the upstream configuration from a non‑public repo would:

  • Allow organizations to keep a single source‑of‑truth for lint settings while still enforcing it across all projects.
  • Reduce duplication of configuration files across internal projects.
  • Enable the same workflow that works for public repos (automatic merging, comment/whitespace preservation) to be used securely in private environments.

Proposed solution

We can support both of the following approaches – they are independent and can be used together or separately.

1️⃣ Authentication via HTTP token (httpx)

  • Add an optional --auth-token <TOKEN> (or --auth-token-env <ENV_VAR>) flag that passes a token to httpx for authenticated HTTP requests.
  • Detect common environment variables (GITHUB_TOKEN, GH_TOKEN, CI_JOB_TOKEN, etc.) automatically, falling back to the flag if provided.
  • When the upstream URL is an HTTP(S) endpoint (e.g., https://example.com/repo/pyproject.toml), httpx will include the token in the request header (default Authorization: Bearer <TOKEN>).
  • Addendum – Generalising the token handling
    The current design only mentions GitHub‑specific variables, but the same mechanism works for any service that accepts bearer tokens or basic auth:
    • Bearer‑style APIs – set --auth-token (or an env var such as API_TOKEN) and the client will send Authorization: Bearer <TOKEN>.
    • Basic‑auth – if the token contains a colon (username:password) we can automatically switch to httpx.BasicAuth.
    • Custom header – expose a --auth-header <NAME> flag to allow callers to specify a different header name (e.g., X-API-Key).
    • This makes the feature usable with GitLab, Bitbucket, private Artifactory servers, or any generic HTTP‑served TOML file that requires authentication.

2️⃣ SSH‑based cloning (git)

  • If the upstream URL starts with git@ or ssh://, clone the repository via git (using git+ssh) into a temporary directory, read the pyproject.toml, and then delete the temporary clone.
  • This re‑uses the host’s git binary and respects the user’s SSH configuration (keys, ssh‑config, etc.).
  • The cloned repo can be shallow (--depth 1) to minimise network traffic.
  • After extracting the [tool.ruff] section, the temporary directory is removed automatically.

3️⃣ Error handling

  • Provide clear error messages when authentication fails (e.g., 401 Unauthorized).
  • Offer a --no-auth flag to explicitly disable authentication for cases where the repo is public but the URL includes credentials.

4️⃣ Documentation

  • Update the --help output and README with examples for both token‑based and SSH usage, showing the command‑line syntax for each approach.
  • Document the generalised token options (--auth-token, --auth-header, and the supported env‑var conventions).

Impact

  • No breaking changes for existing public‑repo usage.
  • Adds a small, optional dependency on httpx’s authentication handling (already present).
  • Requires updating the CI pipeline to set a token when testing private‑repo functionality (can be gated behind a feature flag).

Additional notes

  • The feature should be gated behind a new --auth-token flag so that existing scripts continue to work unchanged.
  • Consider adding a unit test that mocks an authenticated request using respx to verify the token is sent correctly.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions