Enhance check command with structured output formats (--output-format)

[peterfarrell]

Code of Conduct

• I agree to follow Django's Code of Conduct

Feature Description

The check command is an invaluable tool for ensuring project health. However, its current string-based output to stderr makes it difficult to parse and integrate into modern CI/CD pipelines and external tooling (e.g., Code Climate, GitHub Actions, GitLab CI). This proposal suggests adding a new optional argument, --output-format, to allow the check command to produce structured, machine-readable output. Additionally, it proposes a --exit-zero flag to prevent CI jobs from failing on warnings.

Problem

Currently, when a check command is run, any violations are outputted as human-readable strings. For many CI/CD workflows, this requires complex and brittle string parsing to extract meaningful data about the issues found. This prevents Django check output from being directly ingested by code quality and security scanning tools, such as those that use the Code Climate or SARIF formats.

By providing structured output formats, developers can:

• Automatically upload code quality reports to GitLab, making them visible directly in merge requests.
• Annotate GitHub Actions pull requests with inline error messages.
• Ingest security vulnerability data into tools that support SARIF format.
• Programmatically process check results without relying on fragile string manipulation.

Request or proposal

proposal

Additional Details

No response

Implementation Suggestions

1. --output-format argument
   Add a new optional argument, --output-format, to the check command. The command will retain its existing default behavior for backwards compatibility.

Suggested formats:

• full (default): This option maintains the current behavior, outputting all messages (including stdout and stderr) as human-readable strings.
• json: This format would output a JSON array of objects. Each object would represent a single check violation and contain fields such as code, message, level (INFO, WARNING, ERROR), hint (if available), and path (if the check is tied to a specific file). This is a generic, machine-readable format that could be easily consumed by any third-party tool.
• github: This format would output messages formatted for GitHub Actions' workflow commands, specifically for creating file annotations. This would allow check violations to appear directly in the "Files changed" tab of a pull request.
• gitlab: This format would output a Code Climate-style JSON report, which is a standard format used by GitLab CI/CD's "Code Quality" feature.

Example of a --output-format github annotation:

::error file=some_file.py,line=42,title=my.check.W101::A custom check warning here.

2. --exit-zero flag
   Currently, check returns a non-zero exit code if it finds any errors. This is often the desired behavior for enforcing code quality. However, for a CI job that just reports on code quality, a non-zero exit code can be undesirable as it will fail the entire job. This proposal suggests adding an optional --exit-zero flag.

• With --exit-zero, the check command will exit with a status code of 0, even if errors or warnings are found.
• A non-zero exit code would only occur if the command terminates abnormally due to an unhandled exception (i.e., not a check violation).

Use Case Examples

GitHub Actions Workflow:

name: Django Check
on: [push]
jobs:
  run-check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.x'
      - name: Install dependencies
        run: pip install -r requirements.txt
      - name: Run Django check
        run: python manage.py check --output-format github --exit-zero

[github-actions]

Thank you peterfarrell for sharing your idea! We have a lot of them so please be patient. You can see the current queue here.

Community instructions

For commenters, please use the emoji reactions on the issue to express support, and/or concern easily. Please use the comments to ask questions or contribute knowledge about the idea. It is unhelpful to post comments of "I'd love this" or "What's the state of this?"

Reaction Guide

• 👍 This is something I would use
• 👎 This is something that would cause problems for me or Django
• 😕 I’m indifferent to this
• 🎉 This is an easy win

[LilyFirefly]

This sounds useful to me. I'd lean towards only including the default and json modes you suggest, rather than also including github and gitlab. There might be some value in making this pluggable, but it might be simpler for github or gitlab support to be added by a third-party package just wrapping the json output.

I'm also open to the --exit-zero flag, though I think these two ideas could be evaluated separately. It might make sense to split this issue in two.

[peterfarrell]

I appreciate the feedback! The --exit-zero flag is particularly important because a non-zero exit code currently causes CI/CD jobs to fail, which makes it much harder to use the new output formats for reporting without halting the build.

Regarding the specific formats, github and gitlab are already standard output formats for popular tools like Ruff (see output-format) and Hadolint (format), which highlights the value of including them for native integration with these platforms.

[knyghty]

I would quite like to have (at least) the github format. It's what we use and it's where a plurality (at least) of people are. But also because I think it makes a really good example of how to extend it.

If that's not accepted for whatever reason, the github format could at least be documented, but not included, so it's somewhat drop-in and also documenting how to extend it in a practical way, and it's easier to remove if people move away from GitHub (looking at you, bpython support in the shell command).

[peterfarrell]

I agree that additional output formats should be considered out of scope for now.

My initial suggestion for multiple formats was driven by a desire for "batteries included" integration into CI/CD, as seen with tools like Ruff or Hadolint, which use diverse output formats to provide code quality feedback.

However, a single, stable machine-readable format is sufficient for automation. Limiting the scope drastically reduces the ongoing KTLO (Keep The Lights On) maintenance overhead if those external formats were to change however it does it by shifting it to third-party library adds-ons.

What are the next steps? I'd like to offer an PR but I don't want to expend effort on a feature request that does not have agreement.