Use tree-sitter highlighting for pycon (Python console) code blocks

[clintval]

🏷️ Discussion Type

Product Feedback

💬 Feature/Topic Area

Other

Body

Use tree-sitter highlighting for pycon (Python console) code blocks

Fenced code blocks tagged python are highlighted with GitHub's tree-sitter
highlighter. Blocks tagged pycon (Linguist's "Python console", scope
text.python.console) fall back to the MagicPython TextMate grammar. So the
code after >>> and ... prompts gets far less highlighting than the same
code in a python block: class names, calls, attributes and keyword
arguments stay uncolored.

To reproduce, render this with POST /markdown (mode gfm), once tagged
pycon and once tagged python without the prompt:

>>> transcript = Bed12("chr1", start=100, strand=BedStrand.Positive)

The pycon output only uses pl-k, pl-s and pl-c1. The python output
also uses pl-en, pl-v and pl-s1. A full example is this README:
https://github.com/clintval/bedspec/blob/3c768d2/README.md

pycon is the usual way to show Python examples with their output, and many
projects test their READMEs as doctests. Right now authors have to choose
between examples that can be tested and readable highlighting.

Request: highlight the code lines of pycon blocks (those after >>> and
... ) with the same tree-sitter Python highlighter used for python blocks,
and leave output lines plain. Linguist maintainers have said tree-sitter
highlighting is managed by GitHub rather than Linguist
(github-linguist/linguist#7787).

[github-actions[bot]]

💬 Your Product Feedback Has Been Submitted 🎉

Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users.

Here's what you can expect moving forward ⏩

• Your input will be carefully reviewed and cataloged by members of our product teams. 
  • Due to the high volume of submissions, we may not always be able to provide individual responses.
  • Rest assured, your feedback will help chart our course for product improvements.
• Other users may engage with your post, sharing their own perspectives or experiences. 
• GitHub staff may reach out for further clarification or insight. 
  • We may 'Answer' your discussion if there is a current solution, workaround, or roadmap/changelog post related to the feedback.

Where to look to see what's shipping 👀

• Read the Changelog for real-time updates on the latest GitHub features, enhancements, and calls for feedback.
• Explore our Product Roadmap, which details upcoming major releases and initiatives.

What you can do in the meantime 💻

• Upvote and comment on other user feedback Discussions that resonate with you.
• Add more information at any point! Useful details include: use cases, relevant labels, desired outcomes, and any accompanying screenshots.

As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities.

Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐

[KasraK2K]

Summary of the Issue

You've identified an important feature request regarding syntax highlighting for pycon (Python console) code blocks. Here's the core problem:

The Problem

• Current behavior: Code blocks tagged as pycon use the MagicPython TextMate grammar for syntax highlighting, which provides limited color coding (mainly pl-k, pl-s, pl-c1 for keywords, strings, and constants)
• • Desired behavior: Use GitHub's tree-sitter Python highlighter (which supports pl-en, pl-v, pl-s1 for class names, attributes, and other identifiers) for the code lines after >>> and ... prompts, while keeping output lines plain

Why This Matters

• pycon is the standard format for documenting Python examples with their output
• • Many projects test their READMEs as doctests using this format
• • Currently, authors must choose between:
• • Testable examples (using pycon with poor highlighting)
• • Well-highlighted examples (using python without actual output)

The Solution

Implement tree-sitter-based syntax highlighting specifically for:

1. Lines starting with >>> (input prompts)
2. 2. Continuation lines starting with ...
3. 3. Leave output lines (without prompts) unformatted
      This would match the behavior already demonstrated in tools like John Paton's console highlighter example, which shows that pycon blocks can intelligently distinguish between input and output while applying proper syntax highlighting.

Technical Note

As you correctly noted, this requires GitHub's implementation rather than Linguist's changes, since tree-sitter highlighting is managed by GitHub's rendering system.

This is a valuable feature request that would significantly improve documentation readability without sacrificing testability.