API method to get list of available syntax definitions.

[Thom1729]

Overview

Sublime maintains an internal list of available syntax definitions. This list, I presume, is used to implement several important features, including:

• The Syntax menus.
• The Set Syntax entries in the command palette.
• include: scope:... in syntax definitions
• Determining a scope by file extension.
• Determining a scope by first line.

Unfortunately, there is no API method to access this list, making it difficult to implement some syntax-related features in plugins. Replicating the same information in a plugin would require reading and parsing a lot of files, including YAML 1.2 files that pyyaml doesn't understand. In addition, finding a syntax definition by file extension, first line, or scope would require determining the exact load order of syntax definitions, and this is surprisingly hard to get right.

Exposing this list via the API would be extremely useful.

Suggested API

sublime.get_syntax_definitions()

Return a list of objects in loading order with the following properties:

• path: the resource path (e.g. "Packages/JavaScript/JavaScript.sublime-syntax").
• name: the name as it would be displayed in the Syntax menu.
• hidden: whether the syntax is hidden (having hidden: true, being a hidden-tmLanguage).
• scope: the top-level key that is used for include: scope:....
• file_extensions: the top-level key, or an empty list.
• first_line_match: the top-level key, or None.

tmLanguage files that are "shadowed" by sublime-syntax files should be absent.

Hidden syntaxes should be present. Depending on the application, users can filter them out.

(It might be convenient to have API methods to find a syntax definition by scope, by file extension, or by first line, but these could be trivially implemented by users or by libraries in terms of sublime.get_syntax_definitions, whereas the reverse is not true.)

Uses

Using this API, package developers could:

• Provide a new quick-panel syntax selector with fancy features (live preview, including/excluding some packages, etc).
• Set a view's syntax to the proper choice for a given scope (regardless of what package provides that syntax).
• Verify that a specific syntax definition has been correctly loaded (e.g., in JS Custom's unit tests).
• Implement an application-specific syntax guesser that falls back to the base functionality.

Prior work

sublime_lib tries to provide this list. However:

• pyyaml can't handle sublime-syntax files, and ruamel.yaml is too slow, so we use some sketchy manual parsing code.
• Even given that, it's doing a lot of work, and it's not practical to cache the result.
• It may not be loading syntaxes in the correct order, making some features buggy.
• Broken or invalid syntaxes will still be listed.
• Overall, there is no guarantee that it handles corner cases the same way that Sublime does.

[Thom1729]

I ran into another example application today. I wrote a command that creates a new view with text from a template:

import sublime
import sublime_plugin

import yaml

def parse(template):
    i = template.find('\n---\n')
    ret = yaml.load(template[:i])
    ret['text'] = template[i+5:]
    return ret

class NewWithTemplateCommand(sublime_plugin.WindowCommand):
    def run(self):
        templates = [
            parse(sublime.load_resource(path))
            for path in sublime.find_resources('*.sublime-template')
        ]

        def callback(i):
            if i == -1: return
            text = templates[i]['text']
            view = self.window.new_file()
            view.run_command('insert_snippet', {'contents': text})

        self.window.show_quick_panel(
            [
                [ template['name'], template['description'] ]
                for template in templates
            ],
            callback
        )

Example template:

name: Test!
description: This is a test template.
---
Hello, ${0:World}!

An obvious omission in this system is a way to specify a syntax for the new view. I could allow templates to specify a syntax definition file (e.g. Packages/JavaScript/JavaScript.sublime-syntax), but a much better approach would be to have the template specify a scope (e.g. source.js). Given the suggested API function, it would be easy to find the appropriate syntax for the given scope. Additionally or alternatively, a separate API method sublime.get_syntax_for_scope(scope) would be useful.

[wbond]

Based on internal discussions, we wouldn't be able to provide file_extensions or first_line_match. Actually using file_extensions properly is rather non-obvious and first_line_match requires Oniguruma.

[deathaxe]

ST 4050+ provides sublime.list_syntaxes() to list all available syntaxes and sublime.find_syntax() to find one for certain files or by content.

• sublime.find_syntax("/path/source.py") returns 'Packages/Python/Python.sublime-syntax'
• sublime.find_syntax("", "#!/usr/bin/sh") returns 'Packages/ShellScript/Bash.sublime-syntax'