Provide Go template functions to format table output

[heaths]

Describe the feature or problem you’d like to solve

Related to #3487 there was no great way to provide table output like you do for most (or perhaps all) your other commands like gh issue list. You expose a color function and I can use printf to align and pad strings, but what would be really great is a table function.

Proposed solution

The Docker CLI lets you specify a table function that works like this:

docker images --format "table {{.Repository}}\t{{.Tag}}"

It would then output something like:

REPOSITORY                               TAG
azuresecuritykeyvaultcertificatestests   latest
dotnet-sdk                               5.0-ubuntu-18.04
jupyter/base-notebook                    latest

This is done via, mostly, https://github.com/docker/cli/blob/daf5f126ad698f34548c21f672450afa5754b7a2/cli/command/formatter/formatter.go. It is effectively like {{range}} but splits on tabs and determines optimal width automatically.

Alternatively, if there was some way to declare the scope of nodes to format as a table automatically, perhaps you could - when specified - format that with survey as you do your own table output.

[heaths]

Looking through the built-in formatting more, I wonder if this could instead be used for built-in command output rather than manually formatting the output. The crux of Docker's table template function is "text/tabwriter" which uses elastic tabstops, so even paginated results should align properly. I'm interested in taking a crack at this if there's interest in accepting it. At the minimum, I think making it available for gh api --template would be a good start, and whether to adopt it for built-in commands could be a separate discussion. You could combine that with your existing color functions, though it seems you'll need to add a couple more color options to match what you use currently for built-in command colored output.

[mislav]

@heaths I'd be interested in having a table template function!

I wonder if this could instead be used for built-in command output rather than manually formatting the output.

If our templating support gets full-featured enough, we could theoretically switch built-in output to the template approach. That would also serve as a starting template for someone who would like to tweak the output in their own scripts. But for now, I'd propose expanding --template functionality first, then exploring if built-in commands can switch to it separately.

[vilmibm]

I am +1 on these ideas but defer to @mislav on the details as he is the one who's been thinking most about programmatic output like this.

[mislav]

@heaths You are welcome to send a PR adding a table helper. I'm interested in how it will work 👍

[heaths]

I'll try to take a look this weekend. I need to play around with things a little (probably just a scratch program) to see if {{with pipeline}} will work since your GraphQL results typically have the juicy bits nested lower. Using Docker's table as a pattern, it only wants leaf nodes and seems to expect strings. If {{with}} doesn't work, may need to tweak this table function a little more to handle nesting, but I don't foresee a problem.

[heaths]

After playing around with a few options last night, I think this might be better served by a PR to the text/template package itself. Docker's "function" is really just a hack: if the template starts with "table" (or a couple others like "raw"), that prefix is stripped from the template and it does some simple string transforms on the remaining pipeline. It works well when your output is already an array, but if you're array is arbitrarily nested (like a GraphQL query might often end up), it wouldn't work. Even if we scanned the string for any occurrence of "table" (or similar), there's no obvious end. The text/template/parse.Tree that is compiled is where the change really needs to happen. I can also make it work with 2 templates, but it seems like an even bigger hack and not very user-friendly to have two arguments for a table, like --template '{{with .Nodes}}{{template "table"}}{{end}} then, if you use a table, --template-table '{{range .}}{{.Id}}{{"\t"}}{{.Name}}{{end}}'.

For now, let me see if there's any interest in a change to the text/template package. I think it could be an all-around useful addition.

[mislav]

Thanks for exploring all this! I have little faith that the proposal to text/template will be accepted, since I think you will be told to just implement this programmatically (either via custom helper function or nested templates), but even if it doesn't get into Go, I think we can just invent some approach ourselves, even if it's slightly more clumsy than a native approach would be. For example:

{{range .}}{{ tableRow .number .title .author }}{{end}}
{{tableRender}}

[heaths]

I've been considering some similar approaches under the assumption it won't be accepted because it actually emits formatted text, which seems out of scope.

One idea was similar to what you suggested, but the problem is that you can't pass fields to functions if, for example, you wanted to color text. Maybe some was to do that would be more acceptable, like inline pipelines via syntax like (partial) {{tableRow (.Id | color "green")}}.

If not even that, I've been thinking that we could pre-parse the template and, like Docker CLI, manipulate it to make another named template internally. I'll continue to play around with this in my scratch program.

Edit: updated proposal with this alternative suggestion: golang/go#45752 (comment)

[heaths]

Apparently parenthesis already work. While perusing through the text/template/parse code and making a few preliminary changes to support my proposal, I failed to see how parsed parenthesis were used.

This commit shows my idea above already works:

I'll proceed with this course of action, but would also like to add a couple more helper functions that I think will also help in the future if we wanted to replace the built-in formatting with this new set of functions, e.g. truncate, though I think I might reverse the uint and string params so you can do either {{truncate 30 .Title | color "green"}} or {{.Title | truncate 30 | color "green"}}.

[heaths]

After further consideration I propose support for multiple tables, which could be beneficial down the road. Effectively, this:

`{{table}}{{range .Books}}{{row (.Title | truncate 30 | green) .ISBN13}}{{end}}{{endTable}}`

We could use {{tableRow}} as well, but with {{table}} I figured it would be fairly obvious and slightly prefer brevity over specificity in a rather limited scope like this.

{{table}} would set the current table for subsequent {{row}} calls, while {{endTable}} would flush the current table an nilify it. Additionally, {{table}} could maybe take some options to pass to text/tabwriter.

[mislav]

@heaths Looks pretty good! I think you're definitely on to something here.

Does text/tabwriter work when the lines passed in contain ANSI escape sequences? I would have guessed that it doesn't recognize these escape sequences as non-printable and that they can mess up horizontal alignment.

[heaths]

C-style escape sequences within the text won't be interpreted, but literal escapes will. While the web forms appear to prevent this (or at least make it harder), I was able to add an issue with literal escapes in both the title and text. Seems I'll have to create a replacer that perhaps JS-encodes all but \ux1b used for vterm sequences.

[heaths]

Actually, looks like this shouldn't be a problem:

$ gh api graphql -f query='query {
  repository(name: "templ", owner: "heaths") {
    issue(number: 1) {
      number
      title
      bodyText
    }
  }
}'

{
  "data": {
    "repository": {
      "issue": {
        "number": 1,
        "title": "Title:\u0008 with escape",
        "bodyText": "This\tbody has\nescape sequences"
      }
    }
  }
}

[heaths]

Working on some tests and docs, but a preview of using a table template for gh issues list:

bin/gh issue list --json number,title,labels,updatedAt --template '{{table}}{{range .}}{{row (.number | printf "#%v" | color "green") (.title | ellipsis 50) (.labels | pluck "name" | join ", " | printf "(%s)" | color "gray") (timeago .updatedAt | color "gray")}}{{end}}{{endTable}}'

[pierrejoye]

For those wondering what gh template ended as. Here is an example:

gh issue list --milestone "GD 2.3.3"  -s all  --json number,title,url -t'{{range .}}{{tablerow .number .url .title}}{{end}}{{tablerender}}'

• "range ." loop over all elements of the collections.
• tablerow prepares the table using the element of each item of the collections (the names are the ones provided to --json option
• tablerender will render the final table
• any space between {{ }} will be used as is
• it is not possible to use anything outside {{}} as it will be interpreted as option for the cmd or error out (it would be nice to be able to pass a file btw)

Hope that helps.

PS: It would be awesome to add that to the doc here and there.

[heaths]

I did add that to the formatting help topic, though that page you reference doesn't seem to be rendering it correctly:

With `--template`, the provided Go template is rendered using the JSON data as input.
For the syntax of Go templates, see: https://golang.org/pkg/text/template/

The following functions are available in templates:
- `autocolor`: like `color`, but only emits color to terminals
- `color <style> <input>`: colorize input using https://github.com/mgutz/ansi
- `join <sep> <list>`: joins values in the list using a separator
- `pluck <field> <list>`: collects values of a field from all items in the input
- `tablerow <fields>...`: aligns fields in output vertically as a table
- `tablerender`: renders fields added by tablerow in place
- `timeago <time>`: renders a timestamp as relative to now
- `timefmt <format> <time>`: formats a timestamp using Go's Time.Format function
- `truncate <length> <input>`: ensures input fits within length

EXAMPLES
  # format issues as table
  $ gh issue list --json number,title --template \
    '{{range .}}{{tablerow (printf "#%v" .number | autocolor "green") .title}}{{end}}'

  # format a pull request using multiple tables with headers
  $ gh pr view 3519 --json number,title,body,reviews,assignees --template \
    '{{printf "#%v" .number}} {{.title}}

    {{.body}}

    {{tablerow "ASSIGNEE" "NAME"}}{{range .assignees}}{{tablerow .login .name}}{{end}}{{tablerender}}
    {{tablerow "REVIEWER" "STATE" "COMMENT"}}{{range .reviews}}{{tablerow .author.login .state .body}}{{end}}
    '

Also, you don't need {{tablerender}} unless you want to render any text - plain or even another table - after that. The table is automatically rendered after all records are processed if not rendered explicitly first.

[pierrejoye]

Thanks @heaths :)

[rocketraman]

How would I add, for example, a comma-separated list of label names into a tablerow? I've tried:

.labels.name

but get:

template: :1:135: executing "" at <.labels.name>: can't evaluate field name in type interface {}

I've also tried various forms of nesting {{range}} instead tablerow and using join but nothing seems to work. I've got an equivalent jq expression working, but would rather use a go template for better formatting.

[heaths]

Go templates don't expand slices like that. But you can use some scope operator like range or a specific operator like pluck for a slice of labels like so:

https://gist.github.com/heaths/5c7ba9dda978b3f8cb306be3a899c926#file-config-yml-L7

[rocketraman]

@heaths Thank you, that worked great.