gadget is a tool that allows you to quickly inspect your Go source code. It's effectively a small layer of abstraction built on top of the Go AST package.
It inspects your go code, hence the name:
I was working on another project of mine and thought to myself, "It would be nice if I didn't have to constantly update this readme file every time I made a change." So, I started digging around Go's AST package and came up with gadget.
Yeah, I know. But, I didn't fully realise I was writing a crappier version of pkg.go.dev until about 90% into the project.
- Maybe you don't want people to leave your repository to understand the basics of your package's API.
- Maybe you want to present this data in a different, more personalised, format.
- Maybe you can use this to write a basic linter, or just learn more about Go AST.
It was fun to write and I use the tool almost daily. Perhaps you'll find it useful as well.
Binary releases are regularly published for the most common operating systems and CPU architectures. These can be downloaded from the releases page. Presentingly, gadget has been tested on, and compiled for, the following:
- Windows on
386,arm,amd64 - MacOS (
debian) onamd64,arm64 - Linux on
386,arm,amd64,arm64
Download the appropriate archive and unpack the binary into your machine's local $PATH.
Once added to your machine's local $PATH you can invoke gadget like so:
$ gadget --help
NAME:
gadget - inspect your code via a small layer of abstraction over Go's AST package
USAGE:
gadget [global options] command [command options] [arguments...]
VERSION:
v0.0.12
AUTHOR:
Wilhelm Murdoch <wilhelm@devilmayco.de>
COMMANDS:
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--source value, -s value path to the target go source file, or directory containing go source files. (default: ".")
--format value, -f value the output format of the results as json, template or debug. (default: "json")
--template value, -t value if the template format is selected, this is the path to the template file to use. (default: "README.tpl")
--help, -h show help (default: false)
--version, -v print only the version (default: false)
COPYRIGHT:
(c) 2022 Wilhelm Codes ( https://wilhelm.codes )
One of the primary goals of this project was to link example functions to their associated functions and methods. As I'm regenerating templates, I'd like to have the example body as well as the expected output to live alongside the associated function so I can easily reference it within the Go template.
Invoking the command with no flags will result in gadget searching for *.go files by recursively walking through the present working directory. Results will be displayed as a JSON object following this structure:
packages: a list of discovered packages.files: any*.gofile associated with the package.types: discovered types, eg; structs, interfaces, etc...fields: a list of fields associated with each type.
functions: functions and methodsexamples: mapped example functions ( if any )
values: explicitly-declared values, eg; constants and variables
A full example of the JSON object can be found in here.
When invoking gadget using the --format debug flag, you will get output representing all evaluated source code using ast.Print(...). Use this to follow the structure of the AST.
$ gadget --source /path/to/my/project --format debug
... heaps of AST output ...
1298 . . 15: *ast.FuncDecl {
1299 . . . Doc: *ast.CommentGroup {
1300 . . . . List: []*ast.Comment (len = 1) {
1301 . . . . . 0: *ast.Comment {
1302 . . . . . . Slash: sink/sink.go:81:1
1303 . . . . . . Text: "// GetPrivate is an accessor method that returns a dark secret:"
1304 . . . . . }
1305 . . . . }
1306 . . . }
1307 . . . Recv: *ast.FieldList {
1308 . . . . Opening: sink/sink.go:82:6
1309 . . . . List: []*ast.Field (len = 1) {
1310 . . . . . 0: *ast.Field {
1311 . . . . . . Names: []*ast.Ident (len = 1) {
1312 . . . . . . . 0: *ast.Ident {
1313 . . . . . . . . NamePos: sink/sink.go:82:7
1314 . . . . . . . . Name: "nst"
1315 . . . . . . . . Obj: *ast.Object {
1316 . . . . . . . . . Kind: var
1317 . . . . . . . . . Name: "nst"
1318 . . . . . . . . . Decl: *(obj @ 1310)
1319 . . . . . . . . }
1320 . . . . . . . }
1321 . . . . . . }
... heaps more AST outputUse Go's template engine, along with sprig, to generate technical documents or readme files ( like this one! ).
$ gadget --format template --template README.tpl > README.mdOr, without the --template ... flag as it will use README.tpl as the default template if it exists in the starting directory:
$ gadget --format template > README.md
The best way to understand this is by viewing the following "kitchen sink" examples:
sink/README.tplis a valid Go template.sink/README.mdwas generated using the specified Go template.
gadget makes use of Go's new generics support, so the minimum viable version of the language is 1.18.x. Ensure your local development environment meets this single requirement before continuing. There are also several build flags used when compiling the binary. These populate the output of the gadget --version flag.
$ git clone git@github.com:wilhelm-murdoch/go-gadget.git
$ cd gadget
$ make build
$ ./gadget --version
Version: v99.99.99, Stage: local, Commit: 9932cf9fdc90c0d8223ef85a0fc1ddfa99c28f95, Date: 10-04-2022All major functionality of gadget has been covered by testing. You can run the tests, benchmarks and lints using the following set of Makefile targets:
make test: run the local testing suite.make lint: runstaticcheckon the local source files.make bench: run a series of benchmarks and output the results ascpu.out,mem.outandtrace.outmake pprof-cpu: run a local webserver on port:8800displaying CPU usage stats.make pprof-mem: run a local webserver on port:8900displaying memory usage stats.make trace: view local tracing output from the benchmark run.make coverage: view testing code coverage for the local source files.
While gadget is meant to be used as a CLI, there's no reason you can't make use of it as a library to integrate into your own tools. If you were wondering, yes, this readme file was generated by gadget itself.
Exported Fields:
Name: The name of the field. #IsExported: Determines whether the field is exported. #IsEmbedded: Determines whether the field is an embedded type. #Line: The line number this field appears on in the associated source file. #Signature: The full definition of the field including name, arguments and return values. #Comment: Any inline comments associated with the field. #Doc: The comment block directly above this field's definition. #
NewField returns a field instance and attempts to populate all associatedfields with meaningful values.
Parse is responsible for browsing through f.astField, f.parent to populatethe current fields's fields. ( Chainable )
parseSignature determines the position of the current field within theassociated source file and extracts the relevant line of code. We only wantthe content before any inline comments. This will also replace consecutivespaces with a single space.
String implements the Stringer struct and returns the current package's name.
Exported Fields:
Name: The basename of the file. #Path: The full path to the file as specified by the caller. #Package: The name of the golang package associated with this file. #IsMain: Determines whether this file is part of package main. #IsTest: Determines whether this file is for golang tests. #HasTests: Determines whether this file contains golang tests. #HasBenchmarks: Determines whether this file contains benchmark tests. #HasExamples: Determines whether this file contains example tests. #Imports: A list of strings containing all the current file's package imports. #Values: A collection of declared golang values. #Functions: A collection of declared golang functions. #Types: A collection of declared golang types. #
NewFile returns a file instance representing a file within a golang package.This function creates a new token set and parser instance representing thenew file's abstract syntax tree ( AST ).
Examples:
package main
import (
"fmt"
"strings"
"github.com/wilhelm-murdoch/go-gadget"
)
func main() {
if file, err := gadget.NewFile("sink/sink.go"); err == nil {
file.Functions.Each(func(i int, function *gadget.Function) bool {
fmt.Printf("%s defined between lines %d and %d\n", function.Name, function.LineStart, function.LineEnd)
return false
})
}
}// Output:
// PrintVars defined between lines 30 and 34
// AssignCollection defined between lines 37 and 43
// PrintConst defined between lines 46 and 50
// NewNormalStructTest defined between lines 72 and 79
// GetPrivate defined between lines 82 and 84
// GetOccupation defined between lines 87 and 89
// GetFullName defined between lines 92 and 94
// notExported defined between lines 98 and 100
// NewGenericStructTest defined between lines 111 and 113
// GetPrivate defined between lines 116 and 118
// GetFullName defined between lines 121 and 123
// IsBlank defined between lines 126 and 126
package main
import (
"fmt"
"strings"
"github.com/wilhelm-murdoch/go-gadget"
)
func main() {
if file, err := gadget.NewFile("sink/sink.go"); err == nil {
file.Types.Each(func(i int, t *gadget.Type) bool {
if t.Fields.Length() > 0 {
fmt.Printf("%s is a %s with %d fields:\n", t.Name, t.Kind, t.Fields.Length())
t.Fields.Each(func(i int, f *gadget.Field) bool {
fmt.Printf("- %s on line %d\n", f.Name, f.Line)
return false
})
}
return false
})
}
}// Output:
// InterfaceTest is a interface with 1 fields:
// - ImplementMe on line 54
// EmbeddedStructTest is a struct with 1 fields:
// - Occupation on line 59
// NormalStructTest is a struct with 5 fields:
// - First on line 64
// - Last on line 65
// - Age on line 66
// - private on line 67
// - EmbeddedStructTest on line 68
// GenericStructTest is a struct with 4 fields:
// - First on line 104
// - Last on line 105
// - Age on line 106
// - private on line 107
package main
import (
"fmt"
"strings"
"github.com/wilhelm-murdoch/go-gadget"
)
func main() {
var buffer strings.Builder
if file, err := gadget.NewFile("sink/sink.go"); err == nil {
encoder := json.NewEncoder(&buffer)
if err := encoder.Encode(file.Values.Items()); err != nil {
fmt.Println(err)
}
}
fmt.Println(buffer.String())
}// Output:
// [{"kind":"const","name":"ONE","line":9,"body":"ONE = 1 // represents the number 1"},{"kind":"const","name":"TWO","line":10,"body":"TWO = 2 // represents the number 2"},{"kind":"const","name":"THREE","line":11,"body":"THREE = 3 // represents the number 3"},{"kind":"var","name":"one","line":16,"body":"one = \"one\" // represents the english spelling of 1"},{"kind":"var","name":"two","line":17,"body":"two = \"two\" // represents the english spelling of 2"},{"kind":"var","name":"three","line":18,"body":"three = \"three\" // represents the english spelling of 3"},{"kind":"var","name":"collection","line":27,"body":"var collection map[string]map[string]string // this should be picked up as an inline comment."}]
Parse is responsible for walking through the current file's abstract syntaxtree in order to populate it's fields. This includes imports, definedfunctions and methods, structs and interfaces and other declared values.( Chainable )
parsePackage updates the current file with package-related data.
parseImports is responsible for creating a list of package imports that havebeen defined within the current file and assinging them to the appropriateImports field.
parseFunctions is responsible for creating abstract representations offunctions and methods defined within the current file. All functions areadded to the Functions collection.
parseTypes is responsible for creating abstract representations of declaredgolang types defined within the current file. All findings are added to theTypes collection.
parseValues is responsible for creating abstract representations of variousgeneral values such as const and var blocks. All values are added to theValues collection.
walk implements the walk interface which is used to step through syntaxtrees via a caller-supplied callback.
String implements the Stringer struct and returns the current package's name.
GetAstAttributes returns the values associated with the astFile and tokenSetprivate fields. This is typically used for debug mode.
Exported Fields:
Name: The name of the function. #IsTest: Determines whether this is a test. #IsBenchmark: Determines whether this is a benchmark. #IsExample: Determines whether this is an example. #IsExported: Determines whether this function is exported. #IsMethod: Determines whether this a method. This will be true if this function has a receiver. #Receiver: If this method has a receiver, this field will refer to the name of the associated struct. #Doc: The comment block directly above this funciton's definition. #Output: If IsExample is true, this field should contain the comment block defining expected output. #Body: The body of this function; everything contained within the opening and closing braces. #Signature: The full definition of the function including receiver, name, arguments and return values. #LineStart: The line number in the associated source file where this function is initially defined. #LineEnd: The line number in the associated source file where the definition block ends. #LineCount: The total number of lines, including body, the interface occupies. #Examples: A list of example functions associated with the current function. #
NewFunction returns a function instance and attempts to populate allassociated fields with meaningful values.
Parse is responsible for browsing through f.astFunc, f.tokenSet and f.astFileto populate the current function's fields. ( Chainable )
parseReceiver attemps to assign the receiver of a method, if one even exists,and assigns it to the Function.Receiver field.
parseOutput attempts to fetch the expected output block for an examplefunction and pins it to the current Function for future reference. It assumesall comments immediately following the position of string "// Output:"belong to the output block.
parseLines determines the current function body's line positions within thecurrently evaluated file.
parseBody attempts to make a few adjustments to the *ast.BlockStmt whichrepresents the current function's body. We remove the opening and closingbraces as well as the first occurrent \t sequence on each line. Some peoplewill ask, "wHy dOn't yOu uSe tHe aSt pAcKaGe fOr tHiS" to which I answer,"Because, I'm lazy. We have the file, we know which lines contain the body."
parseSignature attempts to determine the current function's type and assignsit to the Signature field of struct Function.
String implements the Stringer struct and returns the current package's name.
Exported Fields:
Name: The name of the current package. #Files: A collection of golang files associated with this package. #---
NewPackage returns a Package instance with an initialised collection used forassigning and iterating through files.
String implements the Stringer struct and returns the current package's name.
Exported Fields:
Name: The name of the struct. #Kind: Determines the kind of type, eg; interface or struct. #LineStart: The line number in the associated source file where this struct is initially defined. #LineEnd: The line number in the associated source file where the definition block ends. #LineCount: The total number of lines, including body, the struct occupies. #Comment: Any inline comments associated with the struct. #Doc: The comment block directly above this struct's definition. #Signature: The full definition of the struct itself. #Body: The full body of the struct sourced directly from the associated file; comments included. #Fields: A collection of fields and their associated metadata. #
NewType returns an struct instance and attempts to populate all associatedfields with meaningful values.
Parse is responsible for browsing through f.astSpec, f.astType, f.parent topopulate the current struct's fields. ( Chainable )
parseLines determines the current struct's opening and closing linepositions.
parseBody attempts to make a few adjustments to the *ast.BlockStmt whichrepresents the current struct's body. We remove the opening and closingbraces as well as the first occurrent \t sequence on each line.
parseSignature attempts to determine the current structs's type and assignsit to the Signature field of struct Function.
parseFields iterates through the struct's list of defined methods topopulate the Fields collection.
String implements the Stringer struct and returns the current package's name.
GetLinesFromFile creates a byte reader for the file at the target path andreturns a slice of bytes representing the file content. This slice isrestricted the lines specified by the from and to arguments inclusively.This will return an empty byte if an empty file, or any error, is encountered.
WalkGoFiles recursively moves through the directory tree specified by pathproviding a slice of files matching the *.go extention. Explicitlyspecifying a file will return that file.
AdjustSource is a convenience function that strips the opening and closingbraces of a function's ( or other things ) body and removes the first \tcharacter on each remaining line.
Visit steps through each node within the specified syntax tree.
Exported Fields:
Kind: Describes the current value's type, eg; CONST or VAR. #Name: The name of the value. #Line: The line number within the associated source file in which this value was originally defined. #Body: The full content of the associated statement. #
NewValue returns a Value instance.
Parse is responsible for browsing through f.astIdent and f.tokenSet topopulate the current value's fields. ( Chainable )
String implements the Stringer struct and returns the current package's name.
Copyright © 2022 Wilhelm Murdoch.
This project is MIT licensed.
