goast is a Go AST utility with the aim of providing idiomatic meta-programming facilities for Go.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
examples
goast Add missing Len method of ImportSpecs Jun 29, 2016
.gitignore Add .gitignore Dec 19, 2014
LICENSE
README.md minor typo Jun 21, 2016
index.html Add goast.net index.html Mar 26, 2015

README.md

goast

goast is a Go AST (Abstract Syntax Tree) based meta-programming utility with the aim of providing safe, idiomatic code generation facilities by taking advantage of Go's native AST abilities and the go:generate build directive.

goast's core philosophies are

  • Compile time type safety
  • No runtime type casting (see previous)
  • Avoid runtime reflection
  • Prefer pure Go over syntax extensions
  • No text templates (see previous)
  • Prefer inference over annotation

The functionality of goast is currently built on the following axiom and proposition

  1. The empty interface (interface{}) can be replaced with any other type
  2. Any composite type composed at least partially of the empty interface (e.g. map[string]interface{}) can be replaced with any other composite type of the same structure with the empty interface swapped out for a concrete type (e.g. map[string][]int64)

Installing

Installing goast is as easy installing any other go package; Use the go get command to install from the canonical import path:

go get goast.net/x/goast

Usage

goast is designed as a general purpose command line utility for reading, examining, writing, and working with Go files and their AST. It follows a cli command subcommand pattern similar to git and go.

This document will focus primarily on a single subcommand goast write impl, but you can get more information on any other commands by typing goast help.

Please note: While functional, goast is still in an alpha/RFC phase so more subcommands may be added over time and subcommand structure may change (hopefully not)

Simple example

Consider the following generic implementation of a filtering method on a slice of empty interface

//file: slicefilter.go
package gen

type T interface{}
type Slice []T
func (s Slice) Where(fn func(T)bool) (result Slice) {
	for _, v := range s {
		if fn(v) {
			result = append(result, v)
		}
	}
	return
}

This code compiles, and accurately reflects the algorithm that any slice type might implement. There can be some initial confusion where a new gopher would expect the following to work

package main

import "fmt"

type Slice []interface{}

func (s Slice) Where(fn func(interface{}) bool) (result Slice) {
	for _, v := range s {
		if fn(v) {
			result = append(result, v)
		}
	}
	return
}

func main() {

	var s Slice = []int{1, 2, 3, 4, 5, 6, 7, 8, 9}
	evens := s.Where(func(val interface{}) bool {
		i := val.(int)
		return i%2 == 0
	})

	fmt.Printf("Evens: %+v\n", evens)
}

Our intrepid gopher is a little sad about needing to use type casting, but is surprised to discover the following compile error main.go:18: cannot use []int literal (type []int) as type Slice in assignment (Playground). Surely a slice of ints provides the same interface as a slice of empty interface? The explanation for this is that an []int cannot be assigned to an []interface{} because they have different memory layouts. This is unfortunate because algorithmically the code is correct and it seems intuitive that it could be interpreted that way.

It is possible to rewrite the above by using var s Slice = []interface{}{1, 2, 3, 4, 5, 6, 7, 8, 9}, but converting back and forth between slice types on top of already needing to do type casting is onerous and a great place for bugs to creep in.

goast write impl exploits the fact that you can write compilable, generic Go code in that manner to provide typesafe code generation of a specific implementation of that meta-code.

With goast, a developer only needs to provide the following code, and the go generate command will provide the rest

//file main.go
package main

import "fmt"

//go:generate goast write impl slicefilter.go

type Ints []int

func main() {
	var s Ints = []int{1, 2, 3, 4, 5, 6, 7, 8, 9}
	evens := s.Where(func(val int) bool { return val%2 == 0 })
	fmt.Printf("Evens: %+v\n", evens)
}

The go:generate build directive instructs goast to write an implementation of the code in slicefilter.go for the types provided in main.go, resulting in the following file being generated

//file ints_slicefilter.go
package main

func (s Ints) Where(fn func(int)bool) (result Ints) {
	for _, v := range s {
		if fn(v) {
			result = append(result, v)
		}
	}
	return
}

Playground

A reader following the code closely might have noticed that there is no longer a need for type casting within the callback function to Where. goast infers concrete type information from your source file and rewrites the the meta-code to be type-safe for your types, thus providing the ability to write correct, generic code once, and use it in an opt-in, type-safe manner.

A more complete slice iteration library can be seen in the iter package.

A Note About Paths: The examples above use relative paths to implment the code in a single, local file. goast also supports fetching code from the $GOPATH. The above example could be rewritten to use the iter package like so:

package main

import "fmt"

//go:generate goast write impl goast.net/x/iter

type Ints []int

func main() {
	var s Ints = []int{1, 2, 3, 4, 5, 6, 7, 8, 9}
	evens := s.Where(func(val int) bool { return val%2 == 0 })
	fmt.Printf("Evens: %+v\n", evens)
}

Complex Example

Vanilla iteration patterns aren't exciting enough for you?

Want something more Go-centric, maybe even concurrency or parallel oriented?

How about a Fan-Out/Fan-In Concurrent Pipeline?

If you're not familiar with pipelines, this is a great primer on the pattern and pitfalls of implementing it. The concept is strightforward, but it's not the kind of thing I can see trusting myself to write repeatedly.

With goast you can have type-safe pipeline facilities for any type by defining a channel for that type and generating from the corresponding meta-code. A simple implementation of this pattern is provided in the pipeline package.

The following is a simple example of concurrent, parallel squaring of a range of numbers

package main

import (
	"fmt"
	"runtime"
)

//go:generate goast write impl goast.net/x/pipeline
type IntPipe chan int

func init() {
	runtime.GOMAXPROCS(runtime.NumCPU())
}

func main() {
	done := make(chan bool)
	defer close(done)
	workers := runtime.NumCPU()

	var ch IntPipe = make(chan int)
	go func(ch IntPipe) {
		for i := 0; i < 1000; i++ {
			ch <- i
		}
	}(ch)

	//Fan squaring out over a number of workers and collect the first 20 results
	result := ch.
		Fan(done, workers, func(n int) int { return n * n }).
		Collect(done, 20)

	done <- true
	for _, i := range result {
		fmt.Println(i)
	}
}

Related Types

goast also allows for the generation of derived composite types based on the provided type information, and calls these Related Types. Unlike other types seen in examples so far, the end-user does not need to provide an implementation of the Related Types, only the concrete types used to replace the generic segments of it. To accomlish this, goast also needs to ability to generate new, unique names for Related Types, and allows the meta-code developer to specify how these names should look using a convention of leaving a space for type name replacement with an '_' in the identifier of the Related type

The easy example of this would be a slice sorter Related Type

package sort // import "goast.net/x/sort"

import "sort"

type I interface{}
type Slice []I

//Related Type: Enables sorting as a slice operation
type _Sorter struct {
	Slice
	LessFunc func(I, I) bool
}

func (s _Sorter) Less(i, j int) bool {
	return s.LessFunc(s.Slice[i], s.Slice[j])
}

func (s Slice) Len() int {
	return len(s)
}

func (s Slice) Swap(i, j int) {
	s[i], s[j] = s[j], s[i]
}

//Sorts the slice in-place using the related type sorter
func (s Slice) Sort(less func(I, I) bool) {
	sort.Sort(_Sorter{s, less})
}

When that is compiled against a type such as type Contacts []*Contact, the Contacts type gets a Sort(func(*Contact, *Contact)bool) method, and a Related Type that looks like the following is generated

type ContactsSorter struct {
	Contacts
	LessFunc	func(*contact, *Contact) bool
}

This enables easy arbitrary sorts

//go:generate goast write impl goast.net/x/sort

type Contact struct {
	Id    int
	First string
	Last  string
	Email string
}

type Contacts []*Contact

func main() {

	set := getContacts()
	
	set.Sort(func(a, b *Contact)bool {
		return a.Id < b.Id
	})

	set.Sort(func(a, b *Contact)bool {
		return a.Last > b.Last
	})
}

Multiple Type Parameters

Any number of types are allowed to be specified in template code. For each desired type, assign a new identifier to interface{}.

The following defines a type that maps any type to a slice of any other type

type K interface{}
type V interface{}
type SliceMap map[K][]V

Partial Structs

goast write impl also allows for an amount of structural duck-typing with partially defined structs. This provides a type-safe ability to say "If it looks like X, it can do Y".

A toy example of this would a 'quittable struct'

//quittable.go
package main

type T struct {
	quit chan bool
}

func (t *T) Quit() {
	t.quit <- true
}

Implemented against

package main

//go:generate goast write impl quittable.go
type Process {
	data <- chan string
	value int
	quit chan bool

}

File Naming Control

It can be useful for organizational purposes for generated files to have a naming scheme that identifies them as a generated file. goast provides the --prefix and --suffix flags on the impl sub-command to control this behavior.

Example: The following produces a file named gen_ints_iter.go

package main

//goast write impl --prefix=gen_ goast.net/x/iter

type Ints []int

Roadmap

goast is still in an alpha/RFC stage of development. Some features that are planned for v1 are

History and acknowledgements

I originally got interested in code generation as a method of genericty in Go when I learned about the gen package from clipperhouse. When to Go team first announced Go 1.4 and the go:generate proposal, it planted the seed of the idea for goast in my brain and initiated my research into how it might work. In the intervening time I found gotgo, and more recently (and also quite close to my goals) the gonerics package. As projects in the same area as what goast explores, they were all valuble for research and inspiration, as well for providing a contrast against which I wanted to differentiate.

Licence

goast is released under a GPL v2 licence.