PROJECT DEPRECATED! Instead see http://flowbase.org and http://scipipe.org for libraries built on this pattern
Go
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.gitignore
README.md
base_complementer.go
command_executor.go
file_reader.go
printer.go
stdin_reader.go
stdout_writer.go

README.md

UPDATE: Project Deprecated!

Instead, see:

  • FlowBase for a frame-work less library based on the ideas here, but further improved.
  • SciPipe for a scientific workflow library based on these ideas.

Example usage

This is an example program, utilising the StdInReader, BaseComplementer, and the Printer component, to do base complement processing of fasta file content that it piped to the program.

Example component, STDIN reader

First let's just have a look at how a component looks. Every component has one or more "in" and "outports", consisting of struct-fields of type channel (of some type that you choose. []byte arrays in this case). Then it has a run method that initializes a go-routine, and reads on the inports, and writes on the outports, as it processes incoming "data packets". Finally, as you can see in the OutChan() method, it provides a convenience method for each outgoing chan field, that initializes the channel and returns it, which can later be used for easier wiring of the network:

package glow

import (
	"bufio"
	"os"
)

type StdInReader struct {
	Out chan []byte
}

func (self *StdInReader) OutChan() chan []byte {
	self.Out = make(chan []byte, 16)
	return self.Out
}

func (self *StdInReader) Init() {
	go func() {
		scan := bufio.NewScanner(os.Stdin)
		for scan.Scan() {
			self.Out <- append([]byte(nil), scan.Bytes()...)
		}
		close(self.Out)
	}()
}

Connecting channels and components manually

And then, to connect such processes together, we just create a bunch of channels, a bunch of processes, and then stitch them together, and run it!

package main

import (
	"fmt"
	"github.com/samuell/glow"
)

const (
	BUFSIZE = 128 // Set a buffer size to use for channels
)

func main() {
	// Create channels / connections
	chan1 := make(chan []byte, BUFSIZE)
	chan2 := make(chan []byte, BUFSIZE)
	chan3 := make(chan int, 0)

	// Create components, connecting the channels
	stdInReader := new(glow.StdInReader)
	stdInReader.Out = chan1
	stdInReader.Init()

	baseCompler := new(glow.BaseComplementer)
	baseCompler.In = chan1
	baseCompler.Out = chan2
	baseCompler.Init()

	printer := new(glow.Printer)
	printer.In = chan2
	printer.DrivingBelt = chan3
	printer.Init()

	// Loop over the last channel, to drive the execution
	cnt := 0
	for i := range chan3 {
		cnt += i
	}
	fmt.Println("Processed ", cnt, " lines.")
}

Using OutChan() convenience methods

... we can save some keystrokes by using the functions defined for each outport, which initializes the outport with a channel, and returns the channel.

package main

import (
	"fmt"
	"github.com/samuell/glow"
)

func main() {
	// Create channels / connections
	fileReader := new(glow.FileReader)
	baseComplementer := new(glow.BaseComplementer)
	printer := new(glow.Printer)

	// Connect components (THIS IS WHERE THE NETWORK IS DEFINED!)
	baseComplementer.In = fileReader.OutChan()
	printer.In = baseComplementer.OutChan()

	// Initialize / set up go-routines
	fileReader.Init()
	baseComplementer.Init()
	printer.Init()

	// The InFilePath channel has to be created manually
	fileReader.InFilePath = make(chan string)
	fileReader.InFilePath <- "test.fa"

	// Loop over the last channel, to drive the execution
	cnt := 0
	for i := range printer.DrivingBeltChan() {
		cnt += i
	}
	fmt.Println("Processed ", cnt, " lines.")
}

Finally, to compile and run the program above, do like this:

go build basecomplement.go
cat SomeFastaFile.fa | ./basecomplement > SomeFastaFile_Basecomplemented.fa

Related projects / See also