Restructure generated file structure

.gitignore

@@ -5,6 +5,10 @@
 # session
 Session.vi
 
+# generated example services
+_example/*
+!_example/echo.proto
+
 # Prevents accidentally including a problematic folder within the `vendor/`
 # folder.
 vendor/github.com/Sirupsen/logrus/examples/

DEVELOPING.md

@@ -63,8 +63,6 @@ overwrite user code in the /NAME-service/handlers directory
 
 Additional internal packages of note used by these programs are:
 
-- `astmodifier`, located in `gengokit/astmodifier/`, used to
-  modify go files in place, and used by gengokit`
 - `deftree`, located in `deftree/`, which makes sense of the protobuf file
   passed to it by `protoc`, and is used by `gengokit` and
   `gendoc`

README.md

@@ -46,7 +46,7 @@ add [http annotations](
 https://github.com/googleapis/googleapis/blob/928a151b2f871b4239b7707e1bb59258df3fe10a/google/api/http.proto#L36)
 for HTTP 1.1/JSON transport!
 
-Then you open the `handlers/server/server_handler.go`,
+Then you open the `handlers/handlers.go`,
 add you business logic, and you're good to go.
 
 Here is an example service definition: [Echo Service](./_example/echo.proto)

TUTORIAL.md

@@ -69,38 +69,40 @@ In your terminal, go to the folder containing echo.proto and run `truss *.proto`
 ```
 .
 ├── echo-service
-│   ├── docs
-│   │   └── docs.md
-│   ├── generated
-│   │   └── ...
-│   ├── handlers
-│   │   └── server
-│   │       └── server_handler.go
-│   ├── middlewares
-│   │   └── ...
-│   ├── echo-cli-client
-│   │   └── client_main.go
-│   ├── echo-server
-│   │   └── server_main.go
-│   └── echo.pb.go
-├── echo.proto
-```
-From the top down, within echo-service/:
-  - docs/ contains the generated documentation of the service API
-  - generated/ contains the wiring and encoding protocols necessary for service communication
-  - handlers/server/server_handler.go is populated with stubs where you will add the business logic
-  - middlewares/ is where you can put the middlewares (NOP by default)
-  - echo-cli-client/ contains the client side CLI (useful for testing)
-  - echo-server/ contains the service main, which you will build and run shortly
-  - echo.pb.go contains the RPC interface definitions and supporting structures that have been translated from echo.proto to golang
+|   ├── cmd
+|   │   ├── echo
+|   │   │   └── main.go
+|   │   └── echo-server
+|   │       └── main.go
+|   ├── docs
+|   │   └── docs.md
+|   ├── echo.pb.go
+|   ├── handlers
+|   │   ├── handlers.go
+|   │   └── hooks.go
+|   ├── middlewares
+|   │   ├── endpoints.go
+|   │   └── service.go
+|   └── svc
+|       └── ...
+└── echo.proto
+```
+From the top down, within `echo-service/`:
+  - `docs/` contains the generated documentation of the service API
+  - `svc/` contains the wiring and encoding protocols necessary for service communication (generated code)
+  - `handlers/handlers.go` is populated with stubs where you will add the business logic
+  - `middlewares/` is where you can put the middlewares (NOP by default)
+  - `cmd/echo/` contains the client side CLI (useful for testing)
+  - `cmd/echo-server/` contains the service main, which you will build and run shortly
+  - `echo.pb.go` contains the RPC interface definitions and supporting structures that have been translated from `echo.proto` to golang
 
 If you try to build and run your service now, it will respond with empty messages. There is no business logic yet! We shall add it in the next step.
 
 You can safely modify only the files in handlers/ and middlewares/. Changes to any other files will be lost the next time you re-generate the service with truss.
 
 ## Implement business logic
 
-Open `handlers/server/server_handler.go` using your favorite editor. Find the Echo function stub. It should look like this:
+Open `handlers/handlers.go` using your favorite editor. Find the Echo function stub. It should look like this:
 ```
 func (s echoService) Echo(ctx context.Context, in *pb.EchoRequest) (*pb.EchoResponse, error) {
 	var resp pb.EchoResponse
@@ -125,25 +127,25 @@ func (s echoService) Echo(ctx context.Context, in *pb.EchoRequest) (*pb.EchoResp
 ## Build/Run the client and server executables
 
 From the directory containing echo.proto run
-`go build echo-service/echo-cli-client/client_main.go` and
-`go build echo-service/echo-server/server_main.go`
+`go build echo-service/echo` and
+`go build echo-service/echo`
 
 Create another terminal window to run the server in, navigate to the same directory and launch the server:
-`./server_main`
+`./echo-server`
 When server starts up, you should see something like this:
 ```
-ts=2016-12-06T23:25:14Z caller=server_main.go:55 msg=hello
-ts=2016-12-06T23:25:14Z caller=server_main.go:106 transport=HTTP addr=:5050
-ts=2016-12-06T23:25:14Z caller=server_main.go:98 transport=debug addr=:5060
-ts=2016-12-06T23:25:14Z caller=server_main.go:124 transport=gRPC addr=:5040
+ts=2016-12-06T23:25:14Z caller=main.go:55 msg=hello
+ts=2016-12-06T23:25:14Z caller=main.go:106 transport=HTTP addr=:5050
+ts=2016-12-06T23:25:14Z caller=main.go:98 transport=debug addr=:5060
+ts=2016-12-06T23:25:14Z caller=main.go:124 transport=gRPC addr=:5040
 
 ```
 The server is now waiting for incoming messages.
 At this point we can send a request to the server via networking tools (telnet, curl) and construct message directly, or we can use the client CLI.
 
-Let's do the latter, in your first terminal. To learn how to launch client with proper parameters run `./client_main -help`. The printout will tell you what methods the service supports (-method flag options) and all the additional flags that must be set to call a certain method (flags of format -method_name.parameter_name).
+Let's do the latter, in your first terminal. To learn how to launch client with proper parameters run `./echo -help`. The printout will tell you what methods the service supports and all the additional flags that must be set to call a certain method
 
-Now run `./client_main -method echo -echo.in “hello microservices”`
+Now run `./echo echo -in “hello microservices”`
 The client terminal will display messages that were sent and received.
 
 You can also specify the address to send messages to via -grpc.addr or -http.addr flags (e.g. `-grpc.addr localhost:5040`), should you want to change the port the server runs on, or test it out on separate machines.
@@ -152,14 +154,14 @@ To shutdown the server, press Ctrl+C in the server terminal
 
 ## Implement more things!
 
-The following is left as an exersise to the reader:
+The following is left as an exercise to the reader:
   - Implement logic for the Louder call
     - code the logic inside the stub
     - now separate this logic into an unexported helper function
   - Define a new RPC call in echo.proto
     - regenerate service with truss, check that your old logic remains
     - implement the logic for your new call in a separate package, place it ouside of echo-service
-    - wire in the new logic by importing the package in the server_handler.go
+    - wire in the new logic by importing the package in the `handlers.go`
   Suggestion: Save everything the service hears and echo all of it back. See repeated types (protobuf), package variables and init() function (golang).
   - Remove an RPC call definition from echo.proto
   	- regenerate service with truss, verify that the call no longer exists
@@ -173,16 +175,16 @@ The following is left as an exersise to the reader:
 
 You can control the location of the output folders for your service by specifying the following flags when running truss
 ```
-  -svcout {go-style-package-path to where you want the {Name}-service folder to be}
-  -pbout {go-style-package-path to where you want the *.pb.go interface definitions to be}
+  --svcout {go-style-package-path to where you want the contents of {Name}-service folder to be}
+  --pbout {go-style-package-path to where you want the *.pb.go interface definitions to be}
 ```
 
-Note: “go-style-package-path” means exactly the style you use in your golang import statements, relative to your $GOPATH. This is not your system file path, nor it is relative to location of the *.proto file; the start of the path must be accessible from your $GOPATH. Also no “/” at the end.
+Note: “go-style-package-path” means exactly the style you use in your golang import statements, relative to your $GOPATH. This is not your system file path, nor it is relative to location of the *.proto file; the start of the path must be accessible from your $GOPATH.
 For example:
 ```
-truss -pbout truss-demo/interface-defs -svcout truss-demo/service echo.proto
+truss --pbout truss-demo/interface-defs --svcout truss-demo/service echo.proto
 ```
-Executing this command will place the *.pb.go files into `$GOPATH/truss-demo/interface-defs/`, and the entire echo-service directory (excepting the *.pb.go files) to `$GOPATH/truss-demo/service/`.
+Executing this command will place the *.pb.go files into `$GOPATH/truss-demo/interface-defs/`, and the entire echo-service contents (excepting the *.pb.go files) to `$GOPATH/truss-demo/service/`.
 
 ## Middlewares
 

USAGE.md

@@ -1,66 +1,11 @@
 # Using truss
-
-## File structure
-
-Start with your service definition file, let's name it `svc.proto`. 
-For a more detailed example of a simple service definition see [echo.proto](./_example/echo.proto),
-but for now we only care about the following structure:
-```
-// The package name determines the name of the directories that truss creates
-package NAME;
-
-// RPC interface definitions
-...
-```
-
-The current directory should look like this:
-
-```
-.
-└── svc.proto
-```
-
-Run truss on your service definition file: `truss svc.proto`.  
-Upon success, `NAME-service` folder will be created in your current directory. 
-(`NAME-service`, where NAME is the name of the package defined in your definition file.)
-
-Your directory structure will look like this:
-
-```
-.
-├── NAME-service
-│   ├── docs
-│   │   └── docs.md
-│   ├── generated
-│   │   └── ...
-│   ├── handlers
-│   │   └── server
-│   │       └── server_handler.go
-│   ├── middlewares
-│   │   └── ...
-│   ├── NAME-cli-client
-│   │   └── client_main.go
-│   ├── NAME-server
-│   │   └── server_main.go
-│   └── svc.pb.go
-├── svc.proto
-```
-
-Now that you've generated your service, you can install the generated binaries
-with `go install ./...` which will install `NAME-cli-client` and `NAME-server`,
-where NAME is the name of the package defined in your definition file.
-
-To add business logic, edit the `server_handler.go` file in `./NAME-service/handlers/server/`.
-
-To add middlewares, edit ... (TODO)
-
 ## Our Contract
 
 1. Modify ONLY the files in `handlers/` and `middlewares/`.
 
- User logic can be imported and executed within the functions in the handlers. It can also be added as _unexported_ helper functions in the handler file. 
+ User logic can be imported and executed within the functions in the handlers. It can also be added as _unexported_ helper functions in the handler file.
 
- Truss will enforce that exported functions in `server_handler.go` conform to the rpc interface defined in the service *.proto files. All other exported functions will be removed upon next re-run of truss. 
+ Truss will enforce that exported functions in `handlers/handlers.go` conform to the rpc interface defined in the service *.proto files. All other exported functions will be removed upon next re-run of truss.
 
 2. DO NOT create files or directories in `NAME-service/`
  All user logic must exist outside of `NAME-service/`, leaving organization of that logic up to the user.

cmd/_integration-tests/cli/cli_test.go

@@ -29,18 +29,27 @@ func TestMain(m *testing.M) {
 	}
 
 	basePath = filepath.Join(wd, definitionDirectory)
+	// Create a standalone copy of the 'basic' service binary that persists
+	// through all the tests. This binary to be used to test things like flags
+	// and flag-groups.
+
+	exitCode := 1
+	defer func() {
+		if exitCode == 0 {
+			cleanTests(basePath)
+		}
+		os.Exit(exitCode)
+	}()
 
 	clean := flag.Bool("clean", false, "Remove all generated test files and do nothing else")
 	flag.Parse()
 	if *clean {
-		cleanTests(basePath)
-		os.Exit(0)
+		exitCode = 0
+		return
 	}
 
-	// Create a standalone copy of the 'basic' service binary that persists
-	// through all the tests. This binary to be used to test things like flags
-	// and flag-groups.
-	exitCode := 0
+	// Cleanup so that cp works as expected
+	cleanTests(basePath)
 
 	// Copy "1-basic" into a special "0-basic" which is removed
 	copy := exec.Command(
@@ -54,28 +63,23 @@ func TestMain(m *testing.M) {
 	err = copy.Run()
 	if err != nil {
 		fmt.Printf("cannot copy '0-basic' service: %v", err)
-		os.Exit(1)
+		return
 	}
 
 	path := filepath.Join(basePath, "0-basic")
 
 	err = createTrussService(path)
 	if err != nil {
 		fmt.Printf("cannot create truss service: %v", err)
-		os.Exit(1)
+		return
 	}
 
 	err = buildTestService(filepath.Join(path, "test-service"))
 	if err != nil {
 		fmt.Printf("cannot build truss service: %v", err)
-		os.Exit(1)
+		return
 	}
 
-	defer func() {
-		os.RemoveAll(filepath.Join(basePath, "0-basic"))
-		os.Exit(exitCode)
-	}()
-
 	exitCode = m.Run()
 }
 
@@ -210,13 +214,9 @@ func testEndToEnd(defDir string, subcmd string, t *testing.T, trussOptions ...st
 		t.FailNow()
 	}
 
-	// If nothing failed, delete the generated files
-	removeTestFiles(path)
 }
 
 func createTrussService(path string, trussFlags ...string) error {
-	// Remove tests if they exists
-	removeTestFiles(path)
 
 	trussOut, err := truss(path, trussFlags...)
 
@@ -275,14 +275,14 @@ func buildTestService(serviceDir string) (err error) {
 		return err
 	}
 
-	const serverPath = "/test-server"
-	const clientPath = "/test-cli-client"
+	const serverPath = "cmd/test-server"
+	const clientPath = "cmd/test"
 
 	// Build server and client
 	errChan := make(chan error)
 
 	go goBuild("test-server", binDir, filepath.Join(relDir, serverPath), errChan)
-	go goBuild("test-cli-client", binDir, filepath.Join(relDir, clientPath), errChan)
+	go goBuild("test", binDir, filepath.Join(relDir, clientPath), errChan)
 
 	err = <-errChan
 	if err != nil {
@@ -399,7 +399,7 @@ func reapServer(server *exec.Cmd, errc chan error) error {
 }
 
 func runClient(path string, flags ...string) ([]byte, error) {
-	const relativeClientPath = "/bin/test-cli-client"
+	const relativeClientPath = "/bin/test"
 
 	client := exec.Command(
 		path+relativeClientPath,
@@ -420,6 +420,8 @@ func fileExists(path string) bool {
 
 // cleanTests removes all test files from all directories in servicesDir
 func cleanTests(servicesDir string) {
+	// Remove the 0-basic used for non building tests
+	os.RemoveAll(filepath.Join(servicesDir, "0-basic"))
 	// Clean up the service directories in each test
 	dirs, _ := ioutil.ReadDir(servicesDir)
 	for _, d := range dirs {
@@ -434,9 +436,16 @@ func cleanTests(servicesDir string) {
 // removeTestFiles removes all files created by running truss and building the
 // service from a single definition directory
 func removeTestFiles(defDir string) {
+	// svcout dir
+	os.RemoveAll(filepath.Join(defDir, "tunelab"))
+	// service dir
 	os.RemoveAll(filepath.Join(defDir, "test-service"))
+	// where the binaries are compiled to
 	os.RemoveAll(filepath.Join(defDir, "bin"))
+	// test pbout
 	os.RemoveAll(filepath.Join(defDir, "pbout"))
+	// So that the directory exists for pbout
+	// TODO: Make pbout create the directory if it does not exist
 	os.MkdirAll(filepath.Join(defDir, "pbout"), 0777)
 }
 

cmd/_integration-tests/middlewares/middlewares/endpoints.go

@@ -4,7 +4,7 @@ import (
 	"github.com/go-kit/kit/endpoint"
 	"golang.org/x/net/context"
 
-	svc "github.com/TuneLab/go-truss/cmd/_integration-tests/middlewares/middlewarestest-service/generated"
+	svc "github.com/TuneLab/go-truss/cmd/_integration-tests/middlewares/middlewarestest-service/svc"
 )
 
 // WrapEndpoints accepts the service's entire collection of endpoints, so that a

cmd/_integration-tests/middlewares/setup_test.go

@@ -5,7 +5,7 @@ import (
 	"testing"
 
 	pb "github.com/TuneLab/go-truss/cmd/_integration-tests/middlewares/middlewarestest-service"
-	svc "github.com/TuneLab/go-truss/cmd/_integration-tests/middlewares/middlewarestest-service/generated"
+	svc "github.com/TuneLab/go-truss/cmd/_integration-tests/middlewares/middlewarestest-service/svc"
 	handler "github.com/TuneLab/go-truss/cmd/_integration-tests/middlewares/middlewarestest-service/handlers/server"
 	"github.com/TuneLab/go-truss/cmd/_integration-tests/middlewares/middlewarestest-service/middlewares"
 )

cmd/_integration-tests/transport/grpc_test.go

@@ -9,7 +9,7 @@ import (
 	"google.golang.org/grpc"
 
 	pb "github.com/TuneLab/go-truss/cmd/_integration-tests/transport/transportpermutations-service"
-	grpcclient "github.com/TuneLab/go-truss/cmd/_integration-tests/transport/transportpermutations-service/generated/client/grpc"
+	grpcclient "github.com/TuneLab/go-truss/cmd/_integration-tests/transport/transportpermutations-service/svc/client/grpc"
 )
 
 var grpcAddr string