Skip to content
Browse files

more changes in docs

  • Loading branch information...
1 parent 332e47a commit cca8c4599f584e430f329d8f31efde0f469df96a @bketelsen committed May 15, 2012
Showing with 9 additions and 121 deletions.
  1. +7 −68 README.md
  2. +0 −6 documentation/concept.txt
  3. +2 −1 documentation/contributors.txt
  4. +0 −13 documentation/initiators.txt
  5. +0 −21 documentation/routing.txt
  6. +0 −12 documentation/watchers.txt
View
75 README.md
@@ -3,94 +3,33 @@
###SKYNET is under construction right now - docs don't match reality - please be patient for a few days
##Introduction
-Skynet is a virtually–unkillable system for building massively distributed apps in Go.
+Skynet is a system for building massively distributed apps in Go.
##Tell me more:
-Servers die, stop communicating, catch on fire, get killed by robots from the future, and should not be trusted. If your site won’t work with a Chaos Monkey, it isn’t safe. Enter Skynet. Each Skynet module is self–contained, self–aware, and self–replicating – if you have one server with an authentication module on it, and that server melts, Skynet will notice, kill it, and automatically create a new one.
+Servers die, stop communicating, catch on fire, get killed by robots from the future, and should not be trusted. If your site won’t work with a Chaos Monkey, it isn’t safe. Enter Skynet. Each Skynet module is self–contained, self–aware, and self–replicating – if you have one server with an authentication module on it, and that server melts, Skynet will notice, kill it, and automatically create a new one. (if you let it)
Skynet probably won’t die unless your data center gets hit by a comet. We recommend at least 2 data centers in that scenario.
-SkyNet is built on the premise that there will be at least three distinct process types:
+Skynet Services are where the work gets done. These are the processes that service the requests, process the API calls, get the external data, log the requests, authenticate the users, etc.
-1. Initiators - Initiators are the source of inbound requests. On a web-centric system, they'd be running HTTP listeners and accept web based requests. That isn't required, however. We have initiators for flat files and TCP connections, too. If you can get the bytes in using Go, it can be an initiator.
-1. Routers - Routers are the "controller" of the system, they call services according to the stored route configuration that matches the request type.(Technically routers are optional, but if they're not used, Initiators will call Services directly. This is an advanced configuration.)
-1. Services -Services are where the work gets done. These are the processes that service the requests, process the API calls, get the external data, log the requests, authenticate the users, etc. You chain services together in a Route to build an application.
-1. (Optional) Watchers -Watchers are tasks that run and know about the system, but aren't responding to individual requests. An example of a watcher would be a process that watches the other processes in the system and reports on statistics or availability. The Reaper is a specialized watcher that checks each Skynet cluster member, culling dead processes from the configuration file.
-
-##Shut up and tell me what to do!
-Install [Go](http://golang.org) and [doozer](https://github.com/ha/doozerd)
-
- goinstall github.com/bketelsen/skynet/skygen
- goinstall github.com/bketelsen/skynet/skylib
- skygen -packageName=myCompany -serviceName=GetWidgets -targetFullPath="/Users/bketelsen/skynetTest/"
-
-The skygen command generates a source tree with a running sample application. After running skygen, cd into your target directory and build each service. We use the awesome [go-gb](https://github.com/skelterjohn/go-gb). Using gb, you simply issue the command "gb" from the root directory. Each service will be compiled, and the executable will be named the same as its containing folder. If you're following along, you'll have:
-
- skynetTest/
- skynetTest/bin/
- skynetTest/bin/router
- skynetTest/bin/service
- skynetTest/bin/watcher
- skynetTest/bin/reaper
- skynetTest/bin/initiator
Before you can run skynet you'll need to have at least one [doozer](https://github.com/ha/doozerd) process running.
-Now start each service, on a different port:
-
- bin/service -name=getwidgets -port=9200 &
- bin/initiator -name=webinitiator -port=9300 &
- bin/router -name=router -port=9100 &
- bin/reaper -name=reaper -port=9000 &
-
-If you don't specify a -logFileName parameter, they'll all default to using the same log file. Now open a web browser and aim it at http://127.0.0.1:9300
-Enter something in the form and hit enter. You should get a "Hello World" response.
-
-To really spice up your life, start up multiples of each process:
-
- bin/service -name=getwidgets2 -port=9201 &
- bin/initiator -name=webinitiator2 -port=9301 &
- bin/router -name=router2 -port=9101 &
- bin/reaper -name=reaper2 -port=9001 &
-
-Connect to http://127.0.0.1:9300 or :9301 and see the same thing. Kill the first router you started and submit a request... the second will handle the call. Kill all of the services, skynet will return a pretty error message letting you know that there weren't any services available to handle the request.
-
-Now, go to http://127.0.0.1:9100/debug/vars (if you haven't killed that router process... if you have find any other process and substitute the port). Skynet automatically exports statistical variables in JSON format:
-
- {
- ...
- "RouteService.RouteGetACHDataRequest-goroutines": 29,
- "cmdline": ["bin/routers","-name=router","-port=9100"],
- "RouteService.RouteGetACHDataRequest-processed": 3030,
- "RouteService.RouteGetACHDataRequest-errors": 2
- }
-
-##Do you have pretty diagrams?
-
-Yes.
-
-![logo](/bketelsen/skynet/raw/master/documentation/skynet.jpg)
-
-This rudimentary graph represents a service that is available externally via Web on two different ports, FTP, and TCP. Each Initiator sends requests to the Router. The Router looks up the route for this service and calls Logger asynchronously, then Authorization, then GetWidgets. Adding a new step to the route is as simple as creating the code for the new step (maybe you want to send a notification email, or put some data on a queue, or cache the response) then updating the route. The Initiators, Routers and other Services don't need to know or care about the new step added to your route. It all just Works(tm)
##How?
Each process in SkyNet receives its configuration from a centralized configuration repository (currently Doozer - possibly pluggable in the future). Configuration changes are pushed to each process when new skynet services are started. This means that starting a new service automatically
advertises that service's availability to the rest of the members of the skynet cluster.
-A typical transaction will come to an Initiator (via http for example) and be sent to a router that is providing the appropriate service to route that type of requests. The Router checks its routes and calls the services listed in its route configuration for that Route type. Routes also define whether a service can be called Asynchronously (fire and forget) or whether the router must wait for a response. For each service listed in the route the Router calls the service passing in the request and response objects. When all services are run, the router returns a response to the Initiator who is responsible for presenting the data to the remote client appropriately. In our HTTP example, this could mean translating to data using an HTML template, or an XML/JSON template.
-
-SkyNet uses Doozer to store configuration data about the available services and routes. Configuration changes are pushed to Doozer, causing connected clients to immediately become aware of changed configurations.
+SkyNet uses Doozer to store configuration data about the available services. Configuration changes are pushed to Doozer, causing connected clients to immediately become aware of changed configurations.
##Running Processes
* Sending SIGUSR1 to a running process raises the log level one notch.
* Sending SIGUSR2 to a running process lowers the log level one notch.
* Sending SIGINT to a running process gracefully exits.
-##Customizing
-In skynetTest/myCompany there's a file with the input and output structs for your API service. Add your input fields and output fields to these. Don't forget to change the initiator code to accept these fields, too. Now modify the skynetTest/service/service.go file to do something real - retrieve data from your systems - and you've built an API service in Go.
+##Management
+More management information to follow. SKY is here!
-##Documentation
-There are more (still kind of sparse) documents in the documentation folder.
##Communication
* Group name: Skynet-dev
@@ -101,7 +40,7 @@ There are more (still kind of sparse) documents in the documentation folder.
Github Issues now the canonical source of issues for Skynet.
##Open Source - MIT Software License
-Copyright (c) 2011 Brian Ketelsen
+Copyright (c) 2012 Brian Ketelsen
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
View
6 documentation/concept.txt
@@ -1,6 +0,0 @@
-Each process in SkyNet receives its configuration from a centralized configuration repository (currently Doozer - possibly pluggable in the future). Configuration changes are pushed to each process when new skynet services are started. This means that starting a new service automatically
-advertises that service's availability to the rest of the members of the skynet cluster.
-
-A typical transaction will come to an Initiator (via http for example) and be sent to a router that is providing the appropriate service to route that type of requests. The Router checks its routes and calls the services listed in its route configuration for that Route type. Routes also define whether a service can be called Asynchronously (fire and forget) or whether the router must wait for a response. For each service listed in the route the Router calls the service passing in the request and response objects. When all services are run, the router returns a response to the Initiator who is responsible for presenting the data to the remote client appropriately. In our HTTP example, this could mean translating to data using an HTML template, or an XML/JSON template.
-
-SkyNet uses Doozer to store configuration data about the available services and routes. Configuration changes are pushed to Doozer, causing connected clients to immediately become aware of changed configurations.
View
3 documentation/contributors.txt
@@ -1,3 +1,4 @@
Brian Ketelsen bketelsen@gmail.com
Christopher Dunn cdunn2001@gmail.com
-Paul Bellamy
+Paul Bellamy
+Erik St. Martin alakriti@gmail.com
View
13 documentation/initiators.txt
@@ -1,13 +0,0 @@
-Initiators are the source of inbound requests. On a web-centric system, they'd be running HTTP listeners and accept web based requests. That isn't required, however. We have initiators for flat files and TCP connections, too. If you can get the bytes in using Go, it can be an initiator.
-
-Think of the initiators as Skynet's presentation layer, feeding data into the system and presenting it to the consumer of the data.
-
-Initiator ideas:
-
-Web - simple form post marshals variables to Skynet, response sent as html
-File system watcher - waits for arrival of CSV file, strips records out and submits them to skynet one at a time. Results emailed somewhere, or logged to DB.
-Telnet - TCP listener takes commands from telnet session, returns textual responses
-REST - accept REST requests and translate them to skynet requests
-SOAP Proxy - create a soap proxy for skynet requests
-Comet
-
View
21 documentation/routing.txt
@@ -1,21 +0,0 @@
-Routing presents a named public method (not to be confused with a Skynet service, which is a unit of work) and lists the Skynet services that will be performed for that method.
-
-Example:
-
- For a public method GetUser, the Initiator will look for a route called RouteService.RouteGetUserRequest. That Route could contain a list of services like this:
-
- RpcCall{
- Service: "LogService.Log", // Name of the skynet service to call
- Async: true, // Can we call this and move on without waiting for the response
- OkToRetry: true, // If it fails, can we try again?
- ErrOnFail: false } // If it fails, do we return an error to the caller
- RpcCall{
- Service: "GetUserService.GetUser", // Name of the skynet service to call
- Async: false, // Can we call this and move on without waiting for the response
- OkToRetry: false, // If it fails, can we try again?
- ErrOnFail: true } // If it fails, do we return an error to the caller
-
-Services are executed in order. An error at any stage of the route sets adds an entry in the Error array of the response object.
-
-
-If you will NEVER need to call more than one service, you can wire the Initiator directly to the Service. This isn't recommended, simply because the hot-pluggable routes allow you the freedom to insert any other process as needed in the future. Today you may not need logging, but when your service becomes the next new internet hotness, you will want both logging and API Access throttling. Routes allow you the flexibility to do that without recompiling your service.
View
12 documentation/watchers.txt
@@ -1,12 +0,0 @@
-Watchers are tasks that run and know about the system, but aren't responding to individual requests. Watchers can't be in a route because they have no service method.
-
-
-Example Watchers:
-
-Collect usage data from Services and aggregate for presentation
-Spawner - create a spawner on each physical server that watches for processes to die. Use modifiable configuration to determine whether to spawn new process.
-Error notifier - when errors occur, notify someone
-Nagios collector - collect data from skynet and provide in nagios friendly format
-
-
-The Reaper is a specialized watcher that checks each Skynet cluster member, culling dead processes from the configuration on Doozer.

0 comments on commit cca8c45

Please sign in to comment.
Something went wrong with that request. Please try again.