Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 237 lines (167 sloc) 8.963 kb
3cc1e1e @gdb Initial import of Einhorn
gdb authored
1 # Einhorn: the language-independent shared socket manager
2
3 ![Einhorn](https://stripe.com/img/blog/posts/meet-einhorn/einhorn.png)
4
5 Let's say you have a server process which processes one request at a
6 time. Your site is becoming increasingly popular, and this one process
7 is no longer able to handle all of your inbound connections. However,
8 you notice that your box's load number is low.
9
10 So you start thinking about how to handle more requests. You could
11 rewrite your server to use threads, but threads are a pain to program
12 against (and maybe you're writing in Python or Ruby where you don't
13 have true threads anyway). You could rewrite your server to be
14 event-driven, but that'd require a ton of effort, and it wouldn't help
15 you go beyond one core. So instead, you decide to just run multiple
16 copies of your server process.
17
18 Enter Einhorn. Einhorn makes it easy to run (and keep alive) multiple
19 copies of a single long-lived process. If that process is a server
20 listening on some socket, Einhorn will open the socket in the master
21 process so that it's shared among the workers.
22
23 Einhorn is designed to be compatible with arbitrary languages and
24 frameworks, requiring minimal modification of your
25 application. Einhorn is simple to configure and run.
26
27 ## Installation
28
29 Install from Rubygems as:
30
31 $ gem install einhorn
32
33 Or build from source by:
34
35 $ gem build einhorn.gemspec
36
37 And then install the built gem.
38
39 ## Usage
40
41 Einhorn is the language-independent shared socket manager. Run
42 `einhorn -h` to see detailed usage. At a high level, usage looks like
43 the following:
44
45 einhorn [options] program
46
47 Einhorn will open one or more shared sockets and run multiple copies
48 of your process. You can seamlessly reload your code, dynamically
49 reconfigure Einhorn, and more.
50
51 ## Overview
52
53 To set Einhorn up as a master process running 3 copies of `sleep 5`:
54
55 $ einhorn -n 3 sleep 5
56
57 You can communicate your running Einhorn process via `einhornsh`:
58
59 $ einhornsh
60 Welcome gdb! You are speaking to Einhorn Master Process 11902
61 Enter 'help' if you're not sure what to do.
62
63 Type "quit" or "exit" to quit at any time
64 > help
65 You are speaking to the Einhorn command socket. You can run the following commands:
66 ...
67
68 ### Server sockets
69
70 If your process is a server and listens on one or more sockets,
71 Einhorn can open these sockets and pass them to the workers. Program
72 arguments of the form
73
74 srv:(IP:PORT)[<,OPT>...]
75 --MY-OPT=srv:(IP:PORT)[<,OPT>...]
76
77 Will be interpreted as a request to open a server socket bound to
78 IP:PORT. The argument will be replaced with `FD` and `---MY-OPT=FD`,
79 respectively, where `FD` is the file descriptor number of the socket.
80 Valid opts are:
81
82 r, so_reuseaddr: set SO_REUSEADDR on the server socket
83 n, o_nonblock: set O_NONBLOCK on the server socket
84
85 You can for example run:
86
87 $ einhorn -m manual -n 4 example/time_server srv:127.0.0.1:2345,r
88
89 Which will run 4 copies of
90
91 example/time_server 6
92
93 Where file descriptor 6 is a server socket bound to `127.0.0.1:2345`
94 and with `SO_REUSEADDR` set. It is then your application's job to
95 figure out how to `accept()` on this file descriptor.
96
97 ### Command socket
98
99 Einhorn opens a UNIX socket to which you can send commands (run
100 `help` in `einhornsh` to see what admin commands you can
101 run). Einhorn relies on file permissions to ensure that no malicious
102 users can gain access. Run with a `-d DIRECTORY` to change the
103 directory where the socket will live.
104
105 ### Seamless upgrades
106
107 You can cause your code to be seamlessly reloaded by upgrading the
108 worker code on disk and running
109
110 $ einhornsh
111 ...
112 > upgrade
113
114 Once the new workers have been spawned, Einhorn will send each old
115 worker a SIGUSR2. SIGUSR2 should be interpreted as a request for a
116 graceful shutdown.
117
118 ### ACKs
119
120 After Einhorn spawns a worker, it will only consider the worker up
121 once it has received an ACK. Currently two ACK mechanisms are
122 supported: manual and timer.
123
124 #### Manual ACK
125
126 A manual ACK (configured by providing a `-m manual`) requires your
127 application to send a command to the command socket once it's
128 ready. This is the safest ACK mechanism. If you're writing in Ruby,
129 just do
130
131 require 'einhorn/worker'
132 Einhorn::Worker.ack!
133
134 in your worker code. If you're writing in a different language, or
135 don't want to include Einhorn in your namespace, you can send the
136 string
137
138 {"command":"worker:ack", "pid":PID}
139
140 to the UNIX socket pointed to by the environment variable
141 `EINHORN_SOCK_PATH`. (Be sure to include a trailing newline.)
142
143 To make things even easier, you can pass a `-b` to Einhorn, in which
144 case you just need to `write()` the above message to the open file
145 descriptor pointed to by `EINHORN_FD`.
146
147 (See `lib/einhorn/worker.rb` for details of these and other socket
148 discovery mechanisms.)
149
150 #### Timer ACK [default]
151
152 By default, Einhorn will use a timer ACK of 1 second. That means that
153 if your process hasn't exited after 1 second, it is considered ACK'd
154 and healthy. You can modify this timeout to be more appropriate for
155 your application (and even set to 0 if desired). Just pass a `-m
156 FLOAT`.
157
158 ### Preloading
159
160 If you're running a Ruby process, Einhorn can optionally preload its
161 code, so it only has to load the code once per upgrade rather than
162 once per worker process. This also saves on memory overhead, since all
163 of the code in these processes will be stored only once using your
164 operating system's copy-on-write features.
165
166 To use preloading, just give Einhorn a `-p PATH_TO_CODE`, and make
167 sure you've defined an `einhorn_main` method.
168
169 In order to maximize compatibility, we've worked to minimize Einhorn's
170 dependencies. At the moment Einhorn imports 'rubygems' and 'json'. (If
171 these turn out to be issues, we could probably find a workaround.)
172
173 ### Command name
174
175 You can set the name that Einhorn and your workers show in PS. Just
176 pass `-c <name>`.
177
178 ### Options
179
180 -b, --command-socket-as-fd Leave the command socket open as a file descriptor, passed in the EINHORN_FD environment variable. This allows your worker processes to ACK without needing to know where on the filesystem the command socket lives.
181 -c, --command-name CMD_NAME Set the command name in ps to this value
182 -d, --socket-path PATH Where to open the Einhorn command socket
183 -e, --pidfile PIDFILE Where to write out the Einhorn pidfile
184 -f, --lockfile LOCKFILE Where to store the Einhorn lockfile
185 -h, --help Display this message
186 -k, --kill-children-on-exit If Einhorn exits unexpectedly, gracefully kill all its children
187 -l, --backlog N Connection backlog (assuming this is a server)
188 -m, --ack-mode MODE What kinds of ACK to expect from workers. Choices: FLOAT (number of seconds until assumed alive), manual (process will speak to command socket when ready). Default is MODE=1.
189 -n, --number N Number of copies to spin up
190 -p, --preload PATH Load this code into memory, and fork but do not exec upon spawn. Must define an "einhorn_main" method
191 -q, --quiet Make output quiet (can be reconfigured on the fly)
192 -s, --seconds N Number of seconds to wait until respawning
193 -v, --verbose Make output verbose (can be reconfigured on the fly)
194 --with-state-fd STATE [Internal option] With file descriptor containing state
195 --version Show version
196
197
198 ## Contributing
199
200 Contributions are definitely welcome. To contribute, just follow the
201 usual workflow:
202
203 1. Fork Einhorn
204 2. Create your feature branch (`git checkout -b my-new-feature`)
205 3. Commit your changes (`git commit -am 'Added some feature'`)
206 4. Push to the branch (`git push origin my-new-feature`)
207 5. Create new Github pull request
208
209 ## History
210
211 Einhorn came about when Stripe was investigating seamless code
212 upgrading solutions for our API worker processes. We really liked the
213 process model of [Unicorn](http://unicorn.bogomips.org/), but didn't
214 want to use its HTTP functionality. So Einhorn was born, providing the
215 master process functionality of Unicorn (and similar preforking
216 servers) to a wider array of applications.
217
218 See https://stripe.com/blog/meet-einhorn for more background.
219
220 Stripe currently uses Einhorn in production for a number of
221 services. Our Thin + EventMachine servers currently require patches to
222 both Thin and EventMachine (to support file-descriptor passing). You
223 can obtain these patches from our public forks of the
224 [respective](https://github.com/stripe/thin)
225 [projects](https://github.com/stripe/eventmachine). Check out
226 `example/thin_example` for an example of running Thin under Einhorn.
227
228 ## Compatibility
229
230 Einhorn was developed and tested under Ruby 1.8.7.
231
232 ## About
233
234 Einhorn is a project of [Stripe](https://stripe.com), led by [Greg
235 Brockman](https://twitter.com/thegdb). Feel free to get in touch at
236 info@stripe.com.
Something went wrong with that request. Please try again.