-
Notifications
You must be signed in to change notification settings - Fork 1
/
protocol.txt
343 lines (323 loc) · 12.7 KB
/
protocol.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
# Periodic protocol
The Periodic protocol operates over TCP on port 5000 by default,
or unix socket on unix:///tmp/periodic.sock.
Communication happens between either a client and periodic server,
or between a worker and periodic server.
In either case, the protocol consists of packets
containing requests and responses. All packets sent to a periodic server
are considered requests, and all packets sent from a periodic server are
considered responses. A simple configuration may look like:
---------- ---------- ---------- ----------
| Client | | Client | | Client | | Client |
---------- ---------- ---------- ----------
\ / \ /
\ / \ /
------------------- -------------------
| Periodic Server | | Periodic Server |
------------------- -------------------
| |
----------------------------------------------
| | | |
---------- ---------- ---------- ----------
| Worker | | Worker | | Worker | | Worker |
---------- ---------- ---------- ----------
Initially, the workers register functions they can perform with each
job server. Clients will then connect to a job server and issue a
request to a job to be run. The job server then notifies each worker
that can perform that job (based on the function it registered) that
a new job is ready. The first worker to wake up and retrieve the job
will then execute it.
All communication between workers or clients and the periodic server
are binary.
## Client type
Once connected the client send it's type, and server will respond the connction id.
Request:
4 byte magic code - This is either "\0REQ" for requests or "\0RES"
for responses.
4 byte size - A big-endian (network-order) integer containing
the size of the data being sent.
4 byte crc32 - A big-endian (network-order) integer containing
the crc32 of the data being sent.
1 byte command - A big-endian (network-order) integer containing
an enumerated packet command. Possible values are:
# Name Type
1 TYPE_CLIENT Client
2 TYPE_WORKER Worker
Response:
4 byte magic code - This is either "\0REQ" for requests or "\0RES"
for responses.
4 byte size - A big-endian (network-order) integer containing
the size of the data being sent.
4 byte crc32 - A big-endian (network-order) integer containing
the crc32 of the data being sent.
? byte connection id - A binary connection id
## Binary Packet
Requests and responses are encapsulated by a binary packet. A binary
packet consists of a header which is optionally followed by data. The
header is:
4 byte magic code - This is either "\0REQ" for requests or "\0RES"
for responses.
4 byte size - A big-endian (network-order) integer containing
the size of the data being sent.
4 byte crc32 - A big-endian (network-order) integer containing
the crc32 of the data being sent.
4 byte message id - A client unique message id.
1 byte command - A big-endian (network-order) integer containing
an enumerated packet command. Possible values are:
# Name Type
0 NOOP Client/Worker
1 GRAB_JOB Worker
2 SCHED_LATER Worker
3 WORK_DONE Worker
4 WORK_FAIL Worker
5 JOB_ASSIGN Worker
6 NO_JOB Worker
7 CAN_DO Worker
8 CANT_DO Worker
9 PING Client/Worker
10 PONG Client/Worker
11 SLEEP Worker
12 UNKNOWN Client/Worker
13 SUBMIT_JOB Client
14 STATUS Client
15 DROP_FUNC Client
16 SUCCESS Client/Worker
17 REMOVE_JOB Client
18 DUMP Client
19 LOAD Client
20 SHUTDOWN Client
21 BROADCAST Worker
22 CONFIG_GET Client
23 CONFIG_SET Client
24 CONFIG Client
25 RUN_JOB Client
26 ACQUIRED Worker
27 ACQUIRE Worker
28 RELEASE Worker
29 NO_WORKER Client
30 DATA Client
## Client/Worker Requests
These request types may be sent by either a client or a worker:
PING
When a periodic server receives this request, it simply generates a
PONG packet. This is primarily used for testing
or debugging.
Arguments:
- None.
## Client/Worker Responses
These response types may be sent to either a client or a worker:
PONG
This is sent in response to a PING request.
Arguments:
- None.
## Client Requests
These request types may only be sent by a client:
SUBMIT_JOB
A client issues one of these when a job needs to be run. The
server will respond with a SUCCESS packet.
Arguments:
- Job binary packet
RUN_JOB
A client issues one of these when a job needs to be run. The
server will respond with a DATA packet or NO_WORKER packet.
Arguments:
- Job binary packet
STATUS
This sends back a list of all registered functions. Next to
each function is the number of jobs in the queue, the number of
running jobs, and the number of capable workers. The format is:
FUNCTIONS WORKERS JOBS PROCESSING LOCKED SCHEDAT
Arguments:
- None.
DROP_FUNC
Drop the function when there is not worker registered, and respond with
a SUCCESS packet.
Arguments:
- 1 byte func size
- ? byte func name
REMOVE_JOB
Remove a job, and respond with a SUCCESS packet.
Arguments:
- 1 byte func size
- ? byte func name
- 1 byte name size
- ? byte name
DUMP
Dump data from server. The server will respond with a binary packet.
Arguments:
- None.
LOAD
Load data to server. The server will respond with a SUCCESS packet.
Arguments:
- None.
CONFIG_GET
Get config from server. The server will respond with a CONFIG packet.
Arguments:
- 1 byte key size.
- ? byte key. the key is one of
- poll-interval
- revert-interval
- timeout
- keepalive
- max-batch-size
CONFIG_GET
Get config from server. The server will respond with a CONFIG packet.
Arguments:
- 1 byte key size.
- ? byte key. the key is one of
- poll-interval
- revert-interval
- timeout
- keepalive
- max-batch-size
- 4 byte value (int32).
SHUTDOWN
Shutdown server.
Arguments:
- None.
## Client Responses
These response types may only be sent to a client:
SUCCESS
This is sent in response to one of the SUBMIT_JOB/DROP_FUNC/REMOVE_JOB/CONFIG_SET
packets. It signifies to the client that a the server successfully
received the job and queued it to be run by a worker.
Arguments:
- None.
CONFIG
This is sent in response to one of the CONFIG_GET packets.
Arguments:
- 4 byte value (int32).
DATA
This is sent in response to one of the STATUS/RUN_JOB/DUMP packets.
Arguments:
- 4 byte value (int32).
- ? byte binary
NO_WORKER
This is sent in response to one of the RUN_JOB packets.
Arguments:
- None
## Worker Requests
These request types may only be sent by a worker:
BROADCAST
This is sent to notify the server that the worker is able to
perform the given function. The worker is then put on a list to be
woken up whenever the job server receives a job for that function.
The server will respond with a SUCCESS packet.
Arguments:
- 1 byte func size
- ? byte func name
CAN_DO
This is sent to notify the server that the worker is able to
perform the given function. The worker is then put on a list to be
woken up whenever the job server receives a job for that function.
The server will respond with a SUCCESS packet.
Arguments:
- 1 byte func size
- ? byte func name
CANT_DO
This is sent to notify the server that the worker is no longer
able to perform the given function.
The server will respond with a SUCCESS packet.
Arguments:
- 1 byte func size
- ? byte func name
SLEEP
This is sent to notify the server that the worker is about to
sleep, and that it should be woken up with a NOOP packet if a
job comes in for a function the worker is able to perform.
Arguments:
- None.
GRAB_JOB
This is sent to the server to request any available jobs on the
queue. The server will respond with either NO_JOB or JOB_ASSIGN,
depending on whether a job is available.
Arguments:
- None.
WORK_DONE
This is to notify the server that the job completed successfully.
The server will respond with a SUCCESS packet.
Arguments:
- ? byte handle
- ? byte data
WORK_FAIL
This is to notify the server that the job failed.
The server will respond with a SUCCESS packet.
Arguments:
- ? byte handle
SCHED_LATER
This is to notify the server to do the job on next time.
The server will respond with a SUCCESS packet.
Arguments:
- ? byte handle
- 8 byte time delay
- 2 byte step counter
ACQUIRE
This is to acquire a lock from the server.
The server will respond with a ACQUIRED packet.
Arguments:
- 1 byte lock name size
- ? byte lock name
- 2 byte lock count
- ? byte handle
RELEASE
This is to release a lock to the server.
The server will respond with a SUCCESS packet.
Arguments:
- 1 byte lock name size
- ? byte lock name
- ? byte handle
## Worker Responses
These response types may only be sent to a worker:
NOOP
This is used to wake up a sleeping worker so that it may grab a
pending job.
Arguments:
- None.
NO_JOB
This is given in response to a GRAB_JOB request to notify the
worker there are no pending jobs that need to run.
Arguments:
- None.
JOB_ASSIGN
This is given in response to a GRAB_JOB request to give the worker
information needed to run the job. All communication about the
job (such as status updates and completion response) should use
the handle, and the worker should run the given function with
the argument.
Arguments:
- Job binary packet
ACQUIRED
This is to response to a ACQUIRE request. if true run the job, else skip.
Arguments:
- 1 byte 1 or 0
## Job binary packets
Job is encode with a binary packet:
- 1 byte func size
- ? byte func name
- 1 byte name size
- ? byte name
- 4 byte workload size
- ? byte workload
- 8 byte sched time (unix time, int64)
- 1 byte job version
- # version
- 0 Ver0
- 1 Ver1
- 2 Ver2
- 3 Ver3
Version Spec:
Ver0:
- None
Ver1:
- 4 byte run count this assign by worker
Ver2:
- 4 byte timeout
Ver3:
- 4 byte run count this assign by worker
- 4 byte timeout
## Job handle packets
Job handle is encode with a binary packet:
- 1 byte func size
- ? byte func name
- 1 byte name size
- ? byte name