Find file
Fetching contributors…
Cannot retrieve contributors at this time
88 lines (65 sloc) 2.82 KB
Notes for The Guide
Last Value Cache device
- XPUB to detect subscriber changes
- cache of last messages... per subscription
- detect dead publishers and turn these into EOS
==1742== Syscall param write(buf) points to uninitialised byte(s)
==1742== at 0x5372100: __write_nocancel (syscall-template.S:82)
==1742== by 0x5306312: _IO_file_write@@GLIBC_2.2.5 (fileops.c:1289)
==1742== by 0x53061D9: new_do_write (fileops.c:543)
==1742== by 0x5307944: _IO_do_write@@GLIBC_2.2.5 (fileops.c:516)
==1742== by 0x5306E6F: _IO_file_close_it@@GLIBC_2.2.5 (fileops.c:170)
==1742== by 0x52FB2CF: fclose@@GLIBC_2.2.5 (iofclose.c:62)
==1742== by 0x4060FF: fmq_file_test (fmq_file.c:342)
==1742== by 0x4021E6: main (fmq_selftest.c:38)
==1742== Address 0x4027000 is not stack'd, malloc'd or (recently) free'd
FILE *handle = fopen ("testdata", "w");
zmq_msg_t msg;
zmq_msg_init_size (&msg, 100);
size_t items = fwrite (zmq_msg_data (&msg), 1, 100, handle);
fclose (handle);
zmq_msg_close (&msg);
Chapter 6 topics:
- heartbeating, credit-based flow control, file transfer, serialization
- version negotiation
Chapter 7 topics:
- how to build a community
- how to contribute to 0MQ, C4 process in some detail
Chapter 8 topics:
- UDP discovery
- bridging 0MQ to other protocols
- web interfaces (including NullMQ)
- in/out process logging
- timestamp
- filter I/W/E
- capture in log file
- cycle log files
- timer service (subscribe to timer events)
- disconnected security (for pub/sub)
Chapter 9 topics
- the wire level protocol
- a minimal TCP stack
- internals of 0MQ
- generate bindings in
- using GSL :-)
Chapter 5 additional topics
++++ Reliable Pipeline (Harmony Pattern)
0MQ's pipeline pattern (using PUSH and PULL sockets) is reliable to the extent that:
* Workers and collectors don't crash;
* Workers and collectors read their data fast enough to avoid queue overflows.
As with all our reliability patterns, we'll ignore what happens if an upstream node (the ventilator for a pipeline pattern) dies. In practice a ventilator will be the client of another reliability pattern, e.g. Clone.
The Harmony pattern takes pipeline and makes it robust against the only failure we can reasonably handle, namely workers and (less commonly) collectors that crash and lose messages or work.
- assume workers are idempotent
- assume batch size is known in advance (because...)
- assume memory enough to hold full batch
- batch: start (address of collector), tasks, end
- messages numbered 0 upwards inside batch
- assume multiple ventilators for same cluster
- assume collector talks to ventilator, (not same to allow walk-up-and use by ventilators)
- call ventilator the 'client'
- if task missing, resend
- if end of batch missing, resend from last response