Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 401 lines (310 sloc) 14.672 kb
a8c0a0d Greg Taylor Beginnings of docs.
authored
1 .. _using:
2
25498e5 Greg Taylor A good deal more code examples.
authored
3 .. include:: global.txt
4
6d644dc Greg Taylor Improving the using docs.
authored
5 Connecting to the EMDR network
6 ==============================
7
8 In order to connect to the EVE Market Data Relay feed, you'll need to use
9 the ZeroMQ_ bindings for the language of your choice. This involves installing
10 ZeroMQ_, then installing the bindings. See your language's section below for
11 a link to its bindings.
12
13 Once you have ZeroMQ_ plus bindings installed, you'll need to choose a
14 Relay to connect to. See :doc:`access` for a list, and any potential special
15 case instructions for each relay. After that, you'll be set to connect and
16 begin receiving data.
17
18 Data Format
19 -----------
a8c0a0d Greg Taylor Beginnings of docs.
authored
20
cd9a07f Greg Taylor Adding a note about the format that the data comes out in.
authored
21 All data coming out of EMDR is in `Unified Uploader Interchange Format`_,
22 which is a JSON-based standard for market orders and history. See the
23 `spec <Unified Uploader Interchange Format>`_ for more details.
24
6d644dc Greg Taylor Improving the using docs.
authored
25 Examples for various languages
26 ------------------------------
27
25498e5 Greg Taylor A good deal more code examples.
authored
28 Below are a few examples of how to connect to the data feed. If you see
29 anything wrong with the examples below, please let us know on the
30 `issue tracker`_. The original author of this documentation is only familiar
31 with Python.
32
33 Python
844b9e1 Greg Taylor Re-working the documentation index, adding consumer considerations doc, ...
authored
34 ^^^^^^
25498e5 Greg Taylor A good deal more code examples.
authored
35
36 The following example uses the pyzmq_ module (available off of PyPi)
d34042a Greg Taylor Adding README.rst files to the example dirs so it looks pretty in github...
authored
37 and simplejson_. For a more complete list of examples, see the
38 `Python examples`_ dir on github.::
25498e5 Greg Taylor A good deal more code examples.
authored
39
d34042a Greg Taylor Adding README.rst files to the example dirs so it looks pretty in github...
authored
40 """
41 Example Python EMDR client.
42 """
3b60488 Greg Taylor Update using instructions to let people know about zlib de-compressing t...
authored
43 import zlib
25498e5 Greg Taylor A good deal more code examples.
authored
44 import zmq
d34042a Greg Taylor Adding README.rst files to the example dirs so it looks pretty in github...
authored
45 # You can substitute the stdlib's json module, if that suits your fancy
25498e5 Greg Taylor A good deal more code examples.
authored
46 import simplejson
47
48 def main():
49 context = zmq.Context()
50 subscriber = context.socket(zmq.SUB)
51
52 # Connect to the first publicly available relay.
c940375 Greg Taylor Updating examples to be valid for the test deploy.
authored
53 subscriber.connect('tcp://relay-linode-atl-1.eve-emdr.com:8050')
25498e5 Greg Taylor A good deal more code examples.
authored
54 # Disable filtering.
55 subscriber.setsockopt(zmq.SUBSCRIBE, "")
56
57 while True:
58 # Receive raw market JSON strings.
3b60488 Greg Taylor Update using instructions to let people know about zlib de-compressing t...
authored
59 market_json = zlib.decompress(subscriber.recv())
25498e5 Greg Taylor A good deal more code examples.
authored
60 # Un-serialize the JSON data to a Python dict.
61 market_data = simplejson.loads(market_json)
62 # Dump the market data to stdout. Or, you know, do more fun
63 # things here.
64 print market_data
65
66 if __name__ == '__main__':
67 main()
68
69 .. _pyzmq: http://pypi.python.org/pypi/pyzmq/
70 .. _simplejson: http://pypi.python.org/pypi/simplejson/
71 .. _json: http://docs.python.org/library/json.html
d34042a Greg Taylor Adding README.rst files to the example dirs so it looks pretty in github...
authored
72 .. _Python examples: https://github.com/gtaylor/EVE-Market-Data-Relay/tree/master/examples/python
25498e5 Greg Taylor A good deal more code examples.
authored
73
74 PHP
844b9e1 Greg Taylor Re-working the documentation index, adding consumer considerations doc, ...
authored
75 ^^^
25498e5 Greg Taylor A good deal more code examples.
authored
76
8dc04c0 Greg Taylor Correctly syntax the PHP and Ruby examples.
authored
77 PHP accesses EMDR via ZeroMQ's `php-zmq`_ PHP bindings:
78
79 .. code-block:: php
25498e5 Greg Taylor A good deal more code examples.
authored
80
81 <?php
82 /*
83 * Example PHP EMDR client.
84 */
85
86 $context = new ZMQContext();
87 $subscriber = $context->getSocket(ZMQ::SOCKET_SUB);
88
89 // Connect to the first publicly available relay.
c940375 Greg Taylor Updating examples to be valid for the test deploy.
authored
90 $subscriber->connect("tcp://relay-linode-atl-1.eve-emdr.com:8050");
25498e5 Greg Taylor A good deal more code examples.
authored
91 // Disable filtering.
92 $subscriber->setSockOpt(ZMQ::SOCKOPT_SUBSCRIBE, "");
93
94 while (true) {
3b60488 Greg Taylor Update using instructions to let people know about zlib de-compressing t...
authored
95 // Receive raw market JSON strings.
96 $market_json = gzuncompress($subscriber->recv());
97 // Un-serialize the JSON data to a named array.
98 $market_data = json_decode($market_json);
99 // Dump the market data to stdout. Or, you know, do more fun things here.
100 printf($market_data);
25498e5 Greg Taylor A good deal more code examples.
authored
101 }
102
103 .. _php-zmq: http://www.zeromq.org/bindings:php
104
105 Ruby
844b9e1 Greg Taylor Re-working the documentation index, adding consumer considerations doc, ...
authored
106 ^^^^
25498e5 Greg Taylor A good deal more code examples.
authored
107
8dc04c0 Greg Taylor Correctly syntax the PHP and Ruby examples.
authored
108 Ruby accesses EMDR via ZeroMQ's zmq_ Ruby bindings:
109
110 .. code-block:: ruby
25498e5 Greg Taylor A good deal more code examples.
authored
111
112 #
113 # Synchronized subscriber
114 #
115
116 require 'rubygems'
117 require 'ffi-rzmq'
541931d Greg Taylor Getting the Ruby example corrected, thanks @Kazu.
authored
118 require 'json'
119 require 'zlib'
25498e5 Greg Taylor A good deal more code examples.
authored
120
121 context = ZMQ::Context.new
122 subscriber = context.socket(ZMQ::SUB)
123
124 // Connect to the first publicly available relay.
c940375 Greg Taylor Updating examples to be valid for the test deploy.
authored
125 subscriber.connect("tcp://relay-linode-atl-1.eve-emdr.com:8050")
25498e5 Greg Taylor A good deal more code examples.
authored
126 subscriber.setsockopt(ZMQ::SUBSCRIBE,"")
127
128 loop do
541931d Greg Taylor Getting the Ruby example corrected, thanks @Kazu.
authored
129 // Receive raw market JSON strings.
25498e5 Greg Taylor A good deal more code examples.
authored
130 subscriber.recv_string(string = '')
541931d Greg Taylor Getting the Ruby example corrected, thanks @Kazu.
authored
131 // Un-compress the stream.
132 market_json = Zlib::Inflate.new(Zlib::MAX_WBITS).inflate(string)
133 // Un-serialize the JSON data.
134 market_data = JSON.parse(market_json)
135 // Dump the market data to stdout. Or, you know, do more fun things here.
136 puts market_data
25498e5 Greg Taylor A good deal more code examples.
authored
137 end
138
541931d Greg Taylor Getting the Ruby example corrected, thanks @Kazu.
authored
139 .. _zmq: http://www.zeromq.org/bindings:ruby
aad461f Greg Taylor Adding C# binding examples. Contributed by Desmont McCallock.
authored
140
141 C#
844b9e1 Greg Taylor Re-working the documentation index, adding consumer considerations doc, ...
authored
142 ^^
aad461f Greg Taylor Adding C# binding examples. Contributed by Desmont McCallock.
authored
143
144 C# accesses EMDR via ZeroMQ's clrzmq_ binding:
145
146 .. code-block:: c#
147
148 using System;
149 using System.Collections.Generic;
150 using System.IO;
151 using System.IO.Compression;
152 using System.Linq;
153 using System.Text;
154 using System.Web.Script.Serialization; // Needs reference to 'System.Web.Extensions.dll'
155 using ZMQ; // Needs reference to 'clrzmq.dll' and adding 'libzmq.dll' to project
156 // 'clrzmq' can be found at: https://github.com/zeromq/clrzmq/downloads
157
158 namespace EMDR_Client
159 {
160 public class Program
161 {
162 private static void Main()
163 {
164 using (var context = new Context())
165 {
166 using (var subscriber = context.Socket(SocketType.SUB))
167 {
168 //Connect to the first publicly available relay.
169 subscriber.Connect("tcp://relay-linode-atl-1.eve-emdr.com:8050");
170
171 // Disable filtering.
172 subscriber.SetSockOpt(SocketOpt.SUBSCRIBE, Encoding.UTF8.GetBytes(""));
173
174 // Alternatively 'Subscribe' can be used
175 //subscriber.Subscribe("", Encoding.UTF8);
176
177 while (true)
178 {
179 try
180 {
181 // Receive compressed raw market data.
182 var receivedData = subscriber.Recv();
183
184 // The following code lines remove the need of 'zlib' usage;
185 // 'zlib' actually uses the same algorith as 'DeflateStream'.
186 // To make the data compatible for 'DeflateStream', we only have to remove
187 // the four last bytes which are the adler32 checksum and
188 // the two first bytes which are the 'zlib' header.
189 byte[] decompressed;
190 byte[] choppedRawData = new byte[(receivedData.Length - 4)];
191 Array.Copy(receivedData, choppedRawData, choppedRawData.Length);
192 choppedRawData = choppedRawData.Skip(2).ToArray();
193
194 // Decompress the raw market data.
195 using (MemoryStream inStream = new MemoryStream(choppedRawData))
196 using (MemoryStream outStream = new MemoryStream())
197 {
198 DeflateStream outZStream = new DeflateStream(inStream, CompressionMode.Decompress);
199 outZStream.CopyTo(outStream);
200 decompressed = outStream.ToArray();
201 }
202
203 // Transform data into JSON strings.
204 string marketJson = Encoding.UTF8.GetString(decompressed);
205
206 // Un-serialize the JSON data to a dictionary.
207 var serializer = new JavaScriptSerializer();
208 var dictionary = serializer.Deserialize<Dictionary<string, object>>(marketJson);
209
210 // Dump the market data to console or, you know, do more fun things here.
211 foreach (KeyValuePair<string, object> pair in dictionary)
212 {
213 Console.WriteLine("{0}: {1}", pair.Key, pair.Value);
214 }
215 Console.WriteLine();
216 }
217 catch (ZMQ.Exception ex)
218 {
219 Console.WriteLine("ZMQ Exception occurred : {0}", ex.Message);
220 }
221 }
222 }
223 }
224 }
225 }
226 }
227
7c1baea Greg Taylor Adding Visual Basic EMDR connection example. Thanks to Desmond McCallock...
authored
228 .. _clrzmq: https://github.com/zeromq/clrzmq/downloads
229
230 Visual Basic
844b9e1 Greg Taylor Re-working the documentation index, adding consumer considerations doc, ...
authored
231 ^^^^^^^^^^^^
7c1baea Greg Taylor Adding Visual Basic EMDR connection example. Thanks to Desmond McCallock...
authored
232
844b9e1 Greg Taylor Re-working the documentation index, adding consumer considerations doc, ...
authored
233 Visual Basic, like C#, accesses EMDR via ZeroMQ's clrzmq_ binding:
7c1baea Greg Taylor Adding Visual Basic EMDR connection example. Thanks to Desmond McCallock...
authored
234
235 .. code-block:: vb.net
236
237 Imports System.Text
238 Imports System.IO
239 Imports System.IO.Compression
240 Imports System.Web.Script.Serialization ' Needs reference to 'System.Web.Extensions.dll'
241 Imports ZMQ ' Needs reference to 'clrzmq.dll' and adding 'libzmq.dll' to project
242 ' 'clrzmq' can be found at: https://github.com/zeromq/clrzmq/downloads
243
244 Module MainModule
245
246 Sub Main()
247 Using context = New Context()
248 Using subscriber = context.Socket(SocketType.SUB)
249
250 'Connect to the first publicly available relay.
251 subscriber.Connect("tcp://relay-linode-atl-1.eve-emdr.com:8050")
252
253 ' Disable filtering.
254 subscriber.SetSockOpt(SocketOpt.SUBSCRIBE, Encoding.UTF8.GetBytes(""))
255
256 ' Alternatively 'Subscribe' can be used.
257 'subscriber.Subscribe("", Encoding.UTF8)
258
259 While True
260 Try
261 ' Receive compressed raw market data.
262 Dim receivedData() = subscriber.Recv()
263
264 ' The following code lines remove the need of 'zlib' usage;
265 ' 'zlib' actually uses the same algorith as 'DeflateStream'.
266 ' To make the data compatible for 'DeflateStream', we only have to remove
267 ' the four last bytes which are the adler32 checksum and
268 ' the two first bytes which are the 'zlib' header.
269 Dim decompressed() As Byte
270 Dim choppedRawData(receivedData.Length - 4) As Byte
271 Array.Copy(receivedData, choppedRawData, choppedRawData.Length)
272 choppedRawData = choppedRawData.Skip(2).ToArray()
273
274 ' Decompress the raw market data.
275 Using inStream = New MemoryStream(choppedRawData)
276 Using outStream = New MemoryStream()
277 Dim outZStream = New DeflateStream(inStream, CompressionMode.Decompress)
278 outZStream.CopyTo(outStream)
279 decompressed = outStream.ToArray
280 End Using
281 End Using
282
283 ' Transform data into JSON strings.
284 Dim marketJson = Encoding.UTF8.GetString(decompressed)
285
286 ' Un-serialize the JSON data to a dictionary.
287 Dim serializer = New JavaScriptSerializer()
288 Dim dictionary = serializer.Deserialize(Of Dictionary(Of String, Object))(marketJson)
289
290 ' Dump the market data to console or, you know, do more fun things here.
291 For Each pair In dictionary
292 Console.WriteLine("{0}: {1}", pair.Key, pair.Value)
293 Next
294 Console.WriteLine()
295 Catch ex As Exception
296 Console.WriteLine("ZMQ Exception occurred : {0}", ex.Message)
297 End Try
298 End While
299 End Using
300 End Using
301 End Sub
a26d841 Greg Taylor Adding Java and Perl connection code samples.
authored
302 End Module
303
304 Perl
305 ^^^^
306
307 Perl uses the `ZeroMQ-Perl`_ binding for Perl:
308
309 .. code-block:: perl
310
311 #!/usr/bin/perl
312 use warnings;
313 use strict;
314 $|=1;
315
316 use ZeroMQ qw/:all/;
317
318 my $cxt = ZeroMQ::Context->new;
319 my $sock = $cxt->socket(ZMQ_SUB);
320 $sock->connect('tcp://relay-linode-atl-1.eve-emdr.com:8050');
321 $sock->setsockopt(ZMQ_SUBSCRIBE, "");
322
323 while (1) {
324 my $msg = $sock->recv();
325 last unless defined $msg;
326
327 use Compress::Zlib;
328 my $json = uncompress($msg->data);
329
330 use JSON;
331 my $data = decode_json($json);
332
333 use Data::Dumper;
334 print Dumper($data),"\n\n";
335 }
336
337 .. _ZeroMQ-Perl: http://www.zeromq.org/bindings:perl
338
339 Java
340 ^^^^
341
342 Java uses jzmq_ binding:
343
344 .. code-block:: java
345
346 /*
347 * Example Java EMDR client.
348 */
349
350 import org.zeromq.*; // https://github.com/zeromq/jzmq
351 import org.json.simple.*; // http://code.google.com/p/json-simple/downloads/list
352 import org.json.simple.parser.*;
353 import java.util.zip.*;
354
355 public class EMDR_Client {
356
357 public static void main(String[] args) throws Exception {
358
359 ZMQ.Context context = ZMQ.context(1);
360 ZMQ.Socket subscriber = context.socket(ZMQ.SUB);
361
362 // Connect to the first publicly available relay.
363 subscriber.connect("tcp://relay-linode-atl-1.eve-emdr.com:8050");
364
365 // Disable filtering.
366 subscriber.subscribe(new byte[0]);
367
368 while (true) {
369 try {
370 // Receive compressed raw market data.
371 byte[] receivedData = subscriber.recv(0);
372
373 // We build a large enough buffer to contain the decompressed data.
374 byte[] decompressed = new byte[receivedData.length * 16];
375
376 // Decompress the raw market data.
377 Inflater inflater = new Inflater();
378 inflater.setInput(receivedData);
379 int decompressedLength = inflater.inflate(decompressed);
380 inflater.end();
381
382 byte[] output = new byte[decompressedLength];
383 System.arraycopy(decompressed, 0, output, 0, decompressedLength);
384
385 // Transform data into JSON strings.
386 String market_json = new String(output, "UTF-8");
387
388 // Un-serialize the JSON data.
389 JSONParser parser = new JSONParser();
390 JSONObject market_data = (JSONObject)parser.parse(market_json);
391
392 // Dump the market data to console or, you know, do more fun things here.
393 System.out.println(market_data);
394 } catch (ZMQException ex) {
395 System.out.println("ZMQ Exception occurred : " + ex.getMessage());
396 }
397 }
398 }
399 }
400
401 .. _jzmq: http://www.zeromq.org/bindings:java
Something went wrong with that request. Please try again.