-
Notifications
You must be signed in to change notification settings - Fork 16
/
Copy pathexample-http.html
168 lines (153 loc) · 8.94 KB
/
example-http.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>10.3. Example HTTP Connector — Presto 0.174 Documentation</title>
<link rel="stylesheet" href="../_static/presto.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
VERSION: '0.174',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="top" title="Presto 0.174 Documentation" href="../index.html" />
<link rel="up" title="10. Developer Guide" href="../develop.html" />
<link rel="next" title="10.4. Types" href="types.html" />
<link rel="prev" title="10.2. Connectors" href="connectors.html" />
</head>
<body role="document">
<div class="header">
<h1 class="heading"><a href="../index.html">
<span>Presto 0.174 Documentation</span></a></h1>
<h2 class="heading"><span>10.3. Example HTTP Connector</span></h2>
</div>
<div class="topnav">
<p class="nav">
<span class="left">
« <a href="connectors.html">10.2. Connectors</a>
</span>
<span class="right">
<a href="types.html">10.4. Types</a> »
</span>
</p>
</div>
<div class="content">
<div class="section" id="example-http-connector">
<h1>10.3. Example HTTP Connector</h1>
<p>The Example HTTP connector has a simple goal: it reads comma-separated
data over HTTP. For example, if you have a large amount of data in a
CSV format, you can point the example HTTP connector at this data and
write a SQL query to process it.</p>
<div class="section" id="code">
<h2>Code</h2>
<p>The Example HTTP connector can be found in the <code class="docutils literal"><span class="pre">presto-example-http</span></code>
directory in the root of the Presto source tree.</p>
</div>
<div class="section" id="plugin-implementation">
<h2>Plugin Implementation</h2>
<p>The plugin implementation in the Example HTTP connector looks very
similar to other plugin implementations. Most of the implementation is
devoted to handling optional configuration and the only function of
interest is the following:</p>
<div class="highlight-java"><div class="highlight"><pre><span></span><span class="nd">@Override</span>
<span class="kd">public</span> <span class="n">Iterable</span><span class="o"><</span><span class="n">ConnectorFactory</span><span class="o">></span> <span class="nf">getConnectorFactories</span><span class="o">()</span>
<span class="o">{</span>
<span class="k">return</span> <span class="n">ImmutableList</span><span class="o">.</span><span class="na">of</span><span class="o">(</span><span class="k">new</span> <span class="n">ExampleConnectorFactory</span><span class="o">());</span>
<span class="o">}</span>
</pre></div>
</div>
<p>Note that the <code class="docutils literal"><span class="pre">ImmutableList</span></code> class is a utility class from Guava.</p>
<p>As with all connectors, this plugin overrides the <code class="docutils literal"><span class="pre">getConnectorFactories()</span></code> method
and returns an <code class="docutils literal"><span class="pre">ExampleConnectorFactory</span></code>.</p>
</div>
<div class="section" id="connectorfactory-implementation">
<h2>ConnectorFactory Implementation</h2>
<p>In Presto, the primary object that handles the connection between
Presto and a particular type of data source is the <code class="docutils literal"><span class="pre">Connector</span></code> object,
which are created using <code class="docutils literal"><span class="pre">ConnectorFactory</span></code>.</p>
<p>This implementation is available in the class <code class="docutils literal"><span class="pre">ExampleConnectorFactory</span></code>.
The first thing the connector factory implementation does is specify the
name of this connector. This is the same string used to reference this
connector in Presto configuration.</p>
<div class="highlight-java"><div class="highlight"><pre><span></span><span class="nd">@Override</span>
<span class="kd">public</span> <span class="n">String</span> <span class="nf">getName</span><span class="o">()</span>
<span class="o">{</span>
<span class="k">return</span> <span class="s">"example-http"</span><span class="o">;</span>
<span class="o">}</span>
</pre></div>
</div>
<p>The real work in a connector factory happens in the <code class="docutils literal"><span class="pre">create()</span></code>
method. In the <code class="docutils literal"><span class="pre">ExampleConnectorFactory</span></code> class, the <code class="docutils literal"><span class="pre">create()</span></code> method
configures the connector and then asks Guice to create the object.
This is the meat of the <code class="docutils literal"><span class="pre">create()</span></code> method without parameter validation
and exception handling:</p>
<div class="highlight-java"><div class="highlight"><pre><span></span><span class="c1">// A plugin is not required to use Guice; it is just very convenient</span>
<span class="n">Bootstrap</span> <span class="n">app</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Bootstrap</span><span class="o">(</span>
<span class="k">new</span> <span class="n">JsonModule</span><span class="o">(),</span>
<span class="k">new</span> <span class="n">ExampleModule</span><span class="o">(</span><span class="n">connectorId</span><span class="o">));</span>
<span class="n">Injector</span> <span class="n">injector</span> <span class="o">=</span> <span class="n">app</span>
<span class="o">.</span><span class="na">strictConfig</span><span class="o">()</span>
<span class="o">.</span><span class="na">doNotInitializeLogging</span><span class="o">()</span>
<span class="o">.</span><span class="na">setRequiredConfigurationProperties</span><span class="o">(</span><span class="n">requiredConfig</span><span class="o">)</span>
<span class="o">.</span><span class="na">initialize</span><span class="o">();</span>
<span class="k">return</span> <span class="n">injector</span><span class="o">.</span><span class="na">getInstance</span><span class="o">(</span><span class="n">ExampleConnector</span><span class="o">.</span><span class="na">class</span><span class="o">);</span>
</pre></div>
</div>
<div class="section" id="connector-exampleconnector">
<h3>Connector: ExampleConnector</h3>
<p>This class allows Presto to obtain references to the various services
provided by the connector.</p>
</div>
<div class="section" id="metadata-examplemetadata">
<h3>Metadata: ExampleMetadata</h3>
<p>This class is responsible for reporting table names, table metadata,
column names, column metadata and other information about the schemas
that are provided by this connector. <code class="docutils literal"><span class="pre">ConnectorMetadata</span></code> is also called
by Presto to ensure that a particular connector can understand and
handle a given table name.</p>
<p>The <code class="docutils literal"><span class="pre">ExampleMetadata</span></code> implementation delegates many of these calls to
<code class="docutils literal"><span class="pre">ExampleClient</span></code>, a class that implements much of the core functionality
of the connector.</p>
</div>
<div class="section" id="split-manager-examplesplitmanager">
<h3>Split Manager: ExampleSplitManager</h3>
<p>The split manager partitions the data for a table into the individual
chunks that Presto will distribute to workers for processing.
In the case of the Example HTTP connector, each table contains one or
more URIs pointing at the actual data. One split is created per URI.</p>
</div>
<div class="section" id="record-set-provider-examplerecordsetprovider">
<h3>Record Set Provider: ExampleRecordSetProvider</h3>
<p>The record set provider creates a record set which in turn creates a
record cursor that returns the actual data to Presto.
<code class="docutils literal"><span class="pre">ExampleRecordCursor</span></code> reads data from a URI via HTTP. Each line
corresponds to a single row. Lines are split on comma into individual
field values which are then returned to Presto.</p>
</div>
</div>
</div>
</div>
<div class="bottomnav">
<p class="nav">
<span class="left">
« <a href="connectors.html">10.2. Connectors</a>
</span>
<span class="right">
<a href="types.html">10.4. Types</a> »
</span>
</p>
</div>
<div class="footer" role="contentinfo">
</div>
</body>
</html>