-
Notifications
You must be signed in to change notification settings - Fork 16
/
Copy pathconnectors.html
125 lines (112 loc) · 5.42 KB
/
connectors.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>10.2. Connectors — Presto 0.161 Documentation</title>
<link rel="stylesheet" href="../_static/presto.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '../',
VERSION: '0.161',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="top" title="Presto 0.161 Documentation" href="../index.html" />
<link rel="up" title="10. Developer Guide" href="../develop.html" />
<link rel="next" title="10.3. Example HTTP Connector" href="example-http.html" />
<link rel="prev" title="10.1. SPI Overview" href="spi-overview.html" />
</head>
<body role="document">
<div class="header">
<h1 class="heading"><a href="../index.html">
<span>Presto 0.161 Documentation</span></a></h1>
<h2 class="heading"><span>10.2. Connectors</span></h2>
</div>
<div class="topnav">
<p class="nav">
<span class="left">
« <a href="spi-overview.html">10.1. SPI Overview</a>
</span>
<span class="right">
<a href="example-http.html">10.3. Example HTTP Connector</a> »
</span>
</p>
</div>
<div class="content">
<div class="section" id="connectors">
<h1>10.2. Connectors</h1>
<p>Connectors are the source of all data for queries in Presto. Even if
your data source doesn’t have underlying tables backing it, as long as
you adapt your data source to the API expected by Presto, you can write
queries against this data.</p>
<div class="section" id="connectorfactory">
<h2>ConnectorFactory</h2>
<p>Instances of your connector are created by a <code class="docutils literal"><span class="pre">ConnectorFactory</span></code>
instance which is created when Presto calls <code class="docutils literal"><span class="pre">getConnectorFactory()</span></code> on the
plugin. The connector factory is a simple interface responsible for creating an
instance of a <code class="docutils literal"><span class="pre">Connector</span></code> object that returns instances of the
following services:</p>
<ul class="simple">
<li><code class="docutils literal"><span class="pre">ConnectorMetadata</span></code></li>
<li><code class="docutils literal"><span class="pre">ConnectorSplitManager</span></code></li>
<li><code class="docutils literal"><span class="pre">ConnectorHandleResolver</span></code></li>
<li><code class="docutils literal"><span class="pre">ConnectorRecordSetProvider</span></code></li>
</ul>
<div class="section" id="connectormetadata">
<h3>ConnectorMetadata</h3>
<p>The connector metadata interface has a large number of important
methods that are responsible for allowing Presto to look at lists of
schemas, lists of tables, lists of columns, and other metadata about a
particular data source.</p>
<p>This interface is too big to list in this documentation, but if you
are interested in seeing strategies for implementing these methods,
look at the <a class="reference internal" href="example-http.html"><span class="doc">Example HTTP Connector</span></a> and the Cassandra connector. If
your underlying data source supports schemas, tables and columns, this
interface should be straightforward to implement. If you are attempting
to adapt something that is not a relational database (as the Example HTTP
connector does), you may need to get creative about how you map your
data source to Presto’s schema, table, and column concepts.</p>
</div>
<div class="section" id="connectorsplitmanger">
<h3>ConnectorSplitManger</h3>
<p>The split manager partitions the data for a table into the individual
chunks that Presto will distribute to workers for processing.
For example, the Hive connector lists the files for each Hive
partition and creates one or more split per file.
For data sources that don’t have partitioned data, a good strategy
here is to simply return a single split for the entire table. This
is the strategy employed by the Example HTTP connector.</p>
</div>
<div class="section" id="connectorrecordsetprovider">
<h3>ConnectorRecordSetProvider</h3>
<p>Given a split and a list of columns, the record set provider is
responsible for delivering data to the Presto execution engine.
It creates a <code class="docutils literal"><span class="pre">RecordSet</span></code>, which in turn creates a <code class="docutils literal"><span class="pre">RecordCursor</span></code>
that is used by Presto to read the column values for each row.</p>
</div>
</div>
</div>
</div>
<div class="bottomnav">
<p class="nav">
<span class="left">
« <a href="spi-overview.html">10.1. SPI Overview</a>
</span>
<span class="right">
<a href="example-http.html">10.3. Example HTTP Connector</a> »
</span>
</p>
</div>
<div class="footer" role="contentinfo">
</div>
</body>
</html>