/
ms-rfc-71.txt
200 lines (161 loc) · 9.03 KB
/
ms-rfc-71.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
.. _rfc71:
=========================================================================
MS RFC 71: Integration of Mod-Geocache in the MapServer project
=========================================================================
:Date: 2011/05/20
:Author: Thomas Bonfort
:Contact: tbonfort at terriscope dot fr
:Status: Adopted
:Version: MapServer 7.0
Description: This RFC proposes the integration of `mod-geocache <http://code.google.com/p/mod-geocache/>`__
in the MapServer project.
1. Overview
-----------
`Mod-Geocache <http://code.google.com/p/mod-geocache/>`__ is a recent tile caching solution that
runs in front of a WMS server either as an apache module or as a fastCGI executable. It can
connect to any WMS compliant server, but given its source code language and the roots of its
core developpers, it would make sense to integrate it tightly with the mapserver project,
to ease configuration for end-users, and provide higher QA through a larger development team.
1.1 The need for a tile caching solution
----------------------------------------
The usage patterns of web mapping have evolved in the past years and is more and more based around
the pre-generation of image tiles rather than the direct creation of images from the source data.
Current solutions for serving map tiles include TileCache, MapProxy, and GeoWebCache, all of which
accomplish the task successfully but in a manner that does not integrate ideally with the MapServer
project:
- They do not run native code, which incurs a minor performance overhead, and can be an inconvenience
in the context of high performance data dissemination
- They do not share their configuration with the server they are proxying, as such requiring users
to duplicate configuration variables and metadata.
- In the case of the python servers, they require an additional component to function (mod-python,
mod-wsgi, or fastcgi wrapper) that is cumbersome to configure on non-mainstream platforms. The GIL
is also a bottleneck for multi-threaded servers for the requests that require an access to native
code (notably related to image format encoding and decoding).
- NIH ;) and as it can be written in C, why not use that directly?
1.2 The mod-geocache project
----------------------------
Mod-geocache is a recent project (oct 2010), distributed under an apache license. It is written in
plain C using a semi object-oriented architecture (heavy use of structure inheritance and function
pointers). Being primarily aimed at being run as an apache module, it is very fast as it runs as native
code inside the process treating the http requests (there is no overhead in cgi process creation or
fastcgi ipc).
Its list of features include:
* services WMS, WMTS, TMS, VirtualEarth/Bing and GoogleMaps requests
* ability to respond to untiled WMS requests by merging tiles from the cache or forwarding
them to the wms source
* responds to WMS/WMTS GetFeatureInfo requests (forwarded to source service)
* KML superoverlay generation
* data provided by WMS backends (GDAL supported sources planned)
* experimental memcached cache
* configurable metatiling, with inter-process/inter-thread locking and synchronization
to avoid overloading the source wms when the tiles are not seeded yet.
* on-the-fly tile merging for combining multiple tiles into a single image
* image post-processing (recompression and quantization) when arriving from a backend
* interprets and produces cache control headers: Last-Modified, If-Modified-Since, Expires
* multithreaded seeding utility
* ability to add a custom watermark on stored tiles
* can produce a fastCGI executable for using with other webservers than apache
* configurable symbolic linking of blank tiles to gain disk storage
* configurable error reporting: plain http error code, textual message, or empty (blank) image
* ability to specify vendor params or dimensions to be forwarded to the WMS backend
(and build a cache that takes these parameters into account)
* rule-based proxying of non-tile requests to (an)other server(s), e.g. for WFS, WFS-T, etc...
1.3 Integration Overview
------------------------
Mod-geocache will be modified to read its configuration from a supplied mapfile rather than its own
xml file as is now. When used in conjunction with mapserver and tinyOWS, it will act as a proxy to
these servers: if an incoming request corresponds to soemthing that can be served from cache it will
act as a classic tile server, if not it will proxy the request to mapserver or tinyows (or any other
OGC compliant server).
2. Governance
-------------
2.1 Finding a Name
------------------
"Mod-Geocache" as a name is a poor choice, and should be changed during the integration with MapServer.
The author has no predefined idea as to what the solution should be named, ideas will be greatly
appreciated.
2.2 Release Cycles
------------------
Mod-geocache being a recent project, it is undergoing development at a pace that is incompatible
with the MapServer release cycle. As such, the release cycle of mod-geocache will be at first
decoupled from the mapserver one: mod-geocache releases will be produced any time there is a mapserver
release, but also at a faster rate if needed.
2.3 Source Code Location
------------------------
The mod-geocache code will be located in a subdirectory of the mapserver repository, e.g.
trunk/mapserver/mod-geocache
2.4 RFCs and Decision Process
-----------------------------
In the early steps of the integration, development surrounding the core of mod-geocache unrelated to
the MapServer project will be undertaken in a relaxed manner compared to the RFC based decision taking
that prevails for MapServer.
All non trivial changes to the mod-geocache core will be announced for discussion on the mapserver-dev
mailing list, but will not undergo the rfc voting process unless there is a direct interaction with the
actual mapserver functionality.
Once the transitionning phase of the integration has been completed, the development on mod-geocache
will follow the traditional RFC based decision taking.
2.5 Licence
-----------
The apache licence of mod-geocache will be switched to the same licence as MapServer.
2.6 Tickets
-----------
Mod-geocache is hosted on google-code and uses its integrated issue tracking system. The MapServer
trac instance will be used, once a dedicated component for mod-geocache has been created.
There are currently no open bugs to transition to the MapServer trac. The open feature enhancements
will be manually migrated to the MapServer trac instance.
2.7 SVN Import
--------------
Mod-geocache trunk will be imported in MapServer SVN.
Previous commit history will be kept on the old google-code SVN, for a while.
3 A common MapFile as config file
-----------------------------------
3.1 LibMapfile API
------------------
A LibMapFile API will be created out of the mapserver source tree, and will require some refactoring
of header and source files. The MapServer API itself should not be changed as the refactoring will mostly
concern moving some functions and declarations out into separate files.
3.2 Configuration Directives
----------------------------
New keywords and/or metadata will have to be added to the mapserver parser. The exact syntax and the
extent of the configuration keywords must be discussed as part of the commenting period of this RFC.
The configuration directives being rather extensive, a tradeoff will have to be found between ease of
syntax and similarity to actual mapserver keywords.
3.3 Documentation
-----------------
The mod-geocache documentation is rather sparse, and consists essentially of a commented configuration
file. There will have to be an effort to add documentation to the main mapserver doc site, and to
migrate the few wiki pages from the google-code hosting.
4. Code Review
---------------------
4.1 Code Origin
---------------
All C code was written from scratch by myself and Steve Woodbridge, so should not be a
real problem. Image io functions based on libpng/libjpeg have been vastly inspired by
the same functions in mapserver, and could be merged together in a second phase.
4.2 Code Quality
----------------
Similar to MapServer.
4.3 Security Audit
------------------
No external security audit has been performed on the code.
Being run as an apache module that potentially receives untrusted data, security has always been a
concern when writing the code, but I'm no security expert.
4.4 External Dependencies
-------------------------
- libcurl (required)
- apr(required) / apr-util(optional)
- libpng (required)
- libjpeg (required)
- pcre (optional)
- OGR (for the seeder, optional)
- libcairo (optional, required for servicing GetMap requests built from cached tiles)
- FastCGI (optional)
4.5 Build system
----------------
mod-geocache like MapServer uses autoconf (without automake) and Makefiles to build stuff.
4.6 IP and Patent overview
--------------------------
Patent review is left as an exercise for users coming from countries with a broken patent system ;)
5. Voting history
-----------------
Passed with +1's from Steve L., Steve W., Tamas, Dan, Howard, Assefa, Thomas and Tom on 8/19/2011.