Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse code

overall cleanup and code improvements

  • Loading branch information...
commit ac47bf03278067cc235cbf8223a5a6487be9d33f 1 parent 166a874
Christian Parpart authored
7 README
@@ -8,7 +8,8 @@ ohloh: http://www.ohloh.net/p/x0
8 8 github: http://github.com/trapni/x0
9 9 gitorious: http://gitorious.org/x0
10 10
11   -x0 aims to be a low-latency scalarable http web server and web service framework written in C++.
  11 +x0 is a low-latency scalarable HTTP web server and web service framework
  12 +written in modern C++.
12 13
13 14 -----------------------------------------------------------------------------
14 15 FEATURES
@@ -19,16 +20,16 @@ FEATURES
19 20 - response output filter API
20 21 - name based virtual hosting
21 22 - zero-copy networking optimization through sendfile() system call
  23 +- transmitting of static files with partial response support and cache-friendly
  24 +- flow-based configuration system
22 25 - SSL connection encryption (plugin)
23 26 - SSL Server Name Indication (SNI) extension
24 27 - dynamic content compression (plugin)
25 28 - request path aliasing (plugin)
26 29 - automatic directory listing generation (plugin)
27   -- transmitting of static files with partial response support and cache-friendly (plugin)
28 30 - apache-style access log (plugin)
29 31 - automatic directory indexing (plugin)
30 32 - user-directory support (plugin)
31   -- LUA-based configuration system
32 33
33 34 -----------------------------------------------------------------------------
34 35 INSTALLATION REQUIREMENTS:
5 README.c++0x
@@ -8,11 +8,8 @@ Most importanly and though probably used by this software project:
8 8 * strongly typed enums
9 9 * initializer lists
10 10 * defaulted and deleted functions
11   -
12   -Not yet implemented by GCC (4.4) but very useful:
13   -
14 11 * lambda expressions and closures
15 12 * null pointer constant (nullptr)
16   -* concepts (especially ranges)
  13 +* range based for-keyword
17 14
18 15 [1] http://gcc.gnu.org/projects/cxx0x.html
2  TODO
... ... @@ -1,5 +1,7 @@
1 1 for a long-term todo list, see ticket system at http://redmine.xzero.ws/projects/x0
2 2
  3 +===================================================================================================
  4 +
3 5 - bug in timeout management, when the timeout has been reached the connection is not
4 6 properly cleaned up (see valgrind).
5 7 - bug in keep-alive connections sending a second+ request?
59 docs/io.txt
@@ -46,7 +46,7 @@ the actual derivate (e.g. *FileSource* or *SocketSink*).
46 46 SystemSource : system-handle chunk producer (reads from its handle)
47 47 FileSource : chunk producer for regular files
48 48 FilterSource : combines a source with a filter, producing filtered chunks
49   - CompositeSource : composes an ordered set of chunks into a virtual single one.
  49 + CompositeSource : composes an ordered set of chunks into a virtual single one.
50 50
51 51 SourceVisitor : visitor-pattern interface for the source types above.
52 52
@@ -69,7 +69,7 @@ the actual derivate (e.g. *FileSource* or *SocketSink*).
69 69
70 70 .: I/O flow :.
71 71
72   -When copying data from one end to another (e.g. a file from a server to a client),
  72 +When copying data from one end to another (e.g. a file from a server to a client)
73 73 common semantics are found:
74 74
75 75 - The server's file is called *Source* and the client is our *Sink*.
@@ -105,20 +105,13 @@ The *Source* class abstracts all kinds of chunk producers (files, buffers,
105 105 network sockets, composite set of different source types).
106 106
107 107 abstract interface:
108   - BufferRef pull(Buffer& output);
109   - bool eof() const;
110   - void accept(SourceVisitor& visitor);
  108 + virtual ssize_t sendto(Sink& sink);
111 109
112   -generic helper:
113   - void pump(Sink& output);
  110 +Every source specialization needs to implement the function(s) above.
114 111
115   -Every source specialization needs to implement the top 3 functions (pull, eof, accept):
116   -- pull() will actually produce the chunks
117   -- eof() will test on whether or not the end of the source stream has been reached.
118   -- accept() is required to invoke the visitor, passing itself, to implement the visitor-pattern.
119   -
120   -The pump() method on the other hand is not overridable, just invokes sink's pump,
121   -passing itself as argument, and is thus, just a convinience method for you.
  112 +A source is always mutable, that means, if you read from it, at least its read
  113 +position offset is adjusted accordingly to continue reading from that position
  114 +on next invokation. This applies to every source object.
122 115
123 116
124 117
@@ -128,13 +121,22 @@ Sinks will actually consume the chunks produced, thus, writing into a network
128 121 socket, a memory region, or regular file onto your local filesystem.
129 122
130 123 abstract interface:
131   - ssize_t push(Source& source);
  124 + virtual ssize_t ssize_t write(const void *buffer, size_t size);
  125 + virtual void accept(SinkVisitor& visitor);
  126 +
  127 +Each sink implementation must implement the methods above. The *write()*
  128 +method is usually invoked by the *Source* objects *sendto()* method
  129 +unless it does not provide a more intelligent way to accelerate
  130 +the data transfer (like splice() and sendfile() system-calls for
  131 +copy-free data transfer).
  132 +
132 133
133   -Each sink implementation just needs to implement this single method.
134   -Instead of passing a chunk here, we require the user to pass it directly the
135   -source, so that the sink implementation can perform possible transfer
136   -optimizations, such as the sendfile() system call instead of a read()
137   -plus write() if possible.
  134 +
  135 +.: Compositing Soources :.
  136 +
  137 +There is one special type of source (*CompositeSource*) that is used to
  138 +create a complex source by compositing multiple (different types of)
  139 +sources together into one linear stream.
138 140
139 141
140 142
@@ -144,18 +146,21 @@ Filters stand between the actual source and its sink to possibly
144 146 *transform* the transferring chunks.
145 147
146 148 Possible use-cases are compressors, chunked transfer encoders or even
147   -template engines (e.g. XSLT on XML formatted sources).
  149 +template engines (e.g. XSLT on XML formatted sources) to "transform"
  150 +the data to be transferred from one form into another before actually
  151 +copying it into the sink.
148 152
149 153 Filters are created on demand for a fixed source and sink.
150 154
151 155 abstract interface:
152   - Buffer process(const BufferRef& input, bool eof);
  156 + virtual Buffer process(const BufferRef& input, bool eof);
  157 +
  158 +Each filter is must implement the above method, which is invoked repeatily
  159 +over all produced chunks, and where its input parameter represents
  160 +the chunk to filter (transform) and eof is the flag, indicating whether
  161 +or not this is last chunk within this stream, thus, this is the last
  162 +invokation of *process()* before this filter is destroyed again.
153 163
154   -Each filter is required to implement the above process() method, which is
155   -invoked repeatily over all produced chunks, and where its input parameter represents
156   -the chunk to filter (transform) and eof is the flag, indicating whether or not this
157   -is last chunk within this stream, thus, this is the last invokation of
158   -process() before this filter is destroyed again.
159 164
160 165
161 166 vim:et:ts=4
4 include/x0/Socket.h
@@ -234,6 +234,10 @@ inline void Socket::setReadyCallback(K *object)
234 234 template<class K, void (K::*cb)(Socket *)>
235 235 inline void Socket::setTimeout(K *object, int value)
236 236 {
  237 +#if !defined(NDEBUG)
  238 + debug("setTimeout(%p, %d) active:%s", object, value, timer_.is_active() ? "true" : "false");
  239 +#endif
  240 +
237 241 timeoutCallback_ = &member_thunk<K, cb>;
238 242 timeoutData_ = object;
239 243
12 include/x0/http/HttpPlugin.h
@@ -27,6 +27,9 @@ namespace x0 {
27 27
28 28 class Params
29 29 {
  30 +public:
  31 + typedef const Flow::Value* iterator;
  32 +
30 33 private:
31 34 size_t count_;
32 35 const Flow::Value *params_;
@@ -36,15 +39,16 @@ class Params
36 39 Params(int count, Flow::Value *params) :
37 40 count_(count), params_(params) {}
38 41
  42 + iterator begin() const { return params_; }
  43 + iterator end() const { return params_ + count_; }
  44 +
39 45 bool empty() const { return count_ == 0; }
40 46 size_t count() const { return count_; }
41 47 const Flow::Value& at(size_t i) const { return params_[i]; }
42 48 const Flow::Value& operator[](size_t i) const { return params_[i]; }
43 49
44   - bool load(size_t i, bool& out) const;
45   - bool load(size_t i, long long& out) const;
46   - bool load(size_t i, std::string& out) const;
47   - // ...
  50 + template<typename T>
  51 + bool load(size_t i, T& out) const { return i < count_ ? at(i).load(out) : false; }
48 52 };
49 53
50 54 /**
28 include/x0/io/BufferSource.h
@@ -20,7 +20,7 @@ namespace x0 {
20 20
21 21 /** buffer source.
22 22 *
23   - * \see buffer, source, sink
  23 + * \see Buffer, Source, Sink
24 24 */
25 25 class X0_API BufferSource :
26 26 public Source
@@ -35,15 +35,9 @@ class X0_API BufferSource :
35 35 bool empty() const;
36 36
37 37 const Buffer& buffer() const;
38   - const Buffer *operator->() const;
39 38
40 39 virtual ssize_t sendto(Sink& sink);
41 40
42   -public:
43   - void clear();
44   - std::size_t bytes_consumed() const;
45   - std::size_t bytes_available() const;
46   -
47 41 private:
48 42 Buffer buffer_;
49 43 std::size_t pos_;
@@ -87,26 +81,6 @@ inline const Buffer& BufferSource::buffer() const
87 81 {
88 82 return buffer_;
89 83 }
90   -
91   -inline const Buffer *BufferSource::operator->() const
92   -{
93   - return &buffer_;
94   -}
95   -
96   -inline void BufferSource::clear()
97   -{
98   - pos_ = 0;
99   -}
100   -
101   -inline std::size_t BufferSource::bytes_consumed() const
102   -{
103   - return pos_;
104   -}
105   -
106   -inline std::size_t BufferSource::bytes_available() const
107   -{
108   - return buffer_.size() - pos_;
109   -}
110 84 // }}}
111 85
112 86 } // namespace x0
4 lib/base/Process.cpp
@@ -86,10 +86,10 @@ int Process::start(const std::string& exe, const ArgumentList& args, const Envir
86 86 */
87 87 void Process::terminate()
88 88 {
89   - fprintf(stderr, "Process(%d).terminate()\n", pid_);
  89 + fprintf(stderr, "Process(%ld).terminate()\n", pid_);
90 90 if (pid_ > 0) {
91 91 if (::kill(pid_, SIGTERM) < 0) {
92   - fprintf(stderr, "error sending SIGTERM to child %d\n", pid_);
  92 + fprintf(stderr, "error sending SIGTERM to child %ld: %s\n", pid_, strerror(errno));
93 93 }
94 94 }
95 95 }
5 lib/base/Socket.cpp
@@ -11,6 +11,7 @@
11 11 #include <x0/BufferRef.h>
12 12 #include <x0/Defines.h>
13 13 #include <x0/StackTrace.h>
  14 +#include <atomic>
14 15
15 16 #include <fcntl.h>
16 17 #include <sys/socket.h>
@@ -52,8 +53,10 @@ Socket::Socket(struct ev_loop *loop, int fd, int af) :
52 53 callback_(nullptr),
53 54 callbackData_(0)
54 55 {
  56 + debug(false);
55 57 #ifndef NDEBUG
56   - setLoggingPrefix("Socket(%s:%d)", remoteIP().c_str(), remotePort());
  58 + static std::atomic<unsigned long long> id(0);
  59 + setLoggingPrefix("Socket(%d, %s:%d)", ++id, remoteIP().c_str(), remotePort());
57 60 #endif
58 61 TRACE("created. fd:%d, local(%s:%d)", fd_, localIP().c_str(), localPort());
59 62
8 lib/http/HttpConnection.cpp
@@ -70,8 +70,10 @@ HttpConnection::HttpConnection(HttpListener& lst, HttpWorker& w, int fd) :
70 70 socket_ = listener_.socketDriver()->create(loop(), fd, lst.addressFamily());
71 71 sink_.setSocket(socket_);
72 72
  73 + debug(false);
73 74 #if !defined(NDEBUG)
74   - setLoggingPrefix("Connection[%s:%d]", remoteIP().c_str(), remotePort());
  75 + static std::atomic<unsigned long long> id(0);
  76 + setLoggingPrefix("Connection[%d,%s:%d]", ++id, remoteIP().c_str(), remotePort());
75 77 #endif
76 78
77 79 TRACE("fd=%d", socket_->handle());
@@ -423,9 +425,7 @@ void HttpConnection::processInput()
423 425 else
424 426 {
425 427 TRACE("processInput(): read %ld bytes", rv);
426   -
427   - //std::size_t offset = buffer_.size();
428   - //buffer_.resize(offset + rv);
  428 + TRACE("%s", buffer_.ref(buffer_.size() - rv).str().c_str());
429 429
430 430 process();
431 431
10 lib/http/HttpRequest.cpp
@@ -52,6 +52,7 @@ HttpRequest::HttpRequest(HttpConnection& conn) :
52 52 hostid_(),
53 53 readCallback_()
54 54 {
  55 + debug(false);
55 56 #ifndef NDEBUG
56 57 static std::atomic<unsigned long long> rid(0);
57 58 setLoggingPrefix("Request(%lld,%s:%d)", ++rid, connection.remoteIP().c_str(), connection.remotePort());
@@ -230,7 +231,7 @@ SourcePtr HttpRequest::serialize()
230 231 if (!connection.worker().server().max_keep_alive_idle())
231 232 keepalive = false;
232 233
233   - keepalive = false; // FIXME workaround
  234 + //keepalive = false; // FIXME workaround
234 235
235 236 if (!keepalive)
236 237 responseHeaders.overwrite("Connection", "close");
@@ -326,10 +327,13 @@ void HttpRequest::onFinished(int ec)
326 327 }
327 328
328 329 // close, if not a keep-alive connection
329   - if (iequals(responseHeaders["Connection"], "keep-alive"))
  330 + if (iequals(responseHeaders["Connection"], "keep-alive")) {
  331 + TRACE("onFinished: resuming");
330 332 connection.resume();
331   - else
  333 + } else {
  334 + TRACE("onFinished: closing");
332 335 connection.close();
  336 + }
333 337 }
334 338
335 339 /** finishes this response by flushing the content into the stream.
36 lib/io/FileInfoService.cpp
@@ -14,15 +14,19 @@
14 14
15 15 namespace x0 {
16 16
17   -#if 1
18   -# define FILEINFO_DEBUG(msg...) printf("FileInfoService: " msg)
  17 +#if 0 // !defined(NDEBUG)
  18 +# define TRACE(msg...) printf("FileInfoService: " msg)
19 19 #else
20   -# define FILEINFO_DEBUG(msg...) /*!*/
  20 +# define TRACE(msg...) /*!*/
  21 +#endif
  22 +
  23 +#if defined(HAVE_SYS_INOTIFY_H)
  24 +# undef HAVE_SYS_INOTIFY_H
21 25 #endif
22 26
23 27 FileInfoService::FileInfoService(struct ::ev_loop *loop, const Config *config) :
24 28 loop_(loop),
25   -#if defined(HAVE_SYS_INOTIFY_H)
  29 +#if 1 // defined(HAVE_SYS_INOTIFY_H)
26 30 handle_(-1),
27 31 inotify_(loop_),
28 32 inotifies_(),
@@ -72,11 +76,11 @@ FileInfoPtr FileInfoService::query(const std::string& _filename)
72 76 if (i != cache_.end()) {
73 77 FileInfoPtr fi = i->second;
74 78 if (isValid(fi.get())) {
75   - FILEINFO_DEBUG("query.cached(%s) len:%ld\n", filename.c_str(), fi->size());
  79 + TRACE("query.cached(%s) len:%ld\n", filename.c_str(), fi->size());
76 80 return fi;
77 81 }
78 82
79   - FILEINFO_DEBUG("query.expired(%s) len:%ld\n", filename.c_str(), fi->size());
  83 + TRACE("query.expired(%s) len:%ld\n", filename.c_str(), fi->size());
80 84 #if defined(HAVE_SYS_INOTIFY_H)
81 85 if (fi->inotifyId_ >= 0) {
82 86 inotify_rm_watch(handle_, fi->inotifyId_);
@@ -98,7 +102,7 @@ FileInfoPtr FileInfoService::query(const std::string& _filename)
98 102 /*IN_ONESHOT |*/ IN_ATTRIB | IN_DELETE_SELF | IN_MOVE_SELF | IN_UNMOUNT |
99 103 IN_DELETE_SELF | IN_MOVE_SELF)
100 104 : -1;
101   - FILEINFO_DEBUG("query(%s).new -> %d len:%ld\n", filename.c_str(), wd, fi->size());
  105 + TRACE("query(%s).new -> %d len:%ld\n", filename.c_str(), wd, fi->size());
102 106
103 107 if (wd != -1) {
104 108 fi->inotifyId_ = wd;
@@ -106,14 +110,14 @@ FileInfoPtr FileInfoService::query(const std::string& _filename)
106 110 }
107 111 cache_[filename] = fi;
108 112 #else
109   - FILEINFO_DEBUG("query(%s)! len:%ld\n", filename.c_str(), fi->size());
  113 + TRACE("query(%s)! len:%ld\n", filename.c_str(), fi->size());
110 114 cache_[filename] = fi;
111 115 #endif
112 116
113 117 return fi;
114 118 }
115 119
116   - FILEINFO_DEBUG("query(%s) failed (%s)\n", filename.c_str(), strerror(errno));
  120 + TRACE("query(%s) failed (%s)\n", filename.c_str(), strerror(errno));
117 121 // either ::stat() or caching failed.
118 122
119 123 return FileInfoPtr();
@@ -123,11 +127,11 @@ FileInfoPtr FileInfoService::query(const std::string& _filename)
123 127 #if defined(HAVE_SYS_INOTIFY_H)
124 128 void FileInfoService::onFileChanged(ev::io& w, int revents)
125 129 {
126   - FILEINFO_DEBUG("onFileChanged()\n");
  130 + TRACE("onFileChanged()\n");
127 131
128 132 char buf[sizeof(inotify_event) * 256];
129 133 ssize_t rv = ::read(handle_, &buf, sizeof(buf));
130   - FILEINFO_DEBUG("read returned: %ld (%% %ld, %ld)\n",
  134 + TRACE("read returned: %ld (%% %ld, %ld)\n",
131 135 rv, sizeof(inotify_event), rv / sizeof(inotify_event));
132 136
133 137 if (rv > 0) {
@@ -136,23 +140,23 @@ void FileInfoService::onFileChanged(ev::io& w, int revents)
136 140 inotify_event *ev = (inotify_event *)i;
137 141
138 142 for (; i < e && ev->wd != 0; i += sizeof(*ev) + ev->len, ev = (inotify_event *)i) {
139   - FILEINFO_DEBUG("traverse: (wd:%d, mask:0x%04x, cookie:%d)\n", ev->wd, ev->mask, ev->cookie);
  143 + TRACE("traverse: (wd:%d, mask:0x%04x, cookie:%d)\n", ev->wd, ev->mask, ev->cookie);
140 144 auto wi = inotifies_.find(ev->wd);
141 145 if (wi == inotifies_.end()) {
142   - FILEINFO_DEBUG("-skipping\n");
  146 + TRACE("-skipping\n");
143 147 continue;
144 148 }
145 149
146 150 auto k = cache_.find(wi->second->filename());
147   - FILEINFO_DEBUG("invalidate: %s\n", k->first.c_str());
  151 + TRACE("invalidate: %s\n", k->first.c_str());
148 152 // onInvalidate(k->first, k->second);
149 153 cache_.erase(k);
150 154 inotifies_.erase(wi);
151 155 int rv = inotify_rm_watch(handle_, ev->wd);
152 156 if (rv < 0) {
153   - FILEINFO_DEBUG("error removing inotify watch (%d, %s): %s\n", ev->wd, ev->name, strerror(errno));
  157 + TRACE("error removing inotify watch (%d, %s): %s\n", ev->wd, ev->name, strerror(errno));
154 158 } else {
155   - FILEINFO_DEBUG("inotify_rm_watch: %d (ok)\n", rv);
  159 + TRACE("inotify_rm_watch: %d (ok)\n", rv);
156 160 }
157 161 }
158 162 }

0 comments on commit ac47bf0

Please sign in to comment.
Something went wrong with that request. Please try again.