Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

rebased fixes for 1.4.x codeline #2

Open
wants to merge 6 commits into from

4 participants

@lindner

Hi, Finally got around to rebasing my 1.4.x branch

Note that this compiles under gcc 4.4 without problems and without the patch that was submitted for that fix.

Would appreciate it if you could take a second look at these patches.

Paul Lindner and others added some commits
@dustin
Owner

This feels like a lot of changes for a maintenance branch. I don't know think we can easily merge these changes up to the development branch. As most of this is performance, it seems like it's happening in the wrong place.

@lindner

Dustin -- how long have we been waiting for the next branch? These changes are fine and shouldn't affect what you're doing in your fork, plus they happen to fix an issue.

Once you guys get serious about getting an alpha and a roadmap out the door is when I'll get serious about working off your engine branch.

@trondn
Collaborator

I'd rather see the 116 issue being replaced with a proper description about the current threading than just nuking away the document. The threading logic is the same in memcached, the problem is just that you can't disable it anymore... Apart from that all I can see from the change history is unlikely/likely changes.. what is the performance improvement in % for these intrusive changes?

@trondn
Collaborator

Any updates on this?

@lindner

Haven't had time to address your comments yet. Maybe this weekend. Been a bit consumed with the new job and tending to shindig lately...

@zwChan zwChan referenced this pull request
Closed

solution for Issue #260: #67

@dormando
Owner

Never closed this since I'm curious if the likely/unlikely stuff actually makes a difference in performance tests...

Four years is pretty bitrotted though. Sorry. guess I'll leave it open a bit longer before giving up?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Oct 6, 2010
  1. @lindner

    Small optimization for usage of settings.verbose

    Paul Lindner authored lindner committed
    - use unsigned int, since this value can never be negative
    - use unlikely() macro to help with branch prediction logic
    - replace (settings.verbose > 0) with just (settings.verbose)
  2. @lindner

    Improve memory efficiency by avoiding structure padding, especially o…

    Paul Lindner authored lindner committed
    …n 64 bit systems
    
    - saves 32 bytes per connection
  3. @lindner
  4. @lindner
  5. @lindner
  6. @lindner
This page is out of date. Refresh to see the latest.
View
2  .gitignore
@@ -1,4 +1,4 @@
-*.[ao]
+*.[aois]
*.gcov
*.gcda
*.gcno
View
14 assoc.c
@@ -71,8 +71,8 @@ item *assoc_find(const char *key, const size_t nkey) {
item *it;
unsigned int oldbucket;
- if (expanding &&
- (oldbucket = (hv & hashmask(hashpower - 1))) >= expand_bucket)
+ if (unlikely(expanding) &&
+ unlikely((oldbucket = (hv & hashmask(hashpower - 1))) >= expand_bucket))
{
it = old_hashtable[oldbucket];
} else {
@@ -101,7 +101,7 @@ static item** _hashitem_before (const char *key, const size_t nkey) {
item **pos;
unsigned int oldbucket;
- if (expanding &&
+ if (unlikely(expanding) &&
(oldbucket = (hv & hashmask(hashpower - 1))) >= expand_bucket)
{
pos = &old_hashtable[oldbucket];
@@ -121,7 +121,7 @@ static void assoc_expand(void) {
primary_hashtable = calloc(hashsize(hashpower + 1), sizeof(void *));
if (primary_hashtable) {
- if (settings.verbose > 1)
+ if (unlikely(settings.verbose > 1))
fprintf(stderr, "Hash table expansion starting\n");
hashpower++;
expanding = true;
@@ -141,7 +141,7 @@ int assoc_insert(item *it) {
assert(assoc_find(ITEM_key(it), it->nkey) == 0); /* shouldn't have duplicately named things defined */
hv = hash(ITEM_key(it), it->nkey, 0);
- if (expanding &&
+ if (unlikely(expanding) &&
(oldbucket = (hv & hashmask(hashpower - 1))) >= expand_bucket)
{
it->h_next = old_hashtable[oldbucket];
@@ -152,7 +152,7 @@ int assoc_insert(item *it) {
}
hash_items++;
- if (! expanding && hash_items > (hashsize(hashpower) * 3) / 2) {
+ if (! unlikely(expanding) && hash_items > (hashsize(hashpower) * 3) / 2) {
assoc_expand();
}
@@ -213,7 +213,7 @@ static void *assoc_maintenance_thread(void *arg) {
if (expand_bucket == hashsize(hashpower - 1)) {
expanding = false;
free(old_hashtable);
- if (settings.verbose > 1)
+ if (unlikely(settings.verbose > 1))
fprintf(stderr, "Hash table expansion done\n");
}
}
View
68 doc/threads.txt
@@ -1,68 +0,0 @@
-Multithreading support in memcached
-
-OVERVIEW
-
-By default, memcached is compiled as a single-threaded application. This is
-the most CPU-efficient mode of operation, and it is appropriate for memcached
-instances running on single-processor servers or whose request volume is
-low enough that available CPU power is not a bottleneck.
-
-More heavily-used memcached instances can benefit from multithreaded mode.
-To enable it, use the "--enable-threads" option to the configure script:
-
-./configure --enable-threads
-
-You must have the POSIX thread functions (pthread_*) on your system in order
-to use memcached's multithreaded mode.
-
-Once you have a thread-capable memcached executable, you can control the
-number of threads using the "-t" option; the default is 4. On a machine
-that's dedicated to memcached, you will typically want one thread per
-processor core. Due to memcached's nonblocking architecture, there is no
-real advantage to using more threads than the number of CPUs on the machine;
-doing so will increase lock contention and is likely to degrade performance.
-
-
-INTERNALS
-
-The threading support is mostly implemented as a series of wrapper functions
-that protect calls to underlying code with one of a small number of locks.
-In single-threaded mode, the wrappers are replaced with direct invocations
-of the target code using #define; that is done in memcached.h. This approach
-allows memcached to be compiled in either single- or multi-threaded mode.
-
-Each thread has its own instance of libevent ("base" in libevent terminology).
-The only direct interaction between threads is for new connections. One of
-the threads handles the TCP listen socket; each new connection is passed to
-a different thread on a round-robin basis. After that, each thread operates
-on its set of connections as if it were running in single-threaded mode,
-using libevent to manage nonblocking I/O as usual.
-
-UDP requests are a bit different, since there is only one UDP socket that's
-shared by all clients. The UDP socket is monitored by all of the threads.
-When a datagram comes in, all the threads that aren't already processing
-another request will receive "socket readable" callbacks from libevent.
-Only one thread will successfully read the request; the others will go back
-to sleep or, in the case of a very busy server, will read whatever other
-UDP requests are waiting in the socket buffer. Note that in the case of
-moderately busy servers, this results in increased CPU consumption since
-threads will constantly wake up and find no input waiting for them. But
-short of much more major surgery on the I/O code, this is not easy to avoid.
-
-
-TO DO
-
-The locking is currently very coarse-grained. There is, for example, one
-lock that protects all the calls to the hashtable-related functions. Since
-memcached spends much of its CPU time on command parsing and response
-assembly, rather than managing the hashtable per se, this is not a huge
-bottleneck for small numbers of processors. However, the locking will likely
-have to be refined in the event that memcached needs to run well on
-massively-parallel machines.
-
-One cheap optimization to reduce contention on that lock: move the hash value
-computation so it occurs before the lock is obtained whenever possible.
-Right now the hash is performed at the lowest levels of the functions in
-assoc.c. If instead it was computed in memcached.c, then passed along with
-the key and length into the items.c code and down into assoc.c, that would
-reduce the amount of time each thread needs to keep the hashtable lock held.
View
4 items.c
@@ -466,7 +466,7 @@ item *do_item_get(const char *key, const size_t nkey) {
item *it = assoc_find(key, nkey);
int was_found = 0;
- if (settings.verbose > 2) {
+ if (unlikely(settings.verbose > 2)) {
if (it == NULL) {
fprintf(stderr, "> NOT FOUND %s", key);
} else {
@@ -501,7 +501,7 @@ item *do_item_get(const char *key, const size_t nkey) {
DEBUG_REFCNT(it, '+');
}
- if (settings.verbose > 2)
+ if (unlikely(settings.verbose > 2))
fprintf(stderr, "\n");
return it;
View
146 memcached.c
@@ -208,7 +208,7 @@ static int add_msghdr(conn *c)
assert(c != NULL);
- if (c->msgsize == c->msgused) {
+ if (unlikely(c->msgsize == c->msgused)) {
msg = realloc(c->msglist, c->msgsize * 2 * sizeof(struct msghdr));
if (! msg)
return -1;
@@ -224,7 +224,7 @@ static int add_msghdr(conn *c)
msg->msg_iov = &c->iov[c->iovused];
- if (c->request_addr_size > 0) {
+ if (likely(c->request_addr_size > 0)) {
msg->msg_name = &c->request_addr;
msg->msg_namelen = c->request_addr_size;
}
@@ -377,7 +377,7 @@ conn *conn_new(const int sfd, enum conn_states init_state,
c->request_addr_size = 0;
}
- if (settings.verbose > 1) {
+ if (unlikely(settings.verbose > 1)) {
if (init_state == conn_listening) {
fprintf(stderr, "<%d server listening (%s)\n", sfd,
prot_text(c->protocol));
@@ -503,7 +503,7 @@ static void conn_close(conn *c) {
/* delete the event, the socket and the conn */
event_del(&c->event);
- if (settings.verbose > 1)
+ if (unlikely(settings.verbose > 1))
fprintf(stderr, "<%d connection closed.\n", c->sfd);
MEMCACHED_CONN_RELEASE(c->sfd);
@@ -607,8 +607,8 @@ static void conn_set_state(conn *c, enum conn_states state) {
assert(c != NULL);
assert(state >= conn_listening && state < conn_max_state);
- if (state != c->state) {
- if (settings.verbose > 2) {
+ if (likely(state != c->state)) {
+ if (unlikely(settings.verbose > 2)) {
fprintf(stderr, "%d: going from %s to %s\n",
c->sfd, state_text(c->state),
state_text(state));
@@ -754,14 +754,14 @@ static void out_string(conn *c, const char *str) {
assert(c != NULL);
if (c->noreply) {
- if (settings.verbose > 1)
+ if (unlikely(settings.verbose))
fprintf(stderr, ">%d NOREPLY %s\n", c->sfd, str);
c->noreply = false;
conn_set_state(c, conn_new_cmd);
return;
}
- if (settings.verbose > 1)
+ if (unlikely(settings.verbose > 1))
fprintf(stderr, ">%d %s\n", c->sfd, str);
/* Nuke a partial output... */
@@ -802,7 +802,7 @@ static void complete_nread_ascii(conn *c) {
c->thread->stats.slab_stats[it->slabs_clsid].set_cmds++;
pthread_mutex_unlock(&c->thread->stats.mutex);
- if (strncmp(ITEM_data(it) + it->nbytes - 2, "\r\n", 2) != 0) {
+ if (unlikely(strncmp(ITEM_data(it) + it->nbytes - 2, "\r\n", 2) != 0)) {
out_string(c, "CLIENT_ERROR bad data chunk");
} else {
ret = store_item(it, comm, c);
@@ -887,7 +887,7 @@ static void add_bin_header(conn *c, uint16_t err, uint8_t hdr_len, uint16_t key_
c->msgcurr = 0;
c->msgused = 0;
c->iovused = 0;
- if (add_msghdr(c) != 0) {
+ if (unlikely(add_msghdr(c) != 0)) {
/* XXX: out_string is inappropriate here */
out_string(c, "SERVER_ERROR out of memory");
return;
@@ -907,7 +907,7 @@ static void add_bin_header(conn *c, uint16_t err, uint8_t hdr_len, uint16_t key_
header->response.opaque = c->opaque;
header->response.cas = htonll(c->cas);
- if (settings.verbose > 1) {
+ if (unlikely(settings.verbose > 1)) {
int ii;
fprintf(stderr, ">%d Writing bin response:", c->sfd);
for (ii = 0; ii < sizeof(header->bytes); ++ii) {
@@ -960,7 +960,7 @@ static void write_bin_error(conn *c, protocol_binary_response_status err, int sw
fprintf(stderr, ">%d UNHANDLED ERROR: %d\n", c->sfd, err);
}
- if (settings.verbose > 1) {
+ if (unlikely(settings.verbose > 1)) {
fprintf(stderr, ">%d Writing an error: %s\n", c->sfd, errstr);
}
@@ -1011,7 +1011,7 @@ static void complete_incr_bin(conn *c) {
key = binary_get_key(c);
nkey = c->binary_header.request.keylen;
- if (settings.verbose > 1) {
+ if (unlikely(settings.verbose > 1)) {
int i;
fprintf(stderr, "incr ");
@@ -1168,7 +1168,7 @@ static void process_bin_get(conn *c) {
char* key = binary_get_key(c);
size_t nkey = c->binary_header.request.keylen;
- if (settings.verbose > 1) {
+ if (unlikely(settings.verbose > 1)) {
int ii;
fprintf(stderr, "<%d GET ", c->sfd);
for (ii = 0; ii < nkey; ++ii) {
@@ -1294,7 +1294,7 @@ static bool grow_stats_buf(conn *c, size_t needed) {
bool rv = true;
/* Special case: No buffer -- need to allocate fresh */
- if (c->stats.buffer == NULL) {
+ if (unlikely(c->stats.buffer == NULL)) {
nsize = 1024;
available = c->stats.size = c->stats.offset = 0;
}
@@ -1350,7 +1350,7 @@ static void process_bin_stat(conn *c) {
char *subcommand = binary_get_key(c);
size_t nkey = c->binary_header.request.keylen;
- if (settings.verbose > 1) {
+ if (unlikely(settings.verbose > 1)) {
int ii;
fprintf(stderr, "<%d STATS ", c->sfd);
for (ii = 0; ii < nkey; ++ii) {
@@ -1428,13 +1428,13 @@ static void bin_read_key(conn *c, enum bin_substates next_substate, int extra) {
}
if (nsize != c->rsize) {
- if (settings.verbose > 1) {
+ if (unlikely(settings.verbose > 1)) {
fprintf(stderr, "%d: Need to grow buffer from %lu to %lu\n",
c->sfd, (unsigned long)c->rsize, (unsigned long)nsize);
}
char *newm = realloc(c->rbuf, nsize);
if (newm == NULL) {
- if (settings.verbose) {
+ if (unlikely(settings.verbose)) {
fprintf(stderr, "%d: Failed to grow buffer.. closing connection\n",
c->sfd);
}
@@ -1450,7 +1450,7 @@ static void bin_read_key(conn *c, enum bin_substates next_substate, int extra) {
if (c->rbuf != c->rcurr) {
memmove(c->rbuf, c->rcurr, c->rbytes);
c->rcurr = c->rbuf;
- if (settings.verbose > 1) {
+ if (unlikely(settings.verbose > 1)) {
fprintf(stderr, "%d: Repack input buffer\n", c->sfd);
}
}
@@ -1464,7 +1464,7 @@ static void bin_read_key(conn *c, enum bin_substates next_substate, int extra) {
/* Just write an error message and disconnect the client */
static void handle_binary_protocol_error(conn *c) {
write_bin_error(c, PROTOCOL_BINARY_RESPONSE_EINVAL, 0);
- if (settings.verbose) {
+ if (unlikely(settings.verbose)) {
fprintf(stderr, "Protocol error (opcode %02x), close connection %d\n",
c->binary_header.request.opcode, c->sfd);
}
@@ -1482,7 +1482,7 @@ static void init_sasl_conn(conn *c) {
NULL, NULL, NULL, NULL,
NULL, 0, &c->sasl_conn);
if (result != SASL_OK) {
- if (settings.verbose) {
+ if (unlikely(settings.verbose)) {
fprintf(stderr, "Failed to initialize SASL conn.\n");
}
c->sasl_conn = NULL;
@@ -1510,7 +1510,7 @@ static void bin_list_sasl_mechs(conn *c) {
NULL);
if (result != SASL_OK) {
/* Perhaps there's a better error for this... */
- if (settings.verbose) {
+ if (unlikely(settings.verbose)) {
fprintf(stderr, "Failed to list SASL mechanisms.\n");
}
write_bin_error(c, PROTOCOL_BINARY_RESPONSE_AUTH_ERROR, 0);
@@ -1572,7 +1572,7 @@ static void process_bin_complete_sasl_auth(conn *c) {
memcpy(mech, ITEM_key((item*)c->item), nkey);
mech[nkey] = 0x00;
- if (settings.verbose)
+ if (unlikely(settings.verbose))
fprintf(stderr, "mech: ``%s'' with %d bytes of data\n", mech, vlen);
const char *challenge = vlen == 0 ? NULL : ITEM_data((item*) c->item);
@@ -1594,7 +1594,7 @@ static void process_bin_complete_sasl_auth(conn *c) {
assert(false); /* CMD should be one of the above */
/* This code is pretty much impossible, but makes the compiler
happier */
- if (settings.verbose) {
+ if (unlikely(settings.verbose)) {
fprintf(stderr, "Unhandled command %d with challenge %s\n",
c->cmd, challenge);
}
@@ -1603,7 +1603,7 @@ static void process_bin_complete_sasl_auth(conn *c) {
item_unlink(c->item);
- if (settings.verbose) {
+ if (unlikely(settings.verbose)) {
fprintf(stderr, "sasl result code: %d\n", result);
}
@@ -1623,7 +1623,7 @@ static void process_bin_complete_sasl_auth(conn *c) {
c->write_and_go = conn_new_cmd;
break;
default:
- if (settings.verbose)
+ if (unlikely(settings.verbose))
fprintf(stderr, "Unknown sasl response: %d\n", result);
write_bin_error(c, PROTOCOL_BINARY_RESPONSE_AUTH_ERROR, 0);
pthread_mutex_lock(&c->thread->stats.mutex);
@@ -1652,7 +1652,7 @@ static bool authenticated(conn *c) {
}
}
- if (settings.verbose > 1) {
+ if (unlikely(settings.verbose > 1)) {
fprintf(stderr, "authenticated() in cmd 0x%02x is %s\n",
c->cmd, rv ? "true" : "false");
}
@@ -1846,7 +1846,7 @@ static void process_bin_update(conn *c) {
vlen = c->binary_header.request.bodylen - (nkey + c->binary_header.request.extlen);
- if (settings.verbose > 1) {
+ if (unlikely(settings.verbose > 1)) {
int ii;
if (c->cmd == PROTOCOL_BINARY_CMD_ADD) {
fprintf(stderr, "<%d ADD ", c->sfd);
@@ -1931,7 +1931,7 @@ static void process_bin_append_prepend(conn *c) {
nkey = c->binary_header.request.keylen;
vlen = c->binary_header.request.bodylen - nkey;
- if (settings.verbose > 1) {
+ if (unlikely(settings.verbose > 1)) {
fprintf(stderr, "Value len is %d\n", vlen);
}
@@ -2006,7 +2006,7 @@ static void process_bin_delete(conn *c) {
assert(c != NULL);
- if (settings.verbose > 1) {
+ if (unlikely(settings.verbose > 1)) {
fprintf(stderr, "Deleting %s\n", key);
}
@@ -2145,7 +2145,7 @@ enum store_item_type do_store_item(item *it, int comm, conn *c) {
c->thread->stats.slab_stats[old_it->slabs_clsid].cas_badval++;
pthread_mutex_unlock(&c->thread->stats.mutex);
- if(settings.verbose > 1) {
+ if (unlikely(settings.verbose > 1)) {
fprintf(stderr, "CAS: failure: expected %llu, got %llu\n",
(unsigned long long)ITEM_get_cas(old_it),
(unsigned long long)ITEM_get_cas(it));
@@ -2291,7 +2291,7 @@ static size_t tokenize_command(char *command, token_t *tokens, const size_t max_
/* set up a connection to write a buffer then free it, used for stats */
static void write_and_free(conn *c, char *buf, int bytes) {
- if (buf) {
+ if (likely(buf != NULL)) {
c->write_and_free = buf;
c->wcurr = buf;
c->wbytes = bytes;
@@ -2453,7 +2453,7 @@ static void process_stat(conn *c, token_t *tokens, const size_t ntokens) {
const char *subcommand = tokens[SUBCOMMAND_TOKEN].value;
assert(c != NULL);
- if (ntokens < 2) {
+ if (unlikely(ntokens < 2)) {
out_string(c, "CLIENT_ERROR bad command line");
return;
}
@@ -2479,7 +2479,7 @@ static void process_stat(conn *c, token_t *tokens, const size_t ntokens) {
char *buf;
unsigned int bytes, id, limit = 0;
- if (ntokens < 5) {
+ if (unlikely(ntokens < 5)) {
out_string(c, "CLIENT_ERROR bad command line");
return;
}
@@ -2490,7 +2490,7 @@ static void process_stat(conn *c, token_t *tokens, const size_t ntokens) {
return;
}
- if (id >= POWER_LARGEST) {
+ if (unlikely(id >= POWER_LARGEST)) {
out_string(c, "CLIENT_ERROR Illegal slab id");
return;
}
@@ -2502,7 +2502,7 @@ static void process_stat(conn *c, token_t *tokens, const size_t ntokens) {
/* getting here means that the subcommand is either engine specific or
is invalid. query the engine and see. */
if (get_stats(subcommand, strlen(subcommand), &append_stats, c)) {
- if (c->stats.buffer == NULL) {
+ if (unlikely(c->stats.buffer == NULL)) {
out_string(c, "SERVER_ERROR out of memory writing stats");
} else {
write_and_free(c, c->stats.buffer, c->stats.offset);
@@ -2517,7 +2517,7 @@ static void process_stat(conn *c, token_t *tokens, const size_t ntokens) {
/* append terminator and start the transfer */
append_stats(NULL, 0, NULL, 0, c);
- if (c->stats.buffer == NULL) {
+ if (unlikely(c->stats.buffer == NULL)) {
out_string(c, "SERVER_ERROR out of memory writing stats");
} else {
write_and_free(c, c->stats.buffer, c->stats.offset);
@@ -2541,7 +2541,7 @@ static inline void process_get_command(conn *c, token_t *tokens, size_t ntokens,
key = key_token->value;
nkey = key_token->length;
- if(nkey > KEY_MAX_LENGTH) {
+ if (unlikely(nkey > KEY_MAX_LENGTH)) {
out_string(c, "CLIENT_ERROR bad command line format");
return;
}
@@ -2588,7 +2588,7 @@ static inline void process_get_command(conn *c, token_t *tokens, size_t ntokens,
}
suffix = cache_alloc(c->thread->suffix_cache);
- if (suffix == NULL) {
+ if (unlikely(suffix == NULL)) {
out_string(c, "SERVER_ERROR out of memory making CAS suffix");
item_remove(it);
return;
@@ -2621,7 +2621,7 @@ static inline void process_get_command(conn *c, token_t *tokens, size_t ntokens,
}
- if (settings.verbose > 1)
+ if (unlikely(settings.verbose > 1))
fprintf(stderr, ">%d sending key %s\n", c->sfd, ITEM_key(it));
/* item_get() has incremented it->refcount for us */
@@ -2662,7 +2662,7 @@ static inline void process_get_command(conn *c, token_t *tokens, size_t ntokens,
c->suffixleft = i;
}
- if (settings.verbose > 1)
+ if (unlikely(settings.verbose > 1))
fprintf(stderr, ">%d END\n", c->sfd);
/*
@@ -2696,7 +2696,7 @@ static void process_update_command(conn *c, token_t *tokens, const size_t ntoken
set_noreply_maybe(c, tokens, ntokens);
- if (tokens[KEY_TOKEN].length > KEY_MAX_LENGTH) {
+ if (unlikely(tokens[KEY_TOKEN].length > KEY_MAX_LENGTH)) {
out_string(c, "CLIENT_ERROR bad command line format");
return;
}
@@ -2723,7 +2723,7 @@ static void process_update_command(conn *c, token_t *tokens, const size_t ntoken
}
vlen += 2;
- if (vlen < 0 || vlen - 2 < 0) {
+ if (unlikely(vlen < 0 || vlen - 2 < 0)) {
out_string(c, "CLIENT_ERROR bad command line format");
return;
}
@@ -2775,7 +2775,7 @@ static void process_arithmetic_command(conn *c, token_t *tokens, const size_t nt
set_noreply_maybe(c, tokens, ntokens);
- if (tokens[KEY_TOKEN].length > KEY_MAX_LENGTH) {
+ if (unlikely(tokens[KEY_TOKEN].length > KEY_MAX_LENGTH)) {
out_string(c, "CLIENT_ERROR bad command line format");
return;
}
@@ -2783,7 +2783,7 @@ static void process_arithmetic_command(conn *c, token_t *tokens, const size_t nt
key = tokens[KEY_TOKEN].value;
nkey = tokens[KEY_TOKEN].length;
- if (!safe_strtoull(tokens[2].value, &delta)) {
+ if (unlikely(!safe_strtoull(tokens[2].value, &delta))) {
out_string(c, "CLIENT_ERROR invalid numeric delta argument");
return;
}
@@ -2895,7 +2895,7 @@ static void process_delete_command(conn *c, token_t *tokens, const size_t ntoken
bool sets_noreply = set_noreply_maybe(c, tokens, ntokens);
bool valid = (ntokens == 4 && (hold_is_zero || sets_noreply))
|| (ntokens == 5 && hold_is_zero && sets_noreply);
- if (!valid) {
+ if (unlikely(!valid)) {
out_string(c, "CLIENT_ERROR bad command line format. "
"Usage: delete <key> [noreply]");
return;
@@ -2906,7 +2906,7 @@ static void process_delete_command(conn *c, token_t *tokens, const size_t ntoken
key = tokens[KEY_TOKEN].value;
nkey = tokens[KEY_TOKEN].length;
- if(nkey > KEY_MAX_LENGTH) {
+ if (unlikely(nkey > KEY_MAX_LENGTH)) {
out_string(c, "CLIENT_ERROR bad command line format");
return;
}
@@ -2958,7 +2958,7 @@ static void process_command(conn *c, char *command) {
MEMCACHED_PROCESS_COMMAND_START(c->sfd, c->rcurr, c->rbytes);
- if (settings.verbose > 1)
+ if (unlikely(settings.verbose > 1))
fprintf(stderr, "<%d %s\n", c->sfd, command);
/*
@@ -2969,7 +2969,7 @@ static void process_command(conn *c, char *command) {
c->msgcurr = 0;
c->msgused = 0;
c->iovused = 0;
- if (add_msghdr(c) != 0) {
+ if (unlikely(add_msghdr(c) != 0)) {
out_string(c, "SERVER_ERROR out of memory preparing response");
return;
}
@@ -3032,7 +3032,7 @@ static void process_command(conn *c, char *command) {
}
exptime = strtol(tokens[1].value, NULL, 10);
- if(errno == ERANGE) {
+ if (unlikely(errno == ERANGE)) {
out_string(c, "CLIENT_ERROR bad command line format");
return;
}
@@ -3082,7 +3082,7 @@ static int try_read_command(conn *c) {
c->protocol = ascii_prot;
}
- if (settings.verbose > 1) {
+ if (unlikely(settings.verbose > 1)) {
fprintf(stderr, "%d: Client using the %s protocol\n", c->sfd,
prot_text(c->protocol));
}
@@ -3099,7 +3099,7 @@ static int try_read_command(conn *c) {
/* must realign input buffer */
memmove(c->rbuf, c->rcurr, c->rbytes);
c->rcurr = c->rbuf;
- if (settings.verbose > 1) {
+ if (unlikely(settings.verbose > 1)) {
fprintf(stderr, "%d: Realign input buffer\n", c->sfd);
}
}
@@ -3107,7 +3107,7 @@ static int try_read_command(conn *c) {
protocol_binary_request_header* req;
req = (protocol_binary_request_header*)c->rcurr;
- if (settings.verbose > 1) {
+ if (unlikely(settings.verbose > 1)) {
/* Dump the packet before we convert it to host order */
int ii;
fprintf(stderr, "<%d Read binary protocol data:", c->sfd);
@@ -3126,7 +3126,7 @@ static int try_read_command(conn *c) {
c->binary_header.request.cas = ntohll(req->request.cas);
if (c->binary_header.request.magic != PROTOCOL_BINARY_REQ) {
- if (settings.verbose) {
+ if (unlikely(settings.verbose)) {
fprintf(stderr, "Invalid magic: %x\n",
c->binary_header.request.magic);
}
@@ -3137,7 +3137,7 @@ static int try_read_command(conn *c) {
c->msgcurr = 0;
c->msgused = 0;
c->iovused = 0;
- if (add_msghdr(c) != 0) {
+ if (unlikely(add_msghdr(c) != 0)) {
out_string(c, "SERVER_ERROR out of memory");
return 0;
}
@@ -3221,7 +3221,7 @@ static enum try_read_result try_read_udp(conn *c) {
c->request_id = buf[0] * 256 + buf[1];
/* If this is a multi-packet request, drop it. */
- if (buf[4] != 0 || buf[5] != 1) {
+ if (unlikely(buf[4] != 0 || buf[5] != 1)) {
out_string(c, "SERVER_ERROR multi-packet request not supported");
return READ_NO_DATA_RECEIVED;
}
@@ -3268,8 +3268,8 @@ static enum try_read_result try_read_network(conn *c) {
}
++num_allocs;
char *new_rbuf = realloc(c->rbuf, c->rsize * 2);
- if (!new_rbuf) {
- if (settings.verbose > 0)
+ if (unlikely(!new_rbuf)) {
+ if (unlikely(settings.verbose > 0))
fprintf(stderr, "Couldn't realloc input buffer\n");
c->rbytes = 0; /* ignore what we read */
out_string(c, "SERVER_ERROR out of memory reading request");
@@ -3399,7 +3399,7 @@ static enum transmit_result transmit(conn *c) {
}
if (res == -1 && (errno == EAGAIN || errno == EWOULDBLOCK)) {
if (!update_event(c, EV_WRITE | EV_PERSIST)) {
- if (settings.verbose > 0)
+ if (unlikely(settings.verbose > 0))
fprintf(stderr, "Couldn't update event\n");
conn_set_state(c, conn_closing);
return TRANSMIT_HARD_ERROR;
@@ -3408,7 +3408,7 @@ static enum transmit_result transmit(conn *c) {
}
/* if res == 0 or res == -1 and error is not EAGAIN or EWOULDBLOCK,
we have a real error, on which we close the connection */
- if (settings.verbose > 0)
+ if (unlikely(settings.verbose > 0))
perror("Failed to write, and not due to blocking");
if (IS_UDP(c->transport))
@@ -3441,7 +3441,7 @@ static void drive_machine(conn *c) {
/* these are transient, so don't log anything */
stop = true;
} else if (errno == EMFILE) {
- if (settings.verbose > 0)
+ if (unlikely(settings.verbose > 0))
fprintf(stderr, "Too many open connections\n");
accept_new_conns(false);
stop = true;
@@ -3465,7 +3465,7 @@ static void drive_machine(conn *c) {
case conn_waiting:
if (!update_event(c, EV_READ | EV_PERSIST)) {
- if (settings.verbose > 0)
+ if (unlikely(settings.verbose > 0))
fprintf(stderr, "Couldn't update event\n");
conn_set_state(c, conn_closing);
break;
@@ -3521,7 +3521,7 @@ static void drive_machine(conn *c) {
because that should be possible ;-)
*/
if (!update_event(c, EV_WRITE | EV_PERSIST)) {
- if (settings.verbose > 0)
+ if (unlikely(settings.verbose > 0))
fprintf(stderr, "Couldn't update event\n");
conn_set_state(c, conn_closing);
}
@@ -3569,7 +3569,7 @@ static void drive_machine(conn *c) {
}
if (res == -1 && (errno == EAGAIN || errno == EWOULDBLOCK)) {
if (!update_event(c, EV_READ | EV_PERSIST)) {
- if (settings.verbose > 0)
+ if (unlikely(settings.verbose > 0))
fprintf(stderr, "Couldn't update event\n");
conn_set_state(c, conn_closing);
break;
@@ -3578,7 +3578,7 @@ static void drive_machine(conn *c) {
break;
}
/* otherwise we have a real error, on which we close the connection */
- if (settings.verbose > 0) {
+ if (unlikely(settings.verbose > 0)) {
fprintf(stderr, "Failed to read, and not due to blocking:\n"
"errno: %d %s \n"
"rcurr=%lx ritem=%lx rbuf=%lx rlbytes=%d rsize=%d\n",
@@ -3620,7 +3620,7 @@ static void drive_machine(conn *c) {
}
if (res == -1 && (errno == EAGAIN || errno == EWOULDBLOCK)) {
if (!update_event(c, EV_READ | EV_PERSIST)) {
- if (settings.verbose > 0)
+ if (unlikely(settings.verbose > 0))
fprintf(stderr, "Couldn't update event\n");
conn_set_state(c, conn_closing);
break;
@@ -3629,7 +3629,7 @@ static void drive_machine(conn *c) {
break;
}
/* otherwise we have a real error, on which we close the connection */
- if (settings.verbose > 0)
+ if (unlikely(settings.verbose > 0))
fprintf(stderr, "Failed to read, and not due to blocking\n");
conn_set_state(c, conn_closing);
break;
@@ -3642,7 +3642,7 @@ static void drive_machine(conn *c) {
*/
if (c->iovused == 0 || (IS_UDP(c->transport) && c->iovused == 1)) {
if (add_iov(c, c->wcurr, c->wbytes) != 0) {
- if (settings.verbose > 0)
+ if (unlikely(settings.verbose > 0))
fprintf(stderr, "Couldn't build response\n");
conn_set_state(c, conn_closing);
break;
@@ -3653,7 +3653,7 @@ static void drive_machine(conn *c) {
case conn_mwrite:
if (IS_UDP(c->transport) && c->msgcurr == 0 && build_udp_headers(c) != 0) {
- if (settings.verbose > 0)
+ if (unlikely(settings.verbose > 0))
fprintf(stderr, "Failed to build UDP headers\n");
conn_set_state(c, conn_closing);
break;
@@ -3687,7 +3687,7 @@ static void drive_machine(conn *c) {
}
conn_set_state(c, c->write_and_go);
} else {
- if (settings.verbose > 0)
+ if (unlikely(settings.verbose > 0))
fprintf(stderr, "Unexpected state %d\n", c->state);
conn_set_state(c, conn_closing);
}
@@ -3730,7 +3730,7 @@ void event_handler(const int fd, const short which, void *arg) {
/* sanity */
if (fd != c->sfd) {
- if (settings.verbose > 0)
+ if (unlikely(settings.verbose > 0))
fprintf(stderr, "Catastrophic: event fd doesn't match conn fd!\n");
conn_close(c);
return;
@@ -3771,7 +3771,7 @@ static void maximize_sndbuf(const int sfd) {
/* Start with the default size. */
if (getsockopt(sfd, SOL_SOCKET, SO_SNDBUF, &old_size, &intsize) != 0) {
- if (settings.verbose > 0)
+ if (unlikely(settings.verbose > 0))
perror("getsockopt(SO_SNDBUF)");
return;
}
@@ -3790,7 +3790,7 @@ static void maximize_sndbuf(const int sfd) {
}
}
- if (settings.verbose > 1)
+ if (unlikely(settings.verbose > 1))
fprintf(stderr, "<%d send buffer was %d, now %d\n", sfd, old_size, last_good);
}
View
65 memcached.h
@@ -229,12 +229,7 @@ struct thread_stats {
*/
struct stats {
pthread_mutex_t mutex;
- unsigned int curr_items;
- unsigned int total_items;
- uint64_t curr_bytes;
- unsigned int curr_conns;
- unsigned int total_conns;
- unsigned int conn_structs;
+ uint64_t listen_disabled_num;
uint64_t get_cmds;
uint64_t set_cmds;
uint64_t get_hits;
@@ -242,8 +237,13 @@ struct stats {
uint64_t evictions;
uint64_t reclaimed;
time_t started; /* when the process was started */
+ uint64_t curr_bytes;
+ unsigned int curr_items;
+ unsigned int total_items;
+ unsigned int curr_conns;
+ unsigned int total_conns;
+ unsigned int conn_structs;
bool accepting_conns; /* whether we are currently accepting */
- uint64_t listen_disabled_num;
};
#define MAX_VERBOSITY_LEVEL 2
@@ -253,28 +253,28 @@ struct stats {
* Globally accessible settings as derived from the commandline.
*/
struct settings {
+ char *inter;
size_t maxbytes;
int maxconns;
int port;
int udpport;
- char *inter;
- int verbose;
+ unsigned int verbose;
rel_time_t oldest_live; /* ignore existing items older than this */
int evict_to_free;
- char *socketpath; /* path to unix socket if using local socket */
- int access; /* access mask (a la chmod) for unix domain socket */
- double factor; /* chunk size growth factor */
+ char *socketpath; /* path to unix socket if using local socket */
+ int access; /* access mask (a la chmod) for unix domain socket */
int chunk_size;
+ double factor; /* chunk size growth factor */
int num_threads; /* number of worker (without dispatcher) libevent threads to run */
- char prefix_delimiter; /* character that marks a key prefix (for stats) */
int detail_enabled; /* nonzero if we're collecting detailed stats */
int reqs_per_event; /* Maximum number of io to process on each
io-event. */
- bool use_cas;
enum protocol binding_protocol;
int backlog;
- int item_size_max; /* Maximum item size, and upper end for slabs */
+ int item_size_max; /* Maximum item size, and upper end for slabs */
bool sasl; /* SASL on/off */
+ bool use_cas;
+ char prefix_delimiter; /* character that marks a key prefix (for stats) */
};
extern struct stats stats;
@@ -302,6 +302,7 @@ typedef struct _stritem {
uint8_t it_flags; /* ITEM_* above */
uint8_t slabs_clsid;/* which slab class we're in */
uint8_t nkey; /* key length, w/terminating null and padding */
+ /* NOTE: there is wasted space here due to alignment issues... 2 bytes on 32 bit, 6 bytes on 64 bit */
void * end[];
/* if it_flags & ITEM_CAS we have 8 bytes CAS */
/* then null-terminated key */
@@ -330,13 +331,7 @@ typedef struct {
*/
typedef struct conn conn;
struct conn {
- int sfd;
sasl_conn_t *sasl_conn;
- enum conn_states state;
- enum bin_substates substate;
- struct event event;
- short ev_flags;
- short which; /** which events were just triggered */
char *rbuf; /** buffer to read commands into */
char *rcurr; /** but if we parsed some already, this is where we stopped */
@@ -347,6 +342,16 @@ struct conn {
char *wcurr;
int wsize;
int wbytes;
+
+ enum conn_states state;
+ enum bin_substates substate;
+ struct event event;
+
+ int sfd;
+ short ev_flags;
+ short which; /** which events were just triggered */
+
+
/** which state to go into after finishing current write */
enum conn_states write_and_go;
void *write_and_free; /** free this memory after finishing writing */
@@ -354,6 +359,11 @@ struct conn {
char *ritem; /** when we read in an item's value, it goes here */
int rlbytes;
+
+
+ /* data for the swallow state */
+ int sbytes; /* how many bytes to swallow */
+
/* data for the nread state */
/**
@@ -364,9 +374,6 @@ struct conn {
void *item; /* for commands set/add/replace */
- /* data for the swallow state */
- int sbytes; /* how many bytes to swallow */
-
/* data for the mwrite state */
struct iovec *iov;
int iovsize; /* number of elements allocated in iov[] */
@@ -379,13 +386,13 @@ struct conn {
int msgbytes; /* number of bytes in current msg */
item **ilist; /* list of items to write out */
- int isize;
item **icurr;
+ int isize;
int ileft;
char **suffixlist;
- int suffixsize;
char **suffixcurr;
+ int suffixsize;
int suffixleft;
enum protocol protocol; /* which protocol this connection speaks */
@@ -398,7 +405,6 @@ struct conn {
unsigned char *hdrbuf; /* udp packet headers */
int hdrsize; /* number of headers' worth of space is allocated */
- bool noreply; /* True if the reply should not be sent. */
/* current stats command */
struct {
char *buffer;
@@ -410,11 +416,14 @@ struct conn {
/* This is where the binary header goes */
protocol_binary_request_header binary_header;
uint64_t cas; /* the cas to return */
- short cmd; /* current command being processed */
int opaque;
int keylen;
+
conn *next; /* Used for generating a list of conn structures */
LIBEVENT_THREAD *thread; /* Pointer to the thread object serving this connection */
+
+ short cmd; /* current command being processed */
+ bool noreply; /* True if the reply should not be sent. */
};
View
2  sasl_defs.c
@@ -168,7 +168,7 @@ void init_sasl(void) {
fprintf(stderr, "Error initializing sasl.\n");
exit(EXIT_FAILURE);
} else {
- if (settings.verbose) {
+ if (unlikely(settings.verbose)) {
fprintf(stderr, "Initialized SASL.\n");
}
}
View
4 slabs.c
@@ -124,7 +124,7 @@ void slabs_init(const size_t limit, const double factor, const bool prealloc) {
slabclass[i].size = size;
slabclass[i].perslab = settings.item_size_max / slabclass[i].size;
size *= factor;
- if (settings.verbose > 1) {
+ if (unlikely(settings.verbose > 1)) {
fprintf(stderr, "slab class %3d: chunk size %9u perslab %7u\n",
i, slabclass[i].size, slabclass[i].perslab);
}
@@ -133,7 +133,7 @@ void slabs_init(const size_t limit, const double factor, const bool prealloc) {
power_largest = i;
slabclass[power_largest].size = settings.item_size_max;
slabclass[power_largest].perslab = 1;
- if (settings.verbose > 1) {
+ if (unlikely(settings.verbose > 1)) {
fprintf(stderr, "slab class %3d: chunk size %9u perslab %7u\n",
i, slabclass[i].size, slabclass[i].perslab);
}
View
4 thread.c
@@ -257,7 +257,7 @@ static void thread_libevent_process(int fd, short which, void *arg) {
char buf[1];
if (read(fd, buf, 1) != 1)
- if (settings.verbose > 0)
+ if (unlikely(settings.verbose))
fprintf(stderr, "Can't read from libevent pipe\n");
item = cq_pop(me->new_conn_queue);
@@ -270,7 +270,7 @@ static void thread_libevent_process(int fd, short which, void *arg) {
fprintf(stderr, "Can't listen for events on UDP socket\n");
exit(1);
} else {
- if (settings.verbose > 0) {
+ if (unlikely(settings.verbose)) {
fprintf(stderr, "Can't listen for events on fd %d\n",
item->sfd);
}
Something went wrong with that request. Please try again.