Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Handle integer replies, implement a connect timeout and dynamic config in VCL #2

Merged
merged 6 commits into from

2 participants

@noahwilliamsson

Hi,

Please consider pulling the following patches (details available in the commit messages):
Noah Williamsson (6):
Handle REDIS_REPLY_INTEGER in vmod_call() and return it as a string
Initialize config (host, port) once per VCL instead of once every call
Implement a connect timeout for connecting to Redis, defaulting to 200ms
Allow the Redis server and timeout to be set in VCL via redis.init_redis()
Provide a commented out example of redis.init_redis() in examples/example.vcl
Fix typo in examples/example.vcl (redis.call -> redis.send)

Tested on 32- and 64-bit Linux aswell as Mac OS X with Varnish 3.0.2.

noahwilliamsson added some commits
@noahwilliamsson noahwilliamsson Handle REDIS_REPLY_INTEGER in vmod_call() and return it as a string
Signed-off-by: Noah Williamsson <noah.williamsson@gmail.com>
3300bff
@noahwilliamsson noahwilliamsson Initialize config (host, port) once per VCL instead of once every call
Because the VCC specifies PRIV_CALL for vmod_send() and vmod_call(), the
vmod_priv argument is NULL for every call to these functions. This causes
the config struct to be re-allocated for every call.

This patch changes PRIV_CALL to PRIV_VCL, making the vmod_priv argument shared
between all calls to the module's functions within a given instance of a VCL.

The allocation of the config struct is also moved from redis_common() to
init_function().

Signed-off-by: Noah Williamsson <noah.williamsson@gmail.com>
f2c2ff0
@noahwilliamsson noahwilliamsson Implement a connect timeout for connecting to Redis, defaulting to 200ms
Use redisConnectWithTimeout() instead of redisConnect() to prevent
hanging for minutes when the host is down (i.e, not responding).

The default timeout is set to 200ms via REDIS_TIMEOUT_MS in vmod_redis.c.

Signed-off-by: Noah Williamsson <noah.williamsson@gmail.com>
caa623e
@noahwilliamsson noahwilliamsson Allow the Redis server and timeout to be set in VCL via redis.init_re…
…dis()

By default libvmod-redis attempts to connect to the Redis server
at 127.0.0.1, port 6379 with a connect timeout of 200 ms.

This patch allows the Redis hostname, port and timeout to be configured
via VCL using redis.init_redis(hostname, port, timeout_ms);

VCL example:
  import redis;
  sub vcl_init {
    redis.init_redis("localhost", 6379, 200);
  }

Signed-off-by: Noah Williamsson <noah.williamsson@gmail.com>
3b34657
@noahwilliamsson noahwilliamsson Provide a commented out example of redis.init_redis() in examples/exa…
…mple.vcl

Signed-off-by: Noah Williamsson <noah.williamsson@gmail.com>
68ce997
@noahwilliamsson noahwilliamsson Fix typo in examples/example.vcl (redis.call -> redis.send)
Signed-off-by: Noah Williamsson <noah.williamsson@gmail.com>
01d14b8
@andreacampi
Owner

Awesome, thanks!

@andreacampi andreacampi merged commit 4397675 into from
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on May 31, 2012
  1. @noahwilliamsson

    Handle REDIS_REPLY_INTEGER in vmod_call() and return it as a string

    noahwilliamsson authored
    Signed-off-by: Noah Williamsson <noah.williamsson@gmail.com>
  2. @noahwilliamsson

    Initialize config (host, port) once per VCL instead of once every call

    noahwilliamsson authored
    Because the VCC specifies PRIV_CALL for vmod_send() and vmod_call(), the
    vmod_priv argument is NULL for every call to these functions. This causes
    the config struct to be re-allocated for every call.
    
    This patch changes PRIV_CALL to PRIV_VCL, making the vmod_priv argument shared
    between all calls to the module's functions within a given instance of a VCL.
    
    The allocation of the config struct is also moved from redis_common() to
    init_function().
    
    Signed-off-by: Noah Williamsson <noah.williamsson@gmail.com>
  3. @noahwilliamsson

    Implement a connect timeout for connecting to Redis, defaulting to 200ms

    noahwilliamsson authored
    Use redisConnectWithTimeout() instead of redisConnect() to prevent
    hanging for minutes when the host is down (i.e, not responding).
    
    The default timeout is set to 200ms via REDIS_TIMEOUT_MS in vmod_redis.c.
    
    Signed-off-by: Noah Williamsson <noah.williamsson@gmail.com>
  4. @noahwilliamsson

    Allow the Redis server and timeout to be set in VCL via redis.init_re…

    noahwilliamsson authored
    …dis()
    
    By default libvmod-redis attempts to connect to the Redis server
    at 127.0.0.1, port 6379 with a connect timeout of 200 ms.
    
    This patch allows the Redis hostname, port and timeout to be configured
    via VCL using redis.init_redis(hostname, port, timeout_ms);
    
    VCL example:
      import redis;
      sub vcl_init {
        redis.init_redis("localhost", 6379, 200);
      }
    
    Signed-off-by: Noah Williamsson <noah.williamsson@gmail.com>
  5. @noahwilliamsson

    Provide a commented out example of redis.init_redis() in examples/exa…

    noahwilliamsson authored
    …mple.vcl
    
    Signed-off-by: Noah Williamsson <noah.williamsson@gmail.com>
  6. @noahwilliamsson

    Fix typo in examples/example.vcl (redis.call -> redis.send)

    noahwilliamsson authored
    Signed-off-by: Noah Williamsson <noah.williamsson@gmail.com>
This page is out of date. Refresh to see the latest.
View
11 README.md
@@ -14,6 +14,17 @@ So far the module builds and runs on FreeBSD--on other platforms, you are on you
Functions and procedures
------------------------
+*redis.init_redis(host, port, timeout_ms)*
+
+Use the redis server at the given _host_ and _port_ with a timeout
+of _timeout__ms_ milliseconds.
+If _port_ is less than or equal to zero, the default port of 6379 is used.
+If _timeout__ms_ is less than or equal to zero, a default timeout of 200ms is used.
+
+This function is supposed to be called from the Varnish subroutine _vcl__init_.
+If the call is left out, the module will attempt to connect to the Redis server
+at 127.0.0.1:6379 with a connect timeout of 200ms.
+
*redis.send(command)*
Sends the given _command_ to redis; the response will be ignored.
View
13 examples/example.vcl
@@ -9,9 +9,20 @@ backend be1 {
.port = "80";
}
+#sub vcl_init {
+ #
+ # By default, the redis module will attempt to connect to a Redis server
+ # at 127.0.0.1:6379 with a connect timeout of 200 milliseconds.
+ #
+ # The function redis.init_redis(host, port, timeout_ms) may be used to
+ # connect to an alternate Redis server or use a different connect timeout.
+ #
+ # redis.init_redis("localhost", 6379, 200); /* default values */
+#}
+
sub vcl_recv {
#
- # redis.call is a procedure, it will send the command to redis and ignore
+ # redis.send is a procedure, it will send the command to redis and ignore
# the response. If the command errors out, it will be logged but the VCL
# will not know.
#
View
70 src/vmod_redis.c
@@ -1,5 +1,6 @@
#include <stdlib.h>
#include <stdio.h>
+#include <sys/time.h>
#include "vrt.h"
#include "bin/varnishd/cache.h"
@@ -10,6 +11,9 @@
#include <hiredis/hiredis.h>
+#define REDIS_TIMEOUT_MS 200 /* 200 milliseconds */
+
+
#define LOG_E(...) fprintf(stderr, __VA_ARGS__);
#ifdef DEBUG
# define LOG_T(...) fprintf(stderr, __VA_ARGS__);
@@ -20,6 +24,7 @@
typedef struct redisConfig {
char *host;
int port;
+ struct timeval timeout;
} config_t;
static pthread_key_t redis_key;
@@ -44,36 +49,73 @@ make_key()
{
(void)pthread_key_create(&redis_key, NULL);
}
+
+static config_t *
+make_config(const char *host, int port, int timeout_ms)
+{
+ config_t *cfg;
+
+ LOG_T("make_config(%s,%d,%d)\n", host, port, timeout_ms);
+
+ cfg = malloc(sizeof(config_t));
+ if(cfg == NULL)
+ return NULL;
+
+ if(port <= 0)
+ port = 6379;
+
+ if(timeout_ms <= 0)
+ timeout_ms = REDIS_TIMEOUT_MS;
+
+ cfg->host = strdup(host);
+ cfg->port = port;
+
+ cfg->timeout.tv_sec = timeout_ms / 1000;
+ cfg->timeout.tv_usec = (timeout_ms % 1000) * 1000;
+
+ return cfg;
+}
int
init_function(struct vmod_priv *priv, const struct VCL_conf *conf)
{
+ config_t *cfg;
+
LOG_T("redis init called\n");
(void)pthread_once(&redis_key_once, make_key);
+ if (priv->priv == NULL) {
+ priv->priv = make_config("127.0.0.1", 6379, REDIS_TIMEOUT_MS);
+ priv->free = free;
+ }
+
return (0);
}
+void
+vmod_init_redis(struct sess *sp, struct vmod_priv *priv, const char *host, int port, int timeout_ms)
+{
+ config_t *old_cfg = priv->priv;
+
+ priv->priv = make_config(host, port, timeout_ms);
+ if(priv->priv && old_cfg) {
+ free(old_cfg->host);
+ free(old_cfg);
+ }
+}
+
static redisReply *
redis_common(struct sess *sp, struct vmod_priv *priv, const char *command)
{
- config_t *cfg;
+ config_t *cfg = priv->priv;
redisContext *c;
redisReply *reply = NULL;
LOG_T("redis(%x): running %s %p\n", pthread_self(), command, priv->priv);
- cfg = priv->priv;
- if (cfg == NULL) {
- priv->priv = cfg = malloc(sizeof(config_t));
- priv->free = free;
- cfg->host = strdup("127.0.0.1");
- cfg->port = 6379;
- }
-
if ((c = pthread_getspecific(redis_key)) == NULL) {
- c = redisConnect(cfg->host, cfg->port);
+ c = redisConnectWithTimeout(cfg->host, cfg->port, cfg->timeout);
if (c->err) {
LOG_E("redis error (connect): %s\n", c->errstr);
}
@@ -82,7 +124,7 @@ redis_common(struct sess *sp, struct vmod_priv *priv, const char *command)
reply = redisCommand(c, command);
if (reply == NULL && c->err == REDIS_ERR_EOF) {
- c = redisConnect(cfg->host, cfg->port);
+ c = redisConnectWithTimeout(cfg->host, cfg->port, cfg->timeout);
if (c->err) {
LOG_E("redis error (reconnect): %s\n", c->errstr);
redisFree(c);
@@ -114,6 +156,7 @@ vmod_call(struct sess *sp, struct vmod_priv *priv, const char *command)
{
redisReply *reply = NULL;
const char *ret = NULL;
+ char *digits;
reply = redis_common(sp, priv, command);
if (reply == NULL) {
@@ -128,7 +171,10 @@ vmod_call(struct sess *sp, struct vmod_priv *priv, const char *command)
ret = strdup(reply->str);
break;
case REDIS_REPLY_INTEGER:
- ret = strdup("integer"); /* FIXME */
+ digits = malloc(21); /* sizeof(long long) == 8; 20 digits + NUL */
+ if(digits)
+ sprintf(digits, "%lld", reply->integer);
+ ret = digits;
break;
case REDIS_REPLY_NIL:
ret = NULL;
View
5 src/vmod_redis.vcc
@@ -1,4 +1,5 @@
Module redis
Init init_function
-Function VOID send(PRIV_CALL, STRING)
-Function STRING call(PRIV_CALL, STRING)
+Function VOID init_redis(PRIV_VCL, STRING, INT, INT)
+Function VOID send(PRIV_VCL, STRING)
+Function STRING call(PRIV_VCL, STRING)
Something went wrong with that request. Please try again.