Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

Merge pull request #2 from markphelps/master

Added list mode (thanks to @markphelps)
  • Loading branch information...
commit 9d538ee9304c56da5077943c991665c16280834f 2 parents 5bed583 + 25a5a0d
@mattb authored
Showing with 26 additions and 3 deletions.
  1. +3 −1 README.md
  2. +23 −2 src/com/hackdiary/pig/RedisStorer.java
View
4 README.md
@@ -19,10 +19,12 @@ Use:
## Bulkloading strategy
-RedisStorer runs in three modes: kv, set and hash (specified as the first argument to RedisStorer). If no mode is specified, kv is the default.
+RedisStorer runs in four modes: kv, set, hash and list (specified as the first argument to RedisStorer). If no mode is specified, kv is the default.
In kv mode, it takes the first field of the stored tuple as the key, and the second field as the value, and issues [SET key value](http://redis.io/commands/set). Any further fields are ignored.
In set mode, it takes the first field of the stored tuple as the key, and issues [SADD key value](http://redis.io/commands/sadd) once for each subsequent field value in the tuple.
In hash mode, it takes the first field of the stored tuple as the key, and issues [HSET key fieldname value](http://redis.io/commands/hset) once for each subsequent field value, using the same key for each, and taking the fieldname from the tuple's schema fieldnames. This means that it will fail unless the stored tuple has a schema with named fields.
+
+In list mode, it takes the first field of the stored tuple as the key, and issues [LPUSH key value](http://redis.io/commands/lpush) once for each subsequent field value in the tuple.
View
25 src/com/hackdiary/pig/RedisStorer.java
@@ -54,7 +54,7 @@ public void putNext(Tuple f) throws IOException {
_jedis.set(key,values.get(1).toString());
}
}
- if(_mode.equals("set")) {
+ else if(_mode.equals("set")) {
int idx = 0;
Pipeline p = _jedis.pipelined();
for(Object o : values) {
@@ -75,7 +75,7 @@ public void putNext(Tuple f) throws IOException {
}
p.execute();
}
- if(_mode.equals("hash")) {
+ else if(_mode.equals("hash")) {
UDFContext context = UDFContext.getUDFContext();
Properties property = context.getUDFProperties(ResourceSchema.class);
String fieldNames = property.getProperty("redis.field.names");
@@ -91,6 +91,27 @@ public void putNext(Tuple f) throws IOException {
}
p.execute();
}
+ else if(_mode.equals("list")) {
+ int idx = 0;
+ Pipeline p = _jedis.pipelined();
+ for(Object o : values) {
+ if(idx != 0 && o != null) {
+ switch (DataType.findType(o)) {
+ case DataType.TUPLE:
+ case DataType.BAG:
+ for(Object o2 : (Iterable)o) {
+ p.lpush(key, o2.toString());
+ }
+ break;
+ default:
+ p.lpush(key, o.toString());
+ break;
+ }
+ }
+ idx++;
+ }
+ p.execute();
+ }
}
@Override
Please sign in to comment.
Something went wrong with that request. Please try again.