Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

Added list mode (lpush)

  • Loading branch information...
commit 25a5a0dd57f8b5b32ebe308c3743adf5f955ff65 1 parent 5bed583
@markphelps markphelps authored
Showing with 26 additions and 3 deletions.
  1. +3 −1 README.md
  2. +23 −2 src/com/hackdiary/pig/RedisStorer.java
View
4 README.md
@@ -19,10 +19,12 @@ Use:
## Bulkloading strategy
-RedisStorer runs in three modes: kv, set and hash (specified as the first argument to RedisStorer). If no mode is specified, kv is the default.
+RedisStorer runs in four modes: kv, set, hash and list (specified as the first argument to RedisStorer). If no mode is specified, kv is the default.
In kv mode, it takes the first field of the stored tuple as the key, and the second field as the value, and issues [SET key value](http://redis.io/commands/set). Any further fields are ignored.
In set mode, it takes the first field of the stored tuple as the key, and issues [SADD key value](http://redis.io/commands/sadd) once for each subsequent field value in the tuple.
In hash mode, it takes the first field of the stored tuple as the key, and issues [HSET key fieldname value](http://redis.io/commands/hset) once for each subsequent field value, using the same key for each, and taking the fieldname from the tuple's schema fieldnames. This means that it will fail unless the stored tuple has a schema with named fields.
+
+In list mode, it takes the first field of the stored tuple as the key, and issues [LPUSH key value](http://redis.io/commands/lpush) once for each subsequent field value in the tuple.
View
25 src/com/hackdiary/pig/RedisStorer.java
@@ -54,7 +54,7 @@ public void putNext(Tuple f) throws IOException {
_jedis.set(key,values.get(1).toString());
}
}
- if(_mode.equals("set")) {
+ else if(_mode.equals("set")) {
int idx = 0;
Pipeline p = _jedis.pipelined();
for(Object o : values) {
@@ -75,7 +75,7 @@ public void putNext(Tuple f) throws IOException {
}
p.execute();
}
- if(_mode.equals("hash")) {
+ else if(_mode.equals("hash")) {
UDFContext context = UDFContext.getUDFContext();
Properties property = context.getUDFProperties(ResourceSchema.class);
String fieldNames = property.getProperty("redis.field.names");
@@ -91,6 +91,27 @@ public void putNext(Tuple f) throws IOException {
}
p.execute();
}
+ else if(_mode.equals("list")) {
+ int idx = 0;
+ Pipeline p = _jedis.pipelined();
+ for(Object o : values) {
+ if(idx != 0 && o != null) {
+ switch (DataType.findType(o)) {
+ case DataType.TUPLE:
+ case DataType.BAG:
+ for(Object o2 : (Iterable)o) {
+ p.lpush(key, o2.toString());
+ }
+ break;
+ default:
+ p.lpush(key, o.toString());
+ break;
+ }
+ }
+ idx++;
+ }
+ p.execute();
+ }
}
@Override
Please sign in to comment.
Something went wrong with that request. Please try again.