Skip to content
This repository
Browse code

enhanced trident write so it could emit and carry tuple forward, upda…

…ted readme.md
  • Loading branch information...
commit 84ed57a9e554f9dea92764ed5dfda7ff778b8c00 1 parent 4ebcb50
Alex Collautt authored
23 README.md
Source Rendered
@@ -25,12 +25,17 @@ Maven artifacts for releases will be available on maven central.
25 25
26 26 **Basic Usage**
27 27
28   -`CassandraBolt` expects that a Cassandra hostname, port, and keyspace be set in the Storm topology configuration:
29   -
  28 +`CassandraBolt`, `TridentCassandraLookupFunction`, and `TridentCassandraWriteFunction` expects that a Cassandra hostname,
  29 +port, and keyspace be set in the Storm topology configuration. To allow for multiple instances of these in a topology
  30 +and not require that they all connect to the same Cassandra instance the values are added to the Storm configuration
  31 +using a key and a map. The key to indicate which map to use is set in the constructor of these classes when instantiating
  32 +them.
  33 +
  34 + Map<String, Object> cassandraConfig = new HashMap<String, Object>();
  35 + cassandraConfig.put(CassandraBolt.CASSANDRA_HOST, "localhost:9160");
  36 + cassandraConfig.put(CassandraBolt.CASSANDRA_KEYSPACE, "testKeyspace");
30 37 Config config = new Config();
31   - config.put(CassandraBolt.CASSANDRA_HOST, "localhost:9160");
32   - config.put(CassandraBolt.CASSANDRA_KEYSPACE, "testKeyspace");
33   -
  38 + config.put("CassandraLocal", cassandraConfig);
34 39
35 40 The `CassandraBolt` class provides a convenience constructor that takes a column family name, and row key field value as arguments:
36 41
@@ -51,6 +56,14 @@ Would yield the following Cassandra row (as seen from `cassandra-cli`):
51 56 => (column=field1, value=foo, timestamp=1321938505071001)
52 57 => (column=field2, value=bar, timestamp=1321938505072000)
53 58
  59 +**Cassandra Write Function**
  60 +Storm Trident filters out the original Tuple if a function doesn't emit anything. To allow for additional processing after
  61 +writing to Cassandra the `TridentCassandraWriteFunction` can emit a static Object value. The main purpose for this emit is
  62 +to simply allow the Tuple to continue as opposed to filtering it out. The static value can be set in either the constructor
  63 +or by calling the setValueToEmitAfterWrite method. Setting the emit value to NULL will cause the function to not emit anything
  64 +and Storm will filter the Tuple out. Default behavior is to not emit.
  65 +If the function will emit a value don't forget to declare the output field when building the topology.
  66 +
54 67 **Cassandra Counter Columns**
55 68
56 69 The Counter Column concept is similar to the above,
15 src/main/java/com/hmsonline/storm/cassandra/trident/TridentCassandraWriteFunction.java
@@ -10,6 +10,7 @@
10 10 import storm.trident.operation.TridentOperationContext;
11 11 import storm.trident.tuple.TridentTuple;
12 12 import backtype.storm.topology.FailedException;
  13 +import backtype.storm.tuple.Values;
13 14
14 15 import com.hmsonline.storm.cassandra.bolt.mapper.TridentTupleMapper;
15 16 import com.hmsonline.storm.cassandra.client.AstyanaxClient;
@@ -25,6 +26,11 @@
25 26 private Class<K> columnNameClass;
26 27 private Class<V> columnValueClass;
27 28 private String clientConfigKey;
  29 + private Object valueToEmit;
  30 +
  31 + public void setValueToEmitAfterWrite(Object valueToEmit) {
  32 + this.valueToEmit = valueToEmit;
  33 + }
28 34
29 35 public TridentCassandraWriteFunction(String clientConfigKey, TridentTupleMapper<K, V> tupleMapper,
30 36 Class<K> columnNameClass, Class<V> columnValueClass) {
@@ -32,6 +38,12 @@ public TridentCassandraWriteFunction(String clientConfigKey, TridentTupleMapper<
32 38 this.columnNameClass = columnNameClass;
33 39 this.columnValueClass = columnValueClass;
34 40 this.clientConfigKey = clientConfigKey;
  41 + this.valueToEmit = null;
  42 + }
  43 + public TridentCassandraWriteFunction(String clientConfigKey, TridentTupleMapper<K, V> tupleMapper,
  44 + Class<K> columnNameClass, Class<V> columnValueClass, Object valueToEmit) {
  45 + this(clientConfigKey, tupleMapper, columnNameClass, columnValueClass);
  46 + this.valueToEmit = valueToEmit;
35 47 }
36 48
37 49 @Override
@@ -51,6 +63,9 @@ public void cleanup() {
51 63 public void execute(TridentTuple tuple, TridentCollector collector) {
52 64 try {
53 65 writeTuple(tuple);
  66 + if (this.valueToEmit != null) {
  67 + collector.emit(new Values(this.valueToEmit));
  68 + }
54 69 } catch (TupleMappingException e) {
55 70 LOG.error("Skipping tuple: " + tuple, e);
56 71 } catch (StormCassandraException e) {

0 comments on commit 84ed57a

Please sign in to comment.
Something went wrong with that request. Please try again.