Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Added RangeGet to query #1

Closed
wants to merge 1 commit into from

2 participants

Geert-Johan Riemer Carlos Carrasco
Geert-Johan Riemer

Hi carloscm,

To be able to read a complete CF, I have forked your project gossie and added RangeGet to the query interface and implementation.
I was hoping you could add this to your branch again, so others can benefit the feature.
It consists of one commit and one file that changed.

Regards,
GeertJohan

Geert-Johan added RangeGet mehod to Query. the method accepts a gossie.Range and …
…a boolean to indicate if the startkey(given in range) should be excluded from the result.
154e107
Carlos Carrasco
Owner

Cassandra provides no such method, and the way you are using RangeGet is the correct one for a RP cluster. The Cassandra Thrift API is not designed for streaming-like methods and doesn't provide them. The usual way to do full-table processing in Cassandra is to setup a Hadoop cluster parallel to your Cassandra cluster. Cassandra provides specific Hadoop APIs for this. That's usually much more efficient than using RangeGet via Thrift. In production I only use RangeGet for very small tables, everything else is indexed in some way and there is no need to scan a full table.

As for your channel proposal I would like to not implement it for now. I usually shy away from channels in Gossie since Go already provides excellent multiplexed goroutine execution based on I/O waits. Gossie just by virtue of using (via the Thrift libs) Go sockets is already concurrent and asynchronous if you launch multiple goroutines to talk to it (it's designed to work like this, including sharing the Pool objects). If an application wants to add another layer on top of it using channels (for idiomatic purposes like you mention, or for doing parallel I/O waits with both Gossie and other sources of I/O at the same time) they can do it very easily in their side.

Nice to see your interest in Gossie, keep up the good work!

Geert-Johan Riemer

I guess this can be closed :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 1 unique commit by 1 author.

Jul 31, 2012
Geert-Johan added RangeGet mehod to Query. the method accepts a gossie.Range and …
…a boolean to indicate if the startkey(given in range) should be excluded from the result.
154e107
This page is out of date. Refresh to see the latest.

Showing 1 changed file with 30 additions and 0 deletions. Show diff stats Hide diff stats

  1. +30 0 src/gossie/query.go
30 src/gossie/query.go
@@ -50,6 +50,9 @@ type Query interface {
50 50
51 51 // MultiGet looks up multiple rows given the keys.
52 52 MultiGet(keys []interface{}) (Result, error)
  53 +
  54 + // RangeGet looks up a range of rows as specified by the parameter Range type
  55 + RangeGet(rang *Range, excludeStart bool) (Result, error)
53 56 }
54 57
55 58 // Result reads Query results into Go objects, internally buffering them.
@@ -155,6 +158,33 @@ func (q *query) MultiGet(keys []interface{}) (Result, error) {
155 158 return &result{query: *q, buffer: rows}, nil
156 159 }
157 160
  161 +func (q *query) RangeGet(rang *Range, excludeStart bool) (Result, error) {
  162 + var err error
  163 +
  164 + reader := q.pool.Reader().Cf(q.mapping.Cf())
  165 +
  166 + if q.consistencyLevel != 0 {
  167 + reader.ConsistencyLevel(q.consistencyLevel)
  168 + }
  169 +
  170 + q.buildSlice(reader)
  171 +
  172 + if excludeStart && len(rang.Start) > 0 {
  173 + rang.Count += 1
  174 + }
  175 +
  176 + rows, err := reader.RangeGet(rang)
  177 + if err != nil {
  178 + return nil, err
  179 + }
  180 +
  181 + if excludeStart && len(rang.Start) > 0 {
  182 + rows = rows[1:]
  183 + }
  184 +
  185 + return &result{query: *q, buffer: rows}, nil
  186 +}
  187 +
158 188 func (q *query) buildSlice(reader Reader) error {
159 189 start := make([]byte, 0)
160 190 end := make([]byte, 0)

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.