Skip to content
This repository has been archived by the owner on Jun 12, 2020. It is now read-only.

Commit

Permalink
struct mapping cleanup
Browse files Browse the repository at this point in the history
  • Loading branch information
carloscm committed Apr 22, 2012
1 parent e49f29d commit 39cfa7f
Show file tree
Hide file tree
Showing 6 changed files with 302 additions and 637 deletions.
35 changes: 25 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# About

Gossie is a Go library with a low level wrapper for the Cassandra 1.0 Thrift bindings with utilities for connection pooling, primitive type marshaling and easy query building. It also includes a higher level layer that allows mapping structs to Cassandra column famlilies, with support for advanced features like composite column names.
Gossie is a Go library for Apache Cassandra. It includes a wrapper for the Cassandra 1.0 Thrift bindings with utilities for connection pooling, primitive type marshaling and easy query building. It also includes a higher level layer that allows mapping structs to Cassandra column famlilies, with support for advanced features like composite column names.


# Requeriments

Expand All @@ -15,6 +16,7 @@ Installing thrift4go under GOPATH in Go 1:
3) go install thrift
```


# Installing

There is no need to generate a Cassandra Thrift biding, I am providing one with Gossie (and the whole point is not to have to use it!)
Expand All @@ -40,6 +42,7 @@ GOPATH=$GOPATH:`pwd` go test gossie

Launch a Cassandra instance in localhost:9160, create a keyspace named TestGossie, and execute the provided schema-test.txt to create the test column families. Now you can run the Gossie tests.


# Quickstart

### Connection pooling
Expand Down Expand Up @@ -68,23 +71,23 @@ The low level interface is based on passing []byte values for everything, mirror
### Struct maping
The first part of the high level Gossie interface is the Map/Unmap functions. These functions allow to convert Go structs into Row-s, and they have support of advanced features like composites or overriding column names and types.
The first part of the high level Gossie interface is the Map/Unmap functions. These functions allow to convert Go structs into Rows, and they have support of advanced features like composites or overriding column names and types.
```Go
/*
In CQL 3.0:
CREATE TABLE timeline (
user_id varchar,
tweet_id bigint,
author varchar,
body varchar,
PRIMARY KEY (user_id, tweet_id)
CREATE TABLE Timeline (
UserID varchar,
TweetID bigint,
Author varchar,
Body varchar,
PRIMARY KEY (UserID, TweetID)
);
*/

// In Gossie:
type Timeline struct {
UserID string `cf:"Timeline" key:"UserID" col:"TweetID,*name" val:"*value"`
UserID string `cf:"Timeline" key:"UserID,TweetID"`
TweetID int64
Author string
Body string
Expand All @@ -94,6 +97,10 @@ row, err = gossie.Map(&Timeline{"userid", 10000000000004, "Author Name", "Hey th
err = pool.Mutation().Insert("Timeline", row).Run()
````

The `cf` field tag names the column family of this struct, and the `key` field tag starts by naming the field that represents the row key, followed by zero or more fields that represent the components of a composite column name. Any other field not referenced in the `key` will be used as a column value, and its name used as a the column name, or appended to the end of the composite as the last component, if the struct had a composite. The `cf` and `key` field tags can appear at any field in the struct.

The `name` field tag will change the column name to its value when the field it appears on is (un)marhsaled to/from a Cassandra row column. The `type` field tag allows to override the default type Go<->Cassandra type mapping used by Gossie for the field it appears on.

### Cursors

As a convenient wrapper over Map/Unmap and the Query/Mutation interfaces Gossie provides the Cursor interface. This wrapper implements a classic database cursor over rows and composite row slices. Example:
Expand All @@ -117,7 +124,15 @@ err = cursor.Read(tweet2)
// 10000000000004, or gossie.ErrorNotFound was returned in case it was not found
````
Comming soon: range reads for composites with buffering and paging
# Planned features
- Cursor: range reads for composites with buffering and paging
- Cursor: secondary index read with buffering and paging
- Cursor: multiget reads with buffering and paging
- Cursor: batching writes
- High level mapping for Go slices
- High level mapping for Go maps
# License
Expand Down
46 changes: 10 additions & 36 deletions src/gossie/cursor.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ package gossie

import (
"errors"
"fmt"
)

/*
Expand All @@ -12,8 +11,6 @@ todo:
buffering for slicing and range get
Next/Prev for the buffering with autopaging
isSliceColumn == true
Search() and interface(s) for indexed get
multiget
Expand Down Expand Up @@ -57,12 +54,12 @@ func makeCursor(cp *connectionPool) (c *cursor) {

func (c *cursor) Write(source interface{}) error {

row, ms, err := internalMap(source)
row, mi, err := internalMap(source)
if err != nil {
return err
}

if err = c.pool.Mutation().Insert(ms.sm.cf, row).Run(); err != nil {
if err = c.pool.Mutation().Insert(mi.m.cf, row).Run(); err != nil {
return err
}

Expand All @@ -72,61 +69,38 @@ func (c *cursor) Write(source interface{}) error {
func (c *cursor) Read(source interface{}) error {

// deconstruct the source struct into a reflect.Value and a (cached) struct mapping
ms, err := newMappedStruct(source)
mi, err := newMappedInstance(source)
if err != nil {
return err
}

// sanity checks
if ms.sm.isSliceColumn {
return errors.New(fmt.Sprint("Slice field in col tag is unsuported in Cursor for now, check back soon!"))
}

// marshal the key field
key, err := ms.marshalKey()
key, err := mi.m.key.marshalValue(&mi.v)
if err != nil {
return err
}

// start building the query
q := c.pool.Query().Cf(ms.sm.cf)
q := c.pool.Query().Cf(mi.m.cf)

// build a slice composite comparator if needed
if ms.sm.isCompositeColumn {
if len(mi.m.composite) > 0 {
// iterate over the components and set an equality comparison for every simple field
start := make([]byte, 0)
end := make([]byte, 0)
var component int
for component = 0; component < len(ms.sm.columns); component++ {
fm := ms.sm.columns[component]
if fm.fieldKind != baseTypeField {
break
}
b, err := ms.mapColumn(baseTypeField, fm, 0)
for _, f := range mi.m.composite {
b, err := f.marshalValue(&mi.v)
if err != nil {
return err
}
start = packComposite(start, b, eocEquals)
end = packComposite(end, b, eocGreater)
start = append(start, packComposite(b, eocEquals)...)
end = append(end, packComposite(b, eocGreater)...)
}

/*if component < len(ms.sm.columns) {
// we still got one to go, this means the last one was an iterable non-fixed type (*name or go slice)
//fm := ms.sm.columns[component]
// TODO: this will only work for *name
b := make([]byte, 0)
start = packComposite(start, b, o[6], o[7], o[8])
end = packComposite(end, b, o[9], o[10], o[11])
}*/

// TODO: fix hardcoded number of columns
q.Slice(&Slice{Start: start, End: end, Count: 100})
}

//isCompositeColumn bool
//isSliceColumn bool
//isStarNameColumn bool

row, err := q.Get(key)

if err != nil {
Expand Down
2 changes: 1 addition & 1 deletion src/gossie/cursor_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ todo:
*/

type ReasonableOne struct {
Username string `cf:"Reasonable" key:"Username" col:"TweetID,*name" val:"*value"`
Username string `cf:"Reasonable" key:"Username,TweetID"`
TweetID int64
Lat float32
Lon float32
Expand Down
Loading

0 comments on commit 39cfa7f

Please sign in to comment.