diff --git a/README.md b/README.md index b5be064..8626741 100644 --- a/README.md +++ b/README.md @@ -38,6 +38,7 @@ The general idea is to leverage cache-friendly ways of organizing data in [struc - [Expiring Values](#expiring-values) - [Transaction Commit and Rollback](#transaction-commit-and-rollback) - [Streaming Changes](#streaming-changes) +- [Snapshot and Restore](#snapshot-and-restore) - [Complete Example](#complete-example) - [Benchmarks](#benchmarks) - [Contributing](#contributing) @@ -46,7 +47,7 @@ The general idea is to leverage cache-friendly ways of organizing data in [struc In order to get data into the store, you'll need to first create a `Collection` by calling `NewCollection()` method. Each collection requires a schema, which can be either specified manually by calling `CreateColumn()` multiple times or automatically inferred from an object by calling `CreateColumnsOf()` function. -In the example below we're loading some `JSON` data by using `json.Unmarshal()` and auto-creating colums based on the first element on the loaded slice. After this is done, we can then load our data by inserting the objects one by one into the collection. This is accomplished by calling `Insert()` method on the collection itself repeatedly. +In the example below we're loading some `JSON` data by using `json.Unmarshal()` and auto-creating colums based on the first element on the loaded slice. After this is done, we can then load our data by inserting the objects one by one into the collection. This is accomplished by calling `InsertObject()` method on the collection itself repeatedly. ```go data := loadFromJson("players.json") @@ -57,7 +58,7 @@ players.CreateColumnsOf(data[0]) // Insert every item from our loaded data for _, v := range data { - players.Insert(v) + players.InsertObject(v) } ``` @@ -73,16 +74,16 @@ players.CreateColumn("age", column.ForInt16()) // Insert every item from our loaded data for _, v := range loadFromJson("players.json") { - players.Insert(v) + players.InsertObject(v) } ``` -While the previous example demonstrated how to insert many objects, it was doing it one by one and is rather inefficient. This is due to the fact that each `Insert()` call directly on the collection initiates a separate transacion and there's a small performance cost associated with it. If you want to do a bulk insert and insert many values, faster, that can be done by calling `Insert()` on a transaction, as demonstrated in the example below. Note that the only difference is instantiating a transaction by calling the `Query()` method and calling the `txn.Insert()` method on the transaction instead the one on the collection. +While the previous example demonstrated how to insert many objects, it was doing it one by one and is rather inefficient. This is due to the fact that each `InsertObject()` call directly on the collection initiates a separate transacion and there's a small performance cost associated with it. If you want to do a bulk insert and insert many values, faster, that can be done by calling `Insert()` on a transaction, as demonstrated in the example below. Note that the only difference is instantiating a transaction by calling the `Query()` method and calling the `txn.Insert()` method on the transaction instead the one on the collection. ```go players.Query(func(txn *Txn) error { for _, v := range loadFromJson("players.json") { - txn.Insert(v) + txn.InsertObject(v) } return nil // Commit }) @@ -155,79 +156,69 @@ players.Query(func(txn *Txn) error { ## Iterating over Results -In all of the previous examples, we've only been doing `Count()` operation which counts the number of elements in the result set. In this section we'll look how we can iterate over the result set. In short, there's 2 main methods that allow us to do it: +In all of the previous examples, we've only been doing `Count()` operation which counts the number of elements in the result set. In this section we'll look how we can iterate over the result set. -1. `Range()` method which takes in a column name as an argument and allows faster get/set of the values for that column. -2. `Select()` method which doesn't pre-select any specific column, so it's usually a bit slower and it also does not allow any updates. +As before, a transaction needs to be started using the `Query()` method on the collection. After which, we can call the `txn.Range()` method which allows us to iterate over the result set in the transaction. Note that it can be chained right after `With..()` methods, as expected. -Let's first examine the `Range()` method. In the example below we select all of the rogues from our collection and print out their name by using the `Range()` method and providing "name" column to it. The callback containing the `Cursor` allows us to quickly get the value of the column by calling `String()` method to retrieve a string value. It also contains methods such as `Int()`, `Uint()`, `Float()` or more generic `Value()` to pull data of different types. +In order to access the results of the iteration, prior to calling `Range()` method, we need to **first load column reader(s)** we are going to need, using methods such as `txn.String()`, `txn.Float64()`, etc. These prepare read/write buffers necessary to perform efficient lookups while iterating. -```go -players.Query(func(txn *Txn) error { - txn.With("rogue").Range("name", func(v column.Cursor) { - println("rogue name ", v.String()) // Prints the name - }) - return nil -}) -``` - -Now, what if you need two columns? The range only allows you to quickly select a single column, but you can still retrieve other columns by their name during the iteration. This can be accomplished by corresponding `StringAt()`, `FloatAt()`, `IntAt()`, `UintAt()` or `ValueAt()` methods as shown below. +In the example below we select all of the rogues from our collection and print out their name by using the `Range()` method and accessing the "name" column using a column reader which is created by calling `txn.String("name")` method. ```go players.Query(func(txn *Txn) error { - txn.With("rogue").Range("name", func(v column.Cursor) { - println("rogue name ", v.String()) // Prints the name - println("rogue age ", v.IntAt("age")) // Prints the age + names := txn.String("name") // Create a column reader + + return txn.With("rogue").Range(func(i uint32) { + name, _ := names.Get() + println("rogue name", name) }) - return nil }) ``` -On the other hand, `Select()` allows you to do a read-only selection which provides a `Selector` cursor. This cursor does not allow any updates, deletes or inserts and is also not pre-select any particular column. In the example below we print out names of all of the rogues using a selector. +Similarly, if you need to access more columns, you can simply create the appropriate column reader(s) and use them as shown in the example before. ```go players.Query(func(txn *Txn) error { - txn.With("rogue").Select(func(v column.Selector) bool { - println("rogue name ", v.StringAt("name")) // Prints the name - return true - }) - return nil -}) -``` + names := txn.String("name") + ages := txn.Int64("age") -Now, what if you need to quickly delete all some of the data in the collection? In this case `DeleteAll()` or `DeleteIf()` methods come in handy. These methods are very fast (especially `DeleteAll()`) and allow you to quickly delete the appropriate results, transactionally. In the example below we delete all of the rogues from the collection by simply selecting them in the transaction and calling the `DeleteAll()` method. + return txn.With("rogue").Range(func(i uint32) { + name, _ := names.Get() + age, _ := ages.Get() -```go -players.Query(func(txn *Txn) error { - txn.With("rogue").DeleteAll() - return nil + println("rogue name", name) + println("rogue age", age) + }) }) ``` ## Updating Values -In order to update certain items in the collection, you can simply call `Range()` method and the corresponding `Cursor`'s `Set..()` or `Set..At()` methods that allow to update a value of a certain column atomically. The updates won't be directly reflected given that the store supports transactions and only when transaction is commited, then the update will be applied to the collection. This allows for isolation and rollbacks. +In order to update certain items in the collection, you can simply call `Range()` method and use column accessor's `Set()` or `Add()` methods to update a value of a certain column atomically. The updates won't be instantly reflected given that our store supports transactions. Only when transaction is commited, then the update will be applied to the collection, allowing for isolation and rollbacks. In the example below we're selecting all of the rogues and updating both their balance and age to certain values. The transaction returns `nil`, hence it will be automatically committed when `Query()` method returns. ```go players.Query(func(txn *Txn) error { - txn.With("rogue").Range("balance", func(v column.Cursor) { - v.SetFloat64(10.0) // Update the "balance" to 10.0 - v.SetInt64At("age", 50) // Update the "age" to 50 - }) // Select the balance - return nil + balance := txn.Float64("balance") + age := txn.Int64("age") + + return txn.With("rogue").Range(func(i uint32) { + balance.Set(10.0) // Update the "balance" to 10.0 + age.Set(50) // Update the "age" to 50 + }) }) ``` -In certain cases, you might want to atomically increment or decrement numerical values. In order to accomplish this you can use the provided `Add..()` or `Add..At()` operations of the `Cursor` or `Selector`. Note that the indexes will also be updated accordingly and the predicates re-evaluated with the most up-to-date values. In the below example we're incrementing the balance of all our rogues by _500_ atomically. +In certain cases, you might want to atomically increment or decrement numerical values. In order to accomplish this you can use the provided `Add()` operation. Note that the indexes will also be updated accordingly and the predicates re-evaluated with the most up-to-date values. In the below example we're incrementing the balance of all our rogues by _500_ atomically. ```go players.Query(func(txn *Txn) error { - txn.With("rogue").Range("balance", func(v column.Cursor) { - v.AddFloat64(500.0) // Increment the "balance" by 500 + balance := txn.Float64("balance") + + return txn.With("rogue").Range(func(i uint32) { + balance.Add(500.0) // Increment the "balance" by 500 }) - return nil }) ``` @@ -238,7 +229,7 @@ Sometimes, it is useful to automatically delete certain rows when you do not nee In the example below we are inserting an object to the collection and setting the time-to-live to _5 seconds_ from the current time. After this time, the object will be automatically evicted from the collection and its space can be reclaimed. ```go -players.InsertWithTTL(map[string]interface{}{ +players.InsertObjectWithTTL(map[string]interface{}{ "name": "Merlin", "class": "mage", "age": 55, @@ -250,10 +241,14 @@ On an interestig node, since `expire` column which is automatically added to eac ```go players.Query(func(txn *column.Txn) error { - return txn.Range("expire", func(v column.Cursor) { - oldExpire := time.Unix(0, v.Int()) // Convert expiration to time.Time - newExpire := expireAt.Add(1 * time.Hour).UnixNano() // Add some time - v.Set(newExpire) + expire := txn.Int64("expire") + + return txn.Range(func(i uint32) { + if v, ok := expire.Get(); ok && v > 0 { + oldExpire := time.Unix(0, v) // Convert expiration to time.Time + newExpire := expireAt.Add(1 * time.Hour).UnixNano() // Add some time + expire.Set(newExpire) + } }) }) ``` @@ -265,7 +260,8 @@ Transactions allow for isolation between two concurrent operations. In fact, all ```go // Range over all of the players and update (successfully their balance) players.Query(func(txn *column.Txn) error { - txn.Range("balance", func(v column.Cursor) { + balance := txn.Float64("balance") + txn.Range(func(i uint32) { v.Set(10.0) // Update the "balance" to 10.0 }) @@ -279,7 +275,8 @@ Now, in this example, we try to update balance but a query callback returns an e ```go // Range over all of the players and update (successfully their balance) players.Query(func(txn *column.Txn) error { - txn.Range("balance", func(v column.Cursor) { + balance := txn.Float64("balance") + txn.Range(func(i uint32) { v.Set(10.0) // Update the "balance" to 10.0 }) @@ -333,6 +330,34 @@ go func() { }() ``` +## Snapshot and Restore + +The collection can also be saved in a single binary format while the transactions are running. This can allow you to periodically schedule backups or make sure all of the data is persisted when your application terminates. + +In order to take a snapshot, you must first create a valid `io.Writer` destination and then call the `Snapshot()` method on the collection in order to create a snapshot, as demonstrated in the example below. + +```go +dst, err := os.Create("snapshot.bin") +if err != nil { + panic(err) +} + +// Write a snapshot into the dst +err := players.Snapshot(dst) +``` + +Conversely, in order to restore an existing snapshot, you need to first open an `io.Reader` and then call the `Restore()` method on the collection. Note that the collection and its schema must be already initialized, as our snapshots do not carry this information within themselves. + +```go +src, err := os.Open("snapshot.bin") +if err != nil { + panic(err) +} + +// Restore from an existing snapshot +err := players.Restore(src) +``` + ## Complete Example ```go @@ -340,59 +365,49 @@ func main(){ // Create a new columnar collection players := column.NewCollection() + players.CreateColumn("serial", column.ForKey()) + players.CreateColumn("name", column.ForEnum()) + players.CreateColumn("active", column.ForBool()) + players.CreateColumn("class", column.ForEnum()) + players.CreateColumn("race", column.ForEnum()) + players.CreateColumn("age", column.ForFloat64()) + players.CreateColumn("hp", column.ForFloat64()) + players.CreateColumn("mp", column.ForFloat64()) + players.CreateColumn("balance", column.ForFloat64()) + players.CreateColumn("gender", column.ForEnum()) + players.CreateColumn("guild", column.ForEnum()) // index on humans - players.CreateIndex("human", "race", func(v interface{}) bool { - return v == "human" + players.CreateIndex("human", "race", func(r column.Reader) bool { + return r.String() == "human" }) // index for mages - players.CreateIndex("mage", "class", func(v interface{}) bool { - return v == "mage" + players.CreateIndex("mage", "class", func(r column.Reader) bool { + return r.String() == "mage" }) // index for old - players.CreateIndex("old", "age", func(v interface{}) bool { - return v.(float64) >= 30 + players.CreateIndex("old", "age", func(r column.Reader) bool { + return r.Float() >= 30 }) // Load the items into the collection loaded := loadFixture("players.json") - players.CreateColumnsOf(loaded[0]) - for _, v := range loaded { - players.Insert(v) - } - - // This performs a full scan on 3 different columns and compares them given the - // specified predicates. This is not indexed, but does a columnar scan which is - // cache-friendly. players.Query(func(txn *column.Txn) error { - println(txn.WithString("race", func(v string) bool { - return v == "human" - }).WithString("class", func(v string) bool { - return v == "mage" - }).WithFloat("age", func(v float64) bool { - return v >= 30 - }).Count()) // prints the count + for _, v := range loaded { + txn.InsertObject(v) + } return nil }) - // This performs a count, but instead of scanning through the entire dataset, it scans - // over pre-built indexes and combines them using a logical AND operation. The result - // will be the same as the query above but the performance of the query is 10x-100x - // faster depending on the size of the underlying data. + // Run an indexed query players.Query(func(txn *column.Txn) error { - println(txn.With("human", "mage", "old").Count()) // prints the count - return nil - }) - - // Same condition as above, but we also select the actual names of those - // players and iterate through them. - players.Query(func(txn *column.Txn) error { - txn.With("human", "mage", "old").Range("name", func(v column.Cursor) { - println(v.String()) // prints the name - }) // The column to select - return nil + name := txn.Enum("name") + return txn.With("human", "mage", "old").Range(func(idx uint32) { + value, _ := name.Get() + println("old mage, human:", value) + }) }) } ``` @@ -403,46 +418,49 @@ The benchmarks below were ran on a collection of **100,000 items** containing a ``` cpu: Intel(R) Core(TM) i7-9700K CPU @ 3.60GHz -BenchmarkCollection/insert-8 2104 526103 ns/op 1218 B/op 0 allocs/op -BenchmarkCollection/fetch-8 25516224 47.49 ns/op 0 B/op 0 allocs/op -BenchmarkCollection/scan-8 1790 662321 ns/op 1053 B/op 0 allocs/op -BenchmarkCollection/count-8 750022 1541 ns/op 2 B/op 0 allocs/op -BenchmarkCollection/range-8 10000 106408 ns/op 163 B/op 0 allocs/op -BenchmarkCollection/update-at-8 3053438 409.4 ns/op 0 B/op 0 allocs/op -BenchmarkCollection/update-all-8 774 1548279 ns/op 13937 B/op 0 allocs/op -BenchmarkCollection/delete-at-8 6451591 173.6 ns/op 0 B/op 0 allocs/op -BenchmarkCollection/delete-all-8 1318351 901.1 ns/op 1 B/op 0 allocs/op +BenchmarkCollection/insert-8 2523 469481 ns/op 24356 B/op 500 allocs/op +BenchmarkCollection/select-at-8 22194190 54.23 ns/op 0 B/op 0 allocs/op +BenchmarkCollection/scan-8 2068 568953 ns/op 122 B/op 0 allocs/op +BenchmarkCollection/count-8 571449 2057 ns/op 0 B/op 0 allocs/op +BenchmarkCollection/range-8 28660 41695 ns/op 3 B/op 0 allocs/op +BenchmarkCollection/update-at-8 5911978 202.8 ns/op 0 B/op 0 allocs/op +BenchmarkCollection/update-all-8 1280 946272 ns/op 3726 B/op 0 allocs/op +BenchmarkCollection/delete-at-8 6405852 188.9 ns/op 0 B/op 0 allocs/op +BenchmarkCollection/delete-all-8 2073188 562.6 ns/op 0 B/op 0 allocs/op ``` When testing for larger collections, I added a small example (see `examples` folder) and ran it with **20 million rows** inserted, each entry has **12 columns and 4 indexes** that need to be calculated, and a few queries and scans around them. ``` running insert of 20000000 rows... --> insert took 27.8103257s +-> insert took 20.4538183s + +running snapshot of 20000000 rows... +-> snapshot took 2.57960038s running full scan of age >= 30... -> result = 10200000 --> full scan took 55.142806ms +-> full scan took 61.611822ms running full scan of class == "rogue"... -> result = 7160000 --> full scan took 82.8865ms +-> full scan took 81.389954ms running indexed query of human mages... -> result = 1360000 --> indexed query took 544.578µs +-> indexed query took 608.51µs running indexed query of human female mages... -> result = 640000 --> indexed query took 721.409µs +-> indexed query took 794.49µs running update of balance of everyone... -> updated 20000000 rows --> update took 247.224012ms +-> update took 214.182216ms running update of age of mages... -> updated 6040000 rows --> update took 85.669422ms +-> update took 81.292378ms ``` ## Contributing diff --git a/collection.go b/collection.go index 1fd32e6..36f9a86 100644 --- a/collection.go +++ b/collection.go @@ -139,9 +139,9 @@ func (c *Collection) InsertObjectWithTTL(obj Object, ttl time.Duration) (index u } // Insert executes a mutable cursor trasactionally at a new offset. -func (c *Collection) Insert(columnName string, fn func(v Cursor) error) (index uint32, err error) { +func (c *Collection) Insert(fn func(Row) error) (index uint32, err error) { err = c.Query(func(txn *Txn) (innerErr error) { - index, innerErr = txn.Insert(columnName, fn) + index, innerErr = txn.Insert(fn) return }) return @@ -149,56 +149,14 @@ func (c *Collection) Insert(columnName string, fn func(v Cursor) error) (index u // InsertWithTTL executes a mutable cursor trasactionally at a new offset and sets the expiration time // based on the specified time-to-live and returns the allocated index. -func (c *Collection) InsertWithTTL(columnName string, ttl time.Duration, fn func(v Cursor) error) (index uint32, err error) { +func (c *Collection) InsertWithTTL(ttl time.Duration, fn func(Row) error) (index uint32, err error) { err = c.Query(func(txn *Txn) (innerErr error) { - index, innerErr = txn.InsertWithTTL(columnName, ttl, fn) + index, innerErr = txn.InsertWithTTL(ttl, fn) return }) return } -// UpdateAt updates a specific row by initiating a separate transaction for the update. -func (c *Collection) UpdateAt(idx uint32, columnName string, fn func(v Cursor) error) error { - return c.Query(func(txn *Txn) error { - return txn.UpdateAt(idx, columnName, fn) - }) -} - -// UpdateAtKey updates a specific row by initiating a separate transaction for the update. -func (c *Collection) UpdateAtKey(key, columnName string, fn func(v Cursor) error) error { - return c.Query(func(txn *Txn) error { - return txn.UpdateAtKey(key, columnName, fn) - }) -} - -// SelectAt performs a selection on a specific row specified by its index. It returns -// a boolean value indicating whether an element is present at the index or not. -func (c *Collection) SelectAt(idx uint32, fn func(v Selector)) bool { - chunk := uint(idx >> chunkShift) - if idx >= uint32(len(c.fill))<<6 || !c.fill.Contains(idx) { - return false - } - - // Lock the chunk which we are about to read and call the selector delegate - c.slock.RLock(chunk) - fn(Selector{idx: idx, col: c}) - c.slock.RUnlock(chunk) - return true -} - -// SelectAtKey performs a selection on a specific row specified by its key. It returns -// a boolean value indicating whether an element is present at the key or not. -func (c *Collection) SelectAtKey(key string, fn func(v Selector)) (found bool) { - if c.pk == nil { - return false - } - - if idx, ok := c.pk.OffsetOf(key); ok { - found = c.SelectAt(idx, fn) - } - return -} - // DeleteAt attempts to delete an item at the specified index for this collection. If the item // exists, it marks at as deleted and returns true, otherwise it returns false. func (c *Collection) DeleteAt(idx uint32) (deleted bool) { @@ -318,14 +276,28 @@ func (c *Collection) DropIndex(indexName string) error { return nil } +// QueryAt jumps at a particular offset in the collection, sets the cursor to the +// provided position and executes given callback fn. +func (c *Collection) QueryAt(idx uint32, fn func(Row) error) error { + return c.Query(func(txn *Txn) error { + return txn.QueryAt(idx, fn) + }) +} + +// QueryAt jumps at a particular key in the collection, sets the cursor to the +// provided position and executes given callback fn. +func (c *Collection) QueryKey(key string, fn func(Row) error) error { + return c.Query(func(txn *Txn) error { + return txn.QueryKey(key, fn) + }) +} + // Query creates a transaction which allows for filtering and iteration over the // columns in this collection. It also allows for individual rows to be modified or // deleted during iteration (range), but the actual operations will be queued and // executed after the iteration. func (c *Collection) Query(fn func(txn *Txn) error) error { - c.lock.RLock() txn := c.txns.acquire(c) - c.lock.RUnlock() // Execute the query and keep the error for later if err := fn(txn); err != nil { @@ -344,7 +316,6 @@ func (c *Collection) Query(fn func(txn *Txn) error) error { // Close closes the collection and clears up all of the resources. func (c *Collection) Close() error { c.cancel() - return nil } @@ -357,11 +328,12 @@ func (c *Collection) vacuum(ctx context.Context, interval time.Duration) { ticker.Stop() return case <-ticker.C: - now := int(time.Now().UnixNano()) + now := time.Now().UnixNano() c.Query(func(txn *Txn) error { - return txn.With(expireColumn).Range(expireColumn, func(v Cursor) { - if expirateAt := v.Int(); expirateAt != 0 && now >= v.Int() { - v.Delete() + expire := txn.Int64(expireColumn) + return txn.With(expireColumn).Range(func(idx uint32) { + if expirateAt, ok := expire.Get(); ok && expirateAt != 0 && now >= expirateAt { + txn.DeleteAt(idx) } }) }) diff --git a/collection_test.go b/collection_test.go index b39e69b..eeaea53 100644 --- a/collection_test.go +++ b/collection_test.go @@ -6,6 +6,7 @@ package column import ( "context" "encoding/json" + "fmt" "os" "runtime" "sync" @@ -18,17 +19,20 @@ import ( /* cpu: Intel(R) Core(TM) i7-9700K CPU @ 3.60GHz -BenchmarkCollection/insert-8 2174 534746 ns/op 25090 B/op 500 allocs/op -BenchmarkCollection/select-at-8 42206409 28.19 ns/op 0 B/op 0 allocs/op -BenchmarkCollection/scan-8 2116 581193 ns/op 1872 B/op 0 allocs/op -BenchmarkCollection/count-8 748689 1565 ns/op 5 B/op 0 allocs/op -BenchmarkCollection/range-8 16476 73244 ns/op 216 B/op 0 allocs/op -BenchmarkCollection/update-at-8 3717255 316.6 ns/op 1 B/op 0 allocs/op -BenchmarkCollection/update-all-8 1176 1005992 ns/op 7134 B/op 1 allocs/op -BenchmarkCollection/delete-at-8 8403426 145.0 ns/op 0 B/op 0 allocs/op -BenchmarkCollection/delete-all-8 2338410 500.0 ns/op 1 B/op 0 allocs/op +BenchmarkCollection/insert-8 2523 469481 ns/op 24356 B/op 500 allocs/op +BenchmarkCollection/select-at-8 22194190 54.23 ns/op 0 B/op 0 allocs/op +BenchmarkCollection/scan-8 2068 568953 ns/op 122 B/op 0 allocs/op +BenchmarkCollection/count-8 571449 2057 ns/op 0 B/op 0 allocs/op +BenchmarkCollection/range-8 28660 41695 ns/op 3 B/op 0 allocs/op +BenchmarkCollection/update-at-8 5911978 202.8 ns/op 0 B/op 0 allocs/op +BenchmarkCollection/update-all-8 1280 946272 ns/op 3726 B/op 0 allocs/op +BenchmarkCollection/delete-at-8 6405852 188.9 ns/op 0 B/op 0 allocs/op +BenchmarkCollection/delete-all-8 2073188 562.6 ns/op 0 B/op 0 allocs/op */ func BenchmarkCollection(b *testing.B) { + amount := 100000 + players := loadPlayers(amount) + b.Run("insert", func(b *testing.B) { temp := loadPlayers(500) data := loadFixture("players.json") @@ -51,15 +55,14 @@ func BenchmarkCollection(b *testing.B) { } }) - amount := 100000 - players := loadPlayers(amount) b.Run("select-at", func(b *testing.B) { name := "" b.ReportAllocs() b.ResetTimer() for n := 0; n < b.N; n++ { - players.SelectAt(20, func(v Selector) { - name = v.StringAt("name") + players.QueryAt(20, func(r Row) error { + name, _ = r.Enum("name") + return nil }) } assert.NotEmpty(b, name) @@ -99,9 +102,10 @@ func BenchmarkCollection(b *testing.B) { b.ResetTimer() for n := 0; n < b.N; n++ { players.Query(func(txn *Txn) error { - txn.With("human", "mage", "old").Range("name", func(v Cursor) { + names := txn.Enum("name") + txn.With("human", "mage", "old").Range(func(idx uint32) { count++ - name = v.String() + name, _ = names.Get() }) return nil }) @@ -113,8 +117,8 @@ func BenchmarkCollection(b *testing.B) { b.ReportAllocs() b.ResetTimer() for n := 0; n < b.N; n++ { - players.UpdateAt(20, "balance", func(v Cursor) error { - v.Set(1.0) + players.QueryAt(20, func(r Row) error { + r.SetFloat64("balance", 1.0) return nil }) } @@ -125,10 +129,10 @@ func BenchmarkCollection(b *testing.B) { b.ResetTimer() for n := 0; n < b.N; n++ { players.Query(func(txn *Txn) error { - txn.Range("balance", func(v Cursor) { - v.SetFloat64(0.0) + balance := txn.Float64("balance") + return txn.Range(func(idx uint32) { + balance.Set(0.0) }) - return nil }) } }) @@ -179,47 +183,84 @@ func TestCollection(t *testing.T) { })) { // Find the object by its index - assert.True(t, col.SelectAt(idx, func(v Selector) { - assert.Equal(t, "Roman", v.StringAt("name")) + assert.NoError(t, col.QueryAt(idx, func(r Row) error { + name, ok := r.String("name") + assert.True(t, ok) + assert.Equal(t, "Roman", name) + return nil })) } { // Remove the object assert.True(t, col.DeleteAt(idx)) - assert.False(t, col.SelectAt(idx, func(v Selector) { - assert.Fail(t, "unreachable") + assert.Error(t, col.QueryAt(idx, func(r Row) error { + if _, ok := r.String("name"); !ok { + return fmt.Errorf("unreachable") + } + + return nil })) } { // Add a new one, should replace newIdx := col.InsertObject(obj) assert.Equal(t, idx, newIdx) - assert.True(t, col.SelectAt(newIdx, func(v Selector) { - assert.Equal(t, "Roman", v.StringAt("name")) + assert.NoError(t, col.QueryAt(newIdx, func(r Row) error { + name, ok := r.String("name") + assert.True(t, ok) + assert.Equal(t, "Roman", name) + return nil })) } { // Update the wallet - assert.NoError(t, col.UpdateAt(idx, "wallet", func(v Cursor) error { - v.SetFloat64(1000) + col.QueryAt(idx, func(r Row) error { + r.SetFloat64("wallet", 1000) return nil - })) + }) - assert.True(t, col.SelectAt(idx, func(v Selector) { - assert.Equal(t, int64(1000), v.IntAt("wallet")) - assert.Equal(t, true, v.BoolAt("rich")) - })) - } + col.QueryAt(idx, func(r Row) error { + wallet, ok := r.Float64("wallet") + isRich := r.Bool("rich") - { // Drop the colun - col.DropColumn("rich") - col.Query(func(txn *Txn) error { - assert.Equal(t, 0, txn.With("rich").Count()) + assert.True(t, ok) + assert.Equal(t, 1000.0, wallet) + assert.True(t, isRich) return nil }) + + assert.NoError(t, col.QueryAt(idx, func(r Row) error { + wallet, _ := r.Float64("wallet") + isRich := r.Bool("rich") + + assert.Equal(t, 1000.0, wallet) + assert.True(t, isRich) + return nil + })) } } +func TestDropColumn(t *testing.T) { + obj := Object{ + "wallet": 5000, + } + + col := NewCollection() + col.CreateColumnsOf(obj) + assert.NoError(t, col.CreateIndex("rich", "wallet", func(r Reader) bool { + return r.Float() > 100 + })) + + assert.Equal(t, uint32(0), col.InsertObject(obj)) + assert.Equal(t, uint32(1), col.InsertObject(obj)) + + col.DropColumn("rich") + col.Query(func(txn *Txn) error { + assert.Equal(t, 0, txn.With("rich").Count()) + return nil + }) +} + func TestInsertObject(t *testing.T) { col := NewCollection() col.CreateColumn("name", ForString()) @@ -227,10 +268,10 @@ func TestInsertObject(t *testing.T) { col.InsertObject(Object{"name": "B"}) assert.Equal(t, 2, col.Count()) - assert.NoError(t, col.Query(func(txn *Txn) error { - assert.True(t, txn.SelectAt(0, func(v Selector) { - assert.Equal(t, "A", v.StringAt("name")) - })) + assert.NoError(t, col.QueryAt(0, func(r Row) error { + name, ok := r.String("name") + assert.True(t, ok) + assert.Equal(t, "A", name) return nil })) } @@ -253,9 +294,11 @@ func TestExpire(t *testing.T) { // Insert an object col.InsertObjectWithTTL(obj, time.Microsecond) col.Query(func(txn *Txn) error { - return txn.Range(expireColumn, func(v Cursor) { - expireAt := time.Unix(0, int64(v.Int())) - v.SetInt64(expireAt.Add(1 * time.Microsecond).UnixNano()) + expire := txn.Int64(expireColumn) + return txn.Range(func(idx uint32) { + value, _ := expire.Get() + expireAt := time.Unix(0, value) + expire.Set(expireAt.Add(1 * time.Microsecond).UnixNano()) }) }) assert.Equal(t, 1, col.Count()) @@ -400,8 +443,9 @@ func TestConcurrentPointReads(t *testing.T) { // Reader go func() { for i := 0; i < 10000; i++ { - col.SelectAt(99, func(v Selector) { - _ = v.StringAt("name") + col.QueryAt(99, func(r Row) error { + _, _ = r.String("name") + return nil }) atomic.AddInt64(&ops, 1) runtime.Gosched() @@ -412,8 +456,8 @@ func TestConcurrentPointReads(t *testing.T) { // Writer go func() { for i := 0; i < 10000; i++ { - col.UpdateAt(99, "name", func(v Cursor) error { - v.SetString("test") + col.QueryAt(99, func(r Row) error { + r.SetString("name", "test") return nil }) atomic.AddInt64(&ops, 1) @@ -430,8 +474,8 @@ func TestInsert(t *testing.T) { c := NewCollection() c.CreateColumn("name", ForString()) - idx, err := c.Insert("name", func(v Cursor) error { - v.Set("Roman") + idx, err := c.Insert(func(r Row) error { + r.SetString("name", "Roman") return nil }) assert.Equal(t, uint32(0), idx) @@ -442,16 +486,18 @@ func TestInsertWithTTL(t *testing.T) { c := NewCollection() c.CreateColumn("name", ForString()) - idx, err := c.InsertWithTTL("name", time.Hour, func(v Cursor) error { - v.Set("Roman") + idx, err := c.InsertWithTTL(time.Hour, func(r Row) error { + r.SetString("name", "Roman") return nil }) assert.Equal(t, uint32(0), idx) assert.NoError(t, err) - - c.SelectAt(idx, func(v Selector) { - assert.NotZero(t, v.IntAt(expireColumn)) - }) + assert.NoError(t, c.QueryAt(idx, func(r Row) error { + expire, ok := r.Int64(expireColumn) + assert.True(t, ok) + assert.NotZero(t, expire) + return nil + })) } func TestCreateColumnsOfInvalidKind(t *testing.T) { @@ -477,8 +523,8 @@ func TestFindFreeIndex(t *testing.T) { col := NewCollection() assert.NoError(t, col.CreateColumn("name", ForString())) for i := 0; i < 100; i++ { - idx, err := col.Insert("name", func(v Cursor) error { - v.SetString("Roman") + idx, err := col.Insert(func(r Row) error { + r.SetString("name", "Roman") return nil }) assert.NoError(t, err) diff --git a/column.go b/column.go index 55ad216..f1aea53 100644 --- a/column.go +++ b/column.go @@ -185,38 +185,6 @@ func (c *column) Value(idx uint32) (v interface{}, ok bool) { return } -// Value retrieves a value at a specified index -func (c *column) String(idx uint32) (v string, ok bool) { - if column, text := c.Column.(Textual); text { - v, ok = column.LoadString(idx) - } - return -} - -// Float64 retrieves a float64 value at a specified index -func (c *column) Float64(idx uint32) (v float64, ok bool) { - if n, contains := c.Column.(Numeric); contains { - v, ok = n.LoadFloat64(idx) - } - return -} - -// Int64 retrieves an int64 value at a specified index -func (c *column) Int64(idx uint32) (v int64, ok bool) { - if n, contains := c.Column.(Numeric); contains { - v, ok = n.LoadInt64(idx) - } - return -} - -// Uint64 retrieves an uint64 value at a specified index -func (c *column) Uint64(idx uint32) (v uint64, ok bool) { - if n, contains := c.Column.(Numeric); contains { - v, ok = n.LoadUint64(idx) - } - return -} - // --------------------------- booleans ---------------------------- // columnBool represents a boolean column @@ -270,6 +238,94 @@ func (c *columnBool) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { dst.PutBitmap(commit.PutTrue, chunk, c.data) } +// boolReader represens a read-only accessor for boolean values +type boolReader struct { + cursor *uint32 + reader Column +} + +// Get loads the value at the current transaction cursor +func (s boolReader) Get() bool { + return s.reader.Contains(*s.cursor) +} + +// boolReaderFor creates a new reader +func boolReaderFor(txn *Txn, columnName string) boolReader { + column, ok := txn.columnAt(columnName) + if !ok { + panic(fmt.Errorf("column: column '%s' does not exist", columnName)) + } + + return boolReader{ + cursor: &txn.cursor, + reader: column.Column, + } +} + +// boolWriter represents read-write accessor for boolean values +type boolWriter struct { + boolReader + writer *commit.Buffer +} + +// Set sets the value at the current transaction cursor +func (s boolWriter) Set(value bool) { + s.writer.PutBool(*s.cursor, value) +} + +// String returns a string column accessor +func (txn *Txn) Bool(columnName string) boolWriter { + return boolWriter{ + boolReader: boolReaderFor(txn, columnName), + writer: txn.bufferFor(columnName), + } +} + +// --------------------------- Accessor ---------------------------- + +// anyReader represens a read-only accessor for any value +type anyReader struct { + cursor *uint32 + reader Column +} + +// Get loads the value at the current transaction cursor +func (s anyReader) Get() (interface{}, bool) { + return s.reader.Value(*s.cursor) +} + +// anyReaderFor creates a new any reader +func anyReaderFor(txn *Txn, columnName string) anyReader { + column, ok := txn.columnAt(columnName) + if !ok { + panic(fmt.Errorf("column: column '%s' does not exist", columnName)) + } + + return anyReader{ + cursor: &txn.cursor, + reader: column.Column, + } +} + +// anyWriter represents read-write accessor for any column type +type anyWriter struct { + anyReader + writer *commit.Buffer +} + +// Set sets the value at the current transaction cursor +func (s anyWriter) Set(value interface{}) { + s.writer.PutAny(commit.Put, *s.cursor, value) +} + +// Any returns a column accessor +func (txn *Txn) Any(columnName string) anyWriter { + return anyWriter{ + anyReader: anyReaderFor(txn, columnName), + writer: txn.bufferFor(columnName), + } +} + // --------------------------- funcs ---------------------------- // resize calculates the new required capacity and a new index diff --git a/column_generate.go b/column_generate.go index 901beae..4ccee11 100644 --- a/column_generate.go +++ b/column_generate.go @@ -4,6 +4,8 @@ package column import ( + "fmt" + "github.com/kelindar/bitmap" "github.com/kelindar/column/commit" "github.com/kelindar/genny/generic" @@ -13,22 +15,22 @@ import ( type number = generic.Number -// columnNumber represents a generic column -type columnNumber struct { +// numberColumn represents a generic column +type numberColumn struct { fill bitmap.Bitmap // The fill-list data []number // The actual values } // makeNumbers creates a new vector for Numbers func makeNumbers() Column { - return &columnNumber{ + return &numberColumn{ fill: make(bitmap.Bitmap, 0, 4), data: make([]number, 0, 64), } } // Grow grows the size of the column until we have enough to store -func (c *columnNumber) Grow(idx uint32) { +func (c *numberColumn) Grow(idx uint32) { if idx < uint32(len(c.data)) { return } @@ -46,7 +48,7 @@ func (c *columnNumber) Grow(idx uint32) { } // Apply applies a set of operations to the column. -func (c *columnNumber) Apply(r *commit.Reader) { +func (c *numberColumn) Apply(r *commit.Reader) { for r.Next() { switch r.Type { case commit.Put: @@ -68,17 +70,17 @@ func (c *columnNumber) Apply(r *commit.Reader) { } // Contains checks whether the column has a value at a specified index. -func (c *columnNumber) Contains(idx uint32) bool { +func (c *numberColumn) Contains(idx uint32) bool { return c.fill.Contains(idx) } // Index returns the fill list for the column -func (c *columnNumber) Index() *bitmap.Bitmap { +func (c *numberColumn) Index() *bitmap.Bitmap { return &c.fill } // Value retrieves a value at a specified index -func (c *columnNumber) Value(idx uint32) (v interface{}, ok bool) { +func (c *numberColumn) Value(idx uint32) (v interface{}, ok bool) { v = number(0) if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = c.data[idx], true @@ -86,8 +88,16 @@ func (c *columnNumber) Value(idx uint32) (v interface{}, ok bool) { return } +// load retrieves a number value at a specified index +func (c *numberColumn) load(idx uint32) (v number, ok bool) { + if idx < uint32(len(c.data)) && c.fill.Contains(idx) { + v, ok = number(c.data[idx]), true + } + return +} + // LoadFloat64 retrieves a float64 value at a specified index -func (c *columnNumber) LoadFloat64(idx uint32) (v float64, ok bool) { +func (c *numberColumn) LoadFloat64(idx uint32) (v float64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = float64(c.data[idx]), true } @@ -95,7 +105,7 @@ func (c *columnNumber) LoadFloat64(idx uint32) (v float64, ok bool) { } // LoadInt64 retrieves an int64 value at a specified index -func (c *columnNumber) LoadInt64(idx uint32) (v int64, ok bool) { +func (c *numberColumn) LoadInt64(idx uint32) (v int64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = int64(c.data[idx]), true } @@ -103,7 +113,7 @@ func (c *columnNumber) LoadInt64(idx uint32) (v int64, ok bool) { } // LoadUint64 retrieves an uint64 value at a specified index -func (c *columnNumber) LoadUint64(idx uint32) (v uint64, ok bool) { +func (c *numberColumn) LoadUint64(idx uint32) (v uint64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = uint64(c.data[idx]), true } @@ -111,7 +121,7 @@ func (c *columnNumber) LoadUint64(idx uint32) (v uint64, ok bool) { } // FilterFloat64 filters down the values based on the specified predicate. -func (c *columnNumber) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { +func (c *numberColumn) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) bool { idx = offset + idx @@ -120,7 +130,7 @@ func (c *columnNumber) FilterFloat64(offset uint32, index bitmap.Bitmap, predica } // FilterInt64 filters down the values based on the specified predicate. -func (c *columnNumber) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { +func (c *numberColumn) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -129,7 +139,7 @@ func (c *columnNumber) FilterInt64(offset uint32, index bitmap.Bitmap, predicate } // FilterUint64 filters down the values based on the specified predicate. -func (c *columnNumber) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { +func (c *numberColumn) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -138,34 +148,61 @@ func (c *columnNumber) FilterUint64(offset uint32, index bitmap.Bitmap, predicat } // Snapshot writes the entire column into the specified destination buffer -func (c *columnNumber) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { +func (c *numberColumn) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { chunk.Range(c.fill, func(idx uint32) { - dst.PutNumber(commit.Put, idx, c.data[idx]) + dst.PutNumber(idx, c.data[idx]) }) } -// --------------------------- Cursor Update ---------------------------- +// numberReader represens a read-only accessor for number +type numberReader struct { + cursor *uint32 + reader *numberColumn +} -// SetNumber updates a column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetNumber(value number) { - cur.update.PutNumber(commit.Put, cur.idx, value) +// Get loads the value at the current transaction cursor +func (s numberReader) Get() (number, bool) { + return s.reader.load(*s.cursor) } -// AddNumber atomically increments/decrements the current value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddNumber(amount number) { - cur.update.PutNumber(commit.Add, cur.idx, amount) +// numberReaderFor creates a new number reader +func numberReaderFor(txn *Txn, columnName string) numberReader { + column, ok := txn.columnAt(columnName) + if !ok { + panic(fmt.Errorf("column: column '%s' does not exist", columnName)) + } + + reader, ok := column.Column.(*numberColumn) + if !ok { + panic(fmt.Errorf("column: column '%s' is not of type %T", columnName, number(0))) + } + + return numberReader{ + cursor: &txn.cursor, + reader: reader, + } +} + +// numberWriter represents a read-write accessor for number +type numberWriter struct { + numberReader + writer *commit.Buffer +} + +// Set sets the value at the current transaction cursor +func (s numberWriter) Set(value number) { + s.writer.PutNumber(*s.cursor, value) } -// SetNumberAt updates a specified column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetNumberAt(column string, value number) { - cur.txn.bufferFor(column).PutNumber(commit.Put, cur.idx, value) +// Add atomically adds a delta to the value at the current transaction cursor +func (s numberWriter) Add(delta number) { + s.writer.AddNumber(*s.cursor, delta) } -// AddNumberAt atomically increments/decrements the column value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddNumberAt(column string, amount number) { - cur.txn.bufferFor(column).PutNumber(commit.Add, cur.idx, amount) +// Number returns a read-write accessor for number column +func (txn *Txn) Number(columnName string) numberWriter { + return numberWriter{ + numberReader: numberReaderFor(txn, columnName), + writer: txn.bufferFor(columnName), + } } diff --git a/column_index.go b/column_index.go index 91e3a88..46de214 100644 --- a/column_index.go +++ b/column_index.go @@ -4,6 +4,7 @@ package column import ( + "fmt" "sync" "github.com/kelindar/bitmap" @@ -25,7 +26,6 @@ type Reader interface { // Assert reader implementations. Both our cursor and commit reader need to implement // this so that we can feed it to the index transparently. var _ Reader = new(commit.Reader) -var _ Reader = new(Cursor) // --------------------------- Index ---------------------------- @@ -153,3 +153,33 @@ func (c *columnKey) OffsetOf(v string) (uint32, bool) { c.lock.RUnlock() return idx, ok } + +// slice accessor for keys +type keySlice struct { + cursor *uint32 + writer *commit.Buffer + reader *columnKey +} + +// Set sets the value at the current transaction index +func (s keySlice) Set(value string) { + s.writer.PutString(commit.Put, *s.cursor, value) +} + +// Get loads the value at the current transaction index +func (s keySlice) Get() (string, bool) { + return s.reader.LoadString(*s.cursor) +} + +// Enum returns a enumerable column accessor +func (txn *Txn) Key() keySlice { + if txn.owner.pk == nil { + panic(fmt.Errorf("column: primary key column does not exist")) + } + + return keySlice{ + cursor: &txn.cursor, + writer: txn.bufferFor(txn.owner.pk.name), + reader: txn.owner.pk, + } +} diff --git a/column_numbers.go b/column_numbers.go index 7db5bbb..23b7fa8 100644 --- a/column_numbers.go +++ b/column_numbers.go @@ -4,28 +4,30 @@ package column import ( + "fmt" + "github.com/kelindar/bitmap" "github.com/kelindar/column/commit" ) // --------------------------- Float32s ---------------------------- -// columnFloat32 represents a generic column -type columnfloat32 struct { +// float32Column represents a generic column +type float32Column struct { fill bitmap.Bitmap // The fill-list data []float32 // The actual values } // makeFloat32s creates a new vector for Float32s func makeFloat32s() Column { - return &columnfloat32{ + return &float32Column{ fill: make(bitmap.Bitmap, 0, 4), data: make([]float32, 0, 64), } } // Grow grows the size of the column until we have enough to store -func (c *columnfloat32) Grow(idx uint32) { +func (c *float32Column) Grow(idx uint32) { if idx < uint32(len(c.data)) { return } @@ -43,7 +45,7 @@ func (c *columnfloat32) Grow(idx uint32) { } // Apply applies a set of operations to the column. -func (c *columnfloat32) Apply(r *commit.Reader) { +func (c *float32Column) Apply(r *commit.Reader) { for r.Next() { switch r.Type { case commit.Put: @@ -65,17 +67,17 @@ func (c *columnfloat32) Apply(r *commit.Reader) { } // Contains checks whether the column has a value at a specified index. -func (c *columnfloat32) Contains(idx uint32) bool { +func (c *float32Column) Contains(idx uint32) bool { return c.fill.Contains(idx) } // Index returns the fill list for the column -func (c *columnfloat32) Index() *bitmap.Bitmap { +func (c *float32Column) Index() *bitmap.Bitmap { return &c.fill } // Value retrieves a value at a specified index -func (c *columnfloat32) Value(idx uint32) (v interface{}, ok bool) { +func (c *float32Column) Value(idx uint32) (v interface{}, ok bool) { v = float32(0) if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = c.data[idx], true @@ -83,8 +85,16 @@ func (c *columnfloat32) Value(idx uint32) (v interface{}, ok bool) { return } +// load retrieves a float32 value at a specified index +func (c *float32Column) load(idx uint32) (v float32, ok bool) { + if idx < uint32(len(c.data)) && c.fill.Contains(idx) { + v, ok = float32(c.data[idx]), true + } + return +} + // LoadFloat64 retrieves a float64 value at a specified index -func (c *columnfloat32) LoadFloat64(idx uint32) (v float64, ok bool) { +func (c *float32Column) LoadFloat64(idx uint32) (v float64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = float64(c.data[idx]), true } @@ -92,7 +102,7 @@ func (c *columnfloat32) LoadFloat64(idx uint32) (v float64, ok bool) { } // LoadInt64 retrieves an int64 value at a specified index -func (c *columnfloat32) LoadInt64(idx uint32) (v int64, ok bool) { +func (c *float32Column) LoadInt64(idx uint32) (v int64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = int64(c.data[idx]), true } @@ -100,7 +110,7 @@ func (c *columnfloat32) LoadInt64(idx uint32) (v int64, ok bool) { } // LoadUint64 retrieves an uint64 value at a specified index -func (c *columnfloat32) LoadUint64(idx uint32) (v uint64, ok bool) { +func (c *float32Column) LoadUint64(idx uint32) (v uint64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = uint64(c.data[idx]), true } @@ -108,7 +118,7 @@ func (c *columnfloat32) LoadUint64(idx uint32) (v uint64, ok bool) { } // FilterFloat64 filters down the values based on the specified predicate. -func (c *columnfloat32) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { +func (c *float32Column) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) bool { idx = offset + idx @@ -117,7 +127,7 @@ func (c *columnfloat32) FilterFloat64(offset uint32, index bitmap.Bitmap, predic } // FilterInt64 filters down the values based on the specified predicate. -func (c *columnfloat32) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { +func (c *float32Column) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -126,7 +136,7 @@ func (c *columnfloat32) FilterInt64(offset uint32, index bitmap.Bitmap, predicat } // FilterUint64 filters down the values based on the specified predicate. -func (c *columnfloat32) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { +func (c *float32Column) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -135,56 +145,83 @@ func (c *columnfloat32) FilterUint64(offset uint32, index bitmap.Bitmap, predica } // Snapshot writes the entire column into the specified destination buffer -func (c *columnfloat32) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { +func (c *float32Column) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { chunk.Range(c.fill, func(idx uint32) { - dst.PutFloat32(commit.Put, idx, c.data[idx]) + dst.PutFloat32(idx, c.data[idx]) }) } -// --------------------------- Cursor Update ---------------------------- +// float32Reader represens a read-only accessor for float32 +type float32Reader struct { + cursor *uint32 + reader *float32Column +} -// SetFloat32 updates a column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetFloat32(value float32) { - cur.update.PutFloat32(commit.Put, cur.idx, value) +// Get loads the value at the current transaction cursor +func (s float32Reader) Get() (float32, bool) { + return s.reader.load(*s.cursor) } -// AddFloat32 atomically increments/decrements the current value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddFloat32(amount float32) { - cur.update.PutFloat32(commit.Add, cur.idx, amount) +// float32ReaderFor creates a new float32 reader +func float32ReaderFor(txn *Txn, columnName string) float32Reader { + column, ok := txn.columnAt(columnName) + if !ok { + panic(fmt.Errorf("column: column '%s' does not exist", columnName)) + } + + reader, ok := column.Column.(*float32Column) + if !ok { + panic(fmt.Errorf("column: column '%s' is not of type %T", columnName, float32(0))) + } + + return float32Reader{ + cursor: &txn.cursor, + reader: reader, + } +} + +// float32Writer represents a read-write accessor for float32 +type float32Writer struct { + float32Reader + writer *commit.Buffer +} + +// Set sets the value at the current transaction cursor +func (s float32Writer) Set(value float32) { + s.writer.PutFloat32(*s.cursor, value) } -// SetFloat32At updates a specified column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetFloat32At(column string, value float32) { - cur.txn.bufferFor(column).PutFloat32(commit.Put, cur.idx, value) +// Add atomically adds a delta to the value at the current transaction cursor +func (s float32Writer) Add(delta float32) { + s.writer.AddFloat32(*s.cursor, delta) } -// AddFloat32At atomically increments/decrements the column value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddFloat32At(column string, amount float32) { - cur.txn.bufferFor(column).PutFloat32(commit.Add, cur.idx, amount) +// Float32 returns a read-write accessor for float32 column +func (txn *Txn) Float32(columnName string) float32Writer { + return float32Writer{ + float32Reader: float32ReaderFor(txn, columnName), + writer: txn.bufferFor(columnName), + } } // --------------------------- Float64s ---------------------------- -// columnFloat64 represents a generic column -type columnfloat64 struct { +// float64Column represents a generic column +type float64Column struct { fill bitmap.Bitmap // The fill-list data []float64 // The actual values } // makeFloat64s creates a new vector for Float64s func makeFloat64s() Column { - return &columnfloat64{ + return &float64Column{ fill: make(bitmap.Bitmap, 0, 4), data: make([]float64, 0, 64), } } // Grow grows the size of the column until we have enough to store -func (c *columnfloat64) Grow(idx uint32) { +func (c *float64Column) Grow(idx uint32) { if idx < uint32(len(c.data)) { return } @@ -202,7 +239,7 @@ func (c *columnfloat64) Grow(idx uint32) { } // Apply applies a set of operations to the column. -func (c *columnfloat64) Apply(r *commit.Reader) { +func (c *float64Column) Apply(r *commit.Reader) { for r.Next() { switch r.Type { case commit.Put: @@ -224,17 +261,17 @@ func (c *columnfloat64) Apply(r *commit.Reader) { } // Contains checks whether the column has a value at a specified index. -func (c *columnfloat64) Contains(idx uint32) bool { +func (c *float64Column) Contains(idx uint32) bool { return c.fill.Contains(idx) } // Index returns the fill list for the column -func (c *columnfloat64) Index() *bitmap.Bitmap { +func (c *float64Column) Index() *bitmap.Bitmap { return &c.fill } // Value retrieves a value at a specified index -func (c *columnfloat64) Value(idx uint32) (v interface{}, ok bool) { +func (c *float64Column) Value(idx uint32) (v interface{}, ok bool) { v = float64(0) if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = c.data[idx], true @@ -242,8 +279,16 @@ func (c *columnfloat64) Value(idx uint32) (v interface{}, ok bool) { return } +// load retrieves a float64 value at a specified index +func (c *float64Column) load(idx uint32) (v float64, ok bool) { + if idx < uint32(len(c.data)) && c.fill.Contains(idx) { + v, ok = float64(c.data[idx]), true + } + return +} + // LoadFloat64 retrieves a float64 value at a specified index -func (c *columnfloat64) LoadFloat64(idx uint32) (v float64, ok bool) { +func (c *float64Column) LoadFloat64(idx uint32) (v float64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = float64(c.data[idx]), true } @@ -251,7 +296,7 @@ func (c *columnfloat64) LoadFloat64(idx uint32) (v float64, ok bool) { } // LoadInt64 retrieves an int64 value at a specified index -func (c *columnfloat64) LoadInt64(idx uint32) (v int64, ok bool) { +func (c *float64Column) LoadInt64(idx uint32) (v int64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = int64(c.data[idx]), true } @@ -259,7 +304,7 @@ func (c *columnfloat64) LoadInt64(idx uint32) (v int64, ok bool) { } // LoadUint64 retrieves an uint64 value at a specified index -func (c *columnfloat64) LoadUint64(idx uint32) (v uint64, ok bool) { +func (c *float64Column) LoadUint64(idx uint32) (v uint64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = uint64(c.data[idx]), true } @@ -267,7 +312,7 @@ func (c *columnfloat64) LoadUint64(idx uint32) (v uint64, ok bool) { } // FilterFloat64 filters down the values based on the specified predicate. -func (c *columnfloat64) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { +func (c *float64Column) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) bool { idx = offset + idx @@ -276,7 +321,7 @@ func (c *columnfloat64) FilterFloat64(offset uint32, index bitmap.Bitmap, predic } // FilterInt64 filters down the values based on the specified predicate. -func (c *columnfloat64) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { +func (c *float64Column) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -285,7 +330,7 @@ func (c *columnfloat64) FilterInt64(offset uint32, index bitmap.Bitmap, predicat } // FilterUint64 filters down the values based on the specified predicate. -func (c *columnfloat64) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { +func (c *float64Column) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -294,56 +339,83 @@ func (c *columnfloat64) FilterUint64(offset uint32, index bitmap.Bitmap, predica } // Snapshot writes the entire column into the specified destination buffer -func (c *columnfloat64) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { +func (c *float64Column) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { chunk.Range(c.fill, func(idx uint32) { - dst.PutFloat64(commit.Put, idx, c.data[idx]) + dst.PutFloat64(idx, c.data[idx]) }) } -// --------------------------- Cursor Update ---------------------------- +// float64Reader represens a read-only accessor for float64 +type float64Reader struct { + cursor *uint32 + reader *float64Column +} + +// Get loads the value at the current transaction cursor +func (s float64Reader) Get() (float64, bool) { + return s.reader.load(*s.cursor) +} + +// float64ReaderFor creates a new float64 reader +func float64ReaderFor(txn *Txn, columnName string) float64Reader { + column, ok := txn.columnAt(columnName) + if !ok { + panic(fmt.Errorf("column: column '%s' does not exist", columnName)) + } + + reader, ok := column.Column.(*float64Column) + if !ok { + panic(fmt.Errorf("column: column '%s' is not of type %T", columnName, float64(0))) + } + + return float64Reader{ + cursor: &txn.cursor, + reader: reader, + } +} -// SetFloat64 updates a column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetFloat64(value float64) { - cur.update.PutFloat64(commit.Put, cur.idx, value) +// float64Writer represents a read-write accessor for float64 +type float64Writer struct { + float64Reader + writer *commit.Buffer } -// AddFloat64 atomically increments/decrements the current value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddFloat64(amount float64) { - cur.update.PutFloat64(commit.Add, cur.idx, amount) +// Set sets the value at the current transaction cursor +func (s float64Writer) Set(value float64) { + s.writer.PutFloat64(*s.cursor, value) } -// SetFloat64At updates a specified column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetFloat64At(column string, value float64) { - cur.txn.bufferFor(column).PutFloat64(commit.Put, cur.idx, value) +// Add atomically adds a delta to the value at the current transaction cursor +func (s float64Writer) Add(delta float64) { + s.writer.AddFloat64(*s.cursor, delta) } -// AddFloat64At atomically increments/decrements the column value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddFloat64At(column string, amount float64) { - cur.txn.bufferFor(column).PutFloat64(commit.Add, cur.idx, amount) +// Float64 returns a read-write accessor for float64 column +func (txn *Txn) Float64(columnName string) float64Writer { + return float64Writer{ + float64Reader: float64ReaderFor(txn, columnName), + writer: txn.bufferFor(columnName), + } } // --------------------------- Ints ---------------------------- -// columnInt represents a generic column -type columnint struct { +// intColumn represents a generic column +type intColumn struct { fill bitmap.Bitmap // The fill-list data []int // The actual values } // makeInts creates a new vector for Ints func makeInts() Column { - return &columnint{ + return &intColumn{ fill: make(bitmap.Bitmap, 0, 4), data: make([]int, 0, 64), } } // Grow grows the size of the column until we have enough to store -func (c *columnint) Grow(idx uint32) { +func (c *intColumn) Grow(idx uint32) { if idx < uint32(len(c.data)) { return } @@ -361,7 +433,7 @@ func (c *columnint) Grow(idx uint32) { } // Apply applies a set of operations to the column. -func (c *columnint) Apply(r *commit.Reader) { +func (c *intColumn) Apply(r *commit.Reader) { for r.Next() { switch r.Type { case commit.Put: @@ -383,17 +455,17 @@ func (c *columnint) Apply(r *commit.Reader) { } // Contains checks whether the column has a value at a specified index. -func (c *columnint) Contains(idx uint32) bool { +func (c *intColumn) Contains(idx uint32) bool { return c.fill.Contains(idx) } // Index returns the fill list for the column -func (c *columnint) Index() *bitmap.Bitmap { +func (c *intColumn) Index() *bitmap.Bitmap { return &c.fill } // Value retrieves a value at a specified index -func (c *columnint) Value(idx uint32) (v interface{}, ok bool) { +func (c *intColumn) Value(idx uint32) (v interface{}, ok bool) { v = int(0) if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = c.data[idx], true @@ -401,8 +473,16 @@ func (c *columnint) Value(idx uint32) (v interface{}, ok bool) { return } +// load retrieves a int value at a specified index +func (c *intColumn) load(idx uint32) (v int, ok bool) { + if idx < uint32(len(c.data)) && c.fill.Contains(idx) { + v, ok = int(c.data[idx]), true + } + return +} + // LoadFloat64 retrieves a float64 value at a specified index -func (c *columnint) LoadFloat64(idx uint32) (v float64, ok bool) { +func (c *intColumn) LoadFloat64(idx uint32) (v float64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = float64(c.data[idx]), true } @@ -410,7 +490,7 @@ func (c *columnint) LoadFloat64(idx uint32) (v float64, ok bool) { } // LoadInt64 retrieves an int64 value at a specified index -func (c *columnint) LoadInt64(idx uint32) (v int64, ok bool) { +func (c *intColumn) LoadInt64(idx uint32) (v int64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = int64(c.data[idx]), true } @@ -418,7 +498,7 @@ func (c *columnint) LoadInt64(idx uint32) (v int64, ok bool) { } // LoadUint64 retrieves an uint64 value at a specified index -func (c *columnint) LoadUint64(idx uint32) (v uint64, ok bool) { +func (c *intColumn) LoadUint64(idx uint32) (v uint64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = uint64(c.data[idx]), true } @@ -426,7 +506,7 @@ func (c *columnint) LoadUint64(idx uint32) (v uint64, ok bool) { } // FilterFloat64 filters down the values based on the specified predicate. -func (c *columnint) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { +func (c *intColumn) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) bool { idx = offset + idx @@ -435,7 +515,7 @@ func (c *columnint) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate } // FilterInt64 filters down the values based on the specified predicate. -func (c *columnint) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { +func (c *intColumn) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -444,7 +524,7 @@ func (c *columnint) FilterInt64(offset uint32, index bitmap.Bitmap, predicate fu } // FilterUint64 filters down the values based on the specified predicate. -func (c *columnint) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { +func (c *intColumn) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -453,56 +533,83 @@ func (c *columnint) FilterUint64(offset uint32, index bitmap.Bitmap, predicate f } // Snapshot writes the entire column into the specified destination buffer -func (c *columnint) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { +func (c *intColumn) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { chunk.Range(c.fill, func(idx uint32) { - dst.PutInt(commit.Put, idx, c.data[idx]) + dst.PutInt(idx, c.data[idx]) }) } -// --------------------------- Cursor Update ---------------------------- +// intReader represens a read-only accessor for int +type intReader struct { + cursor *uint32 + reader *intColumn +} + +// Get loads the value at the current transaction cursor +func (s intReader) Get() (int, bool) { + return s.reader.load(*s.cursor) +} + +// intReaderFor creates a new int reader +func intReaderFor(txn *Txn, columnName string) intReader { + column, ok := txn.columnAt(columnName) + if !ok { + panic(fmt.Errorf("column: column '%s' does not exist", columnName)) + } -// SetInt updates a column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetInt(value int) { - cur.update.PutInt(commit.Put, cur.idx, value) + reader, ok := column.Column.(*intColumn) + if !ok { + panic(fmt.Errorf("column: column '%s' is not of type %T", columnName, int(0))) + } + + return intReader{ + cursor: &txn.cursor, + reader: reader, + } } -// AddInt atomically increments/decrements the current value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddInt(amount int) { - cur.update.PutInt(commit.Add, cur.idx, amount) +// intWriter represents a read-write accessor for int +type intWriter struct { + intReader + writer *commit.Buffer } -// SetIntAt updates a specified column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetIntAt(column string, value int) { - cur.txn.bufferFor(column).PutInt(commit.Put, cur.idx, value) +// Set sets the value at the current transaction cursor +func (s intWriter) Set(value int) { + s.writer.PutInt(*s.cursor, value) } -// AddIntAt atomically increments/decrements the column value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddIntAt(column string, amount int) { - cur.txn.bufferFor(column).PutInt(commit.Add, cur.idx, amount) +// Add atomically adds a delta to the value at the current transaction cursor +func (s intWriter) Add(delta int) { + s.writer.AddInt(*s.cursor, delta) +} + +// Int returns a read-write accessor for int column +func (txn *Txn) Int(columnName string) intWriter { + return intWriter{ + intReader: intReaderFor(txn, columnName), + writer: txn.bufferFor(columnName), + } } // --------------------------- Int16s ---------------------------- -// columnInt16 represents a generic column -type columnint16 struct { +// int16Column represents a generic column +type int16Column struct { fill bitmap.Bitmap // The fill-list data []int16 // The actual values } // makeInt16s creates a new vector for Int16s func makeInt16s() Column { - return &columnint16{ + return &int16Column{ fill: make(bitmap.Bitmap, 0, 4), data: make([]int16, 0, 64), } } // Grow grows the size of the column until we have enough to store -func (c *columnint16) Grow(idx uint32) { +func (c *int16Column) Grow(idx uint32) { if idx < uint32(len(c.data)) { return } @@ -520,7 +627,7 @@ func (c *columnint16) Grow(idx uint32) { } // Apply applies a set of operations to the column. -func (c *columnint16) Apply(r *commit.Reader) { +func (c *int16Column) Apply(r *commit.Reader) { for r.Next() { switch r.Type { case commit.Put: @@ -542,17 +649,17 @@ func (c *columnint16) Apply(r *commit.Reader) { } // Contains checks whether the column has a value at a specified index. -func (c *columnint16) Contains(idx uint32) bool { +func (c *int16Column) Contains(idx uint32) bool { return c.fill.Contains(idx) } // Index returns the fill list for the column -func (c *columnint16) Index() *bitmap.Bitmap { +func (c *int16Column) Index() *bitmap.Bitmap { return &c.fill } // Value retrieves a value at a specified index -func (c *columnint16) Value(idx uint32) (v interface{}, ok bool) { +func (c *int16Column) Value(idx uint32) (v interface{}, ok bool) { v = int16(0) if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = c.data[idx], true @@ -560,8 +667,16 @@ func (c *columnint16) Value(idx uint32) (v interface{}, ok bool) { return } +// load retrieves a int16 value at a specified index +func (c *int16Column) load(idx uint32) (v int16, ok bool) { + if idx < uint32(len(c.data)) && c.fill.Contains(idx) { + v, ok = int16(c.data[idx]), true + } + return +} + // LoadFloat64 retrieves a float64 value at a specified index -func (c *columnint16) LoadFloat64(idx uint32) (v float64, ok bool) { +func (c *int16Column) LoadFloat64(idx uint32) (v float64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = float64(c.data[idx]), true } @@ -569,7 +684,7 @@ func (c *columnint16) LoadFloat64(idx uint32) (v float64, ok bool) { } // LoadInt64 retrieves an int64 value at a specified index -func (c *columnint16) LoadInt64(idx uint32) (v int64, ok bool) { +func (c *int16Column) LoadInt64(idx uint32) (v int64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = int64(c.data[idx]), true } @@ -577,7 +692,7 @@ func (c *columnint16) LoadInt64(idx uint32) (v int64, ok bool) { } // LoadUint64 retrieves an uint64 value at a specified index -func (c *columnint16) LoadUint64(idx uint32) (v uint64, ok bool) { +func (c *int16Column) LoadUint64(idx uint32) (v uint64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = uint64(c.data[idx]), true } @@ -585,7 +700,7 @@ func (c *columnint16) LoadUint64(idx uint32) (v uint64, ok bool) { } // FilterFloat64 filters down the values based on the specified predicate. -func (c *columnint16) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { +func (c *int16Column) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) bool { idx = offset + idx @@ -594,7 +709,7 @@ func (c *columnint16) FilterFloat64(offset uint32, index bitmap.Bitmap, predicat } // FilterInt64 filters down the values based on the specified predicate. -func (c *columnint16) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { +func (c *int16Column) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -603,7 +718,7 @@ func (c *columnint16) FilterInt64(offset uint32, index bitmap.Bitmap, predicate } // FilterUint64 filters down the values based on the specified predicate. -func (c *columnint16) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { +func (c *int16Column) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -612,56 +727,83 @@ func (c *columnint16) FilterUint64(offset uint32, index bitmap.Bitmap, predicate } // Snapshot writes the entire column into the specified destination buffer -func (c *columnint16) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { +func (c *int16Column) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { chunk.Range(c.fill, func(idx uint32) { - dst.PutInt16(commit.Put, idx, c.data[idx]) + dst.PutInt16(idx, c.data[idx]) }) } -// --------------------------- Cursor Update ---------------------------- +// int16Reader represens a read-only accessor for int16 +type int16Reader struct { + cursor *uint32 + reader *int16Column +} -// SetInt16 updates a column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetInt16(value int16) { - cur.update.PutInt16(commit.Put, cur.idx, value) +// Get loads the value at the current transaction cursor +func (s int16Reader) Get() (int16, bool) { + return s.reader.load(*s.cursor) } -// AddInt16 atomically increments/decrements the current value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddInt16(amount int16) { - cur.update.PutInt16(commit.Add, cur.idx, amount) +// int16ReaderFor creates a new int16 reader +func int16ReaderFor(txn *Txn, columnName string) int16Reader { + column, ok := txn.columnAt(columnName) + if !ok { + panic(fmt.Errorf("column: column '%s' does not exist", columnName)) + } + + reader, ok := column.Column.(*int16Column) + if !ok { + panic(fmt.Errorf("column: column '%s' is not of type %T", columnName, int16(0))) + } + + return int16Reader{ + cursor: &txn.cursor, + reader: reader, + } } -// SetInt16At updates a specified column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetInt16At(column string, value int16) { - cur.txn.bufferFor(column).PutInt16(commit.Put, cur.idx, value) +// int16Writer represents a read-write accessor for int16 +type int16Writer struct { + int16Reader + writer *commit.Buffer } -// AddInt16At atomically increments/decrements the column value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddInt16At(column string, amount int16) { - cur.txn.bufferFor(column).PutInt16(commit.Add, cur.idx, amount) +// Set sets the value at the current transaction cursor +func (s int16Writer) Set(value int16) { + s.writer.PutInt16(*s.cursor, value) +} + +// Add atomically adds a delta to the value at the current transaction cursor +func (s int16Writer) Add(delta int16) { + s.writer.AddInt16(*s.cursor, delta) +} + +// Int16 returns a read-write accessor for int16 column +func (txn *Txn) Int16(columnName string) int16Writer { + return int16Writer{ + int16Reader: int16ReaderFor(txn, columnName), + writer: txn.bufferFor(columnName), + } } // --------------------------- Int32s ---------------------------- -// columnInt32 represents a generic column -type columnint32 struct { +// int32Column represents a generic column +type int32Column struct { fill bitmap.Bitmap // The fill-list data []int32 // The actual values } // makeInt32s creates a new vector for Int32s func makeInt32s() Column { - return &columnint32{ + return &int32Column{ fill: make(bitmap.Bitmap, 0, 4), data: make([]int32, 0, 64), } } // Grow grows the size of the column until we have enough to store -func (c *columnint32) Grow(idx uint32) { +func (c *int32Column) Grow(idx uint32) { if idx < uint32(len(c.data)) { return } @@ -679,7 +821,7 @@ func (c *columnint32) Grow(idx uint32) { } // Apply applies a set of operations to the column. -func (c *columnint32) Apply(r *commit.Reader) { +func (c *int32Column) Apply(r *commit.Reader) { for r.Next() { switch r.Type { case commit.Put: @@ -701,17 +843,17 @@ func (c *columnint32) Apply(r *commit.Reader) { } // Contains checks whether the column has a value at a specified index. -func (c *columnint32) Contains(idx uint32) bool { +func (c *int32Column) Contains(idx uint32) bool { return c.fill.Contains(idx) } // Index returns the fill list for the column -func (c *columnint32) Index() *bitmap.Bitmap { +func (c *int32Column) Index() *bitmap.Bitmap { return &c.fill } // Value retrieves a value at a specified index -func (c *columnint32) Value(idx uint32) (v interface{}, ok bool) { +func (c *int32Column) Value(idx uint32) (v interface{}, ok bool) { v = int32(0) if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = c.data[idx], true @@ -719,8 +861,16 @@ func (c *columnint32) Value(idx uint32) (v interface{}, ok bool) { return } +// load retrieves a int32 value at a specified index +func (c *int32Column) load(idx uint32) (v int32, ok bool) { + if idx < uint32(len(c.data)) && c.fill.Contains(idx) { + v, ok = int32(c.data[idx]), true + } + return +} + // LoadFloat64 retrieves a float64 value at a specified index -func (c *columnint32) LoadFloat64(idx uint32) (v float64, ok bool) { +func (c *int32Column) LoadFloat64(idx uint32) (v float64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = float64(c.data[idx]), true } @@ -728,7 +878,7 @@ func (c *columnint32) LoadFloat64(idx uint32) (v float64, ok bool) { } // LoadInt64 retrieves an int64 value at a specified index -func (c *columnint32) LoadInt64(idx uint32) (v int64, ok bool) { +func (c *int32Column) LoadInt64(idx uint32) (v int64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = int64(c.data[idx]), true } @@ -736,7 +886,7 @@ func (c *columnint32) LoadInt64(idx uint32) (v int64, ok bool) { } // LoadUint64 retrieves an uint64 value at a specified index -func (c *columnint32) LoadUint64(idx uint32) (v uint64, ok bool) { +func (c *int32Column) LoadUint64(idx uint32) (v uint64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = uint64(c.data[idx]), true } @@ -744,7 +894,7 @@ func (c *columnint32) LoadUint64(idx uint32) (v uint64, ok bool) { } // FilterFloat64 filters down the values based on the specified predicate. -func (c *columnint32) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { +func (c *int32Column) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) bool { idx = offset + idx @@ -753,7 +903,7 @@ func (c *columnint32) FilterFloat64(offset uint32, index bitmap.Bitmap, predicat } // FilterInt64 filters down the values based on the specified predicate. -func (c *columnint32) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { +func (c *int32Column) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -762,7 +912,7 @@ func (c *columnint32) FilterInt64(offset uint32, index bitmap.Bitmap, predicate } // FilterUint64 filters down the values based on the specified predicate. -func (c *columnint32) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { +func (c *int32Column) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -771,56 +921,83 @@ func (c *columnint32) FilterUint64(offset uint32, index bitmap.Bitmap, predicate } // Snapshot writes the entire column into the specified destination buffer -func (c *columnint32) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { +func (c *int32Column) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { chunk.Range(c.fill, func(idx uint32) { - dst.PutInt32(commit.Put, idx, c.data[idx]) + dst.PutInt32(idx, c.data[idx]) }) } -// --------------------------- Cursor Update ---------------------------- +// int32Reader represens a read-only accessor for int32 +type int32Reader struct { + cursor *uint32 + reader *int32Column +} + +// Get loads the value at the current transaction cursor +func (s int32Reader) Get() (int32, bool) { + return s.reader.load(*s.cursor) +} + +// int32ReaderFor creates a new int32 reader +func int32ReaderFor(txn *Txn, columnName string) int32Reader { + column, ok := txn.columnAt(columnName) + if !ok { + panic(fmt.Errorf("column: column '%s' does not exist", columnName)) + } + + reader, ok := column.Column.(*int32Column) + if !ok { + panic(fmt.Errorf("column: column '%s' is not of type %T", columnName, int32(0))) + } -// SetInt32 updates a column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetInt32(value int32) { - cur.update.PutInt32(commit.Put, cur.idx, value) + return int32Reader{ + cursor: &txn.cursor, + reader: reader, + } } -// AddInt32 atomically increments/decrements the current value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddInt32(amount int32) { - cur.update.PutInt32(commit.Add, cur.idx, amount) +// int32Writer represents a read-write accessor for int32 +type int32Writer struct { + int32Reader + writer *commit.Buffer } -// SetInt32At updates a specified column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetInt32At(column string, value int32) { - cur.txn.bufferFor(column).PutInt32(commit.Put, cur.idx, value) +// Set sets the value at the current transaction cursor +func (s int32Writer) Set(value int32) { + s.writer.PutInt32(*s.cursor, value) } -// AddInt32At atomically increments/decrements the column value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddInt32At(column string, amount int32) { - cur.txn.bufferFor(column).PutInt32(commit.Add, cur.idx, amount) +// Add atomically adds a delta to the value at the current transaction cursor +func (s int32Writer) Add(delta int32) { + s.writer.AddInt32(*s.cursor, delta) +} + +// Int32 returns a read-write accessor for int32 column +func (txn *Txn) Int32(columnName string) int32Writer { + return int32Writer{ + int32Reader: int32ReaderFor(txn, columnName), + writer: txn.bufferFor(columnName), + } } // --------------------------- Int64s ---------------------------- -// columnInt64 represents a generic column -type columnint64 struct { +// int64Column represents a generic column +type int64Column struct { fill bitmap.Bitmap // The fill-list data []int64 // The actual values } // makeInt64s creates a new vector for Int64s func makeInt64s() Column { - return &columnint64{ + return &int64Column{ fill: make(bitmap.Bitmap, 0, 4), data: make([]int64, 0, 64), } } // Grow grows the size of the column until we have enough to store -func (c *columnint64) Grow(idx uint32) { +func (c *int64Column) Grow(idx uint32) { if idx < uint32(len(c.data)) { return } @@ -838,7 +1015,7 @@ func (c *columnint64) Grow(idx uint32) { } // Apply applies a set of operations to the column. -func (c *columnint64) Apply(r *commit.Reader) { +func (c *int64Column) Apply(r *commit.Reader) { for r.Next() { switch r.Type { case commit.Put: @@ -860,17 +1037,17 @@ func (c *columnint64) Apply(r *commit.Reader) { } // Contains checks whether the column has a value at a specified index. -func (c *columnint64) Contains(idx uint32) bool { +func (c *int64Column) Contains(idx uint32) bool { return c.fill.Contains(idx) } // Index returns the fill list for the column -func (c *columnint64) Index() *bitmap.Bitmap { +func (c *int64Column) Index() *bitmap.Bitmap { return &c.fill } // Value retrieves a value at a specified index -func (c *columnint64) Value(idx uint32) (v interface{}, ok bool) { +func (c *int64Column) Value(idx uint32) (v interface{}, ok bool) { v = int64(0) if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = c.data[idx], true @@ -878,8 +1055,16 @@ func (c *columnint64) Value(idx uint32) (v interface{}, ok bool) { return } +// load retrieves a int64 value at a specified index +func (c *int64Column) load(idx uint32) (v int64, ok bool) { + if idx < uint32(len(c.data)) && c.fill.Contains(idx) { + v, ok = int64(c.data[idx]), true + } + return +} + // LoadFloat64 retrieves a float64 value at a specified index -func (c *columnint64) LoadFloat64(idx uint32) (v float64, ok bool) { +func (c *int64Column) LoadFloat64(idx uint32) (v float64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = float64(c.data[idx]), true } @@ -887,7 +1072,7 @@ func (c *columnint64) LoadFloat64(idx uint32) (v float64, ok bool) { } // LoadInt64 retrieves an int64 value at a specified index -func (c *columnint64) LoadInt64(idx uint32) (v int64, ok bool) { +func (c *int64Column) LoadInt64(idx uint32) (v int64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = int64(c.data[idx]), true } @@ -895,7 +1080,7 @@ func (c *columnint64) LoadInt64(idx uint32) (v int64, ok bool) { } // LoadUint64 retrieves an uint64 value at a specified index -func (c *columnint64) LoadUint64(idx uint32) (v uint64, ok bool) { +func (c *int64Column) LoadUint64(idx uint32) (v uint64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = uint64(c.data[idx]), true } @@ -903,7 +1088,7 @@ func (c *columnint64) LoadUint64(idx uint32) (v uint64, ok bool) { } // FilterFloat64 filters down the values based on the specified predicate. -func (c *columnint64) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { +func (c *int64Column) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) bool { idx = offset + idx @@ -912,7 +1097,7 @@ func (c *columnint64) FilterFloat64(offset uint32, index bitmap.Bitmap, predicat } // FilterInt64 filters down the values based on the specified predicate. -func (c *columnint64) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { +func (c *int64Column) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -921,7 +1106,7 @@ func (c *columnint64) FilterInt64(offset uint32, index bitmap.Bitmap, predicate } // FilterUint64 filters down the values based on the specified predicate. -func (c *columnint64) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { +func (c *int64Column) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -930,56 +1115,83 @@ func (c *columnint64) FilterUint64(offset uint32, index bitmap.Bitmap, predicate } // Snapshot writes the entire column into the specified destination buffer -func (c *columnint64) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { +func (c *int64Column) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { chunk.Range(c.fill, func(idx uint32) { - dst.PutInt64(commit.Put, idx, c.data[idx]) + dst.PutInt64(idx, c.data[idx]) }) } -// --------------------------- Cursor Update ---------------------------- +// int64Reader represens a read-only accessor for int64 +type int64Reader struct { + cursor *uint32 + reader *int64Column +} -// SetInt64 updates a column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetInt64(value int64) { - cur.update.PutInt64(commit.Put, cur.idx, value) +// Get loads the value at the current transaction cursor +func (s int64Reader) Get() (int64, bool) { + return s.reader.load(*s.cursor) } -// AddInt64 atomically increments/decrements the current value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddInt64(amount int64) { - cur.update.PutInt64(commit.Add, cur.idx, amount) +// int64ReaderFor creates a new int64 reader +func int64ReaderFor(txn *Txn, columnName string) int64Reader { + column, ok := txn.columnAt(columnName) + if !ok { + panic(fmt.Errorf("column: column '%s' does not exist", columnName)) + } + + reader, ok := column.Column.(*int64Column) + if !ok { + panic(fmt.Errorf("column: column '%s' is not of type %T", columnName, int64(0))) + } + + return int64Reader{ + cursor: &txn.cursor, + reader: reader, + } +} + +// int64Writer represents a read-write accessor for int64 +type int64Writer struct { + int64Reader + writer *commit.Buffer +} + +// Set sets the value at the current transaction cursor +func (s int64Writer) Set(value int64) { + s.writer.PutInt64(*s.cursor, value) } -// SetInt64At updates a specified column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetInt64At(column string, value int64) { - cur.txn.bufferFor(column).PutInt64(commit.Put, cur.idx, value) +// Add atomically adds a delta to the value at the current transaction cursor +func (s int64Writer) Add(delta int64) { + s.writer.AddInt64(*s.cursor, delta) } -// AddInt64At atomically increments/decrements the column value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddInt64At(column string, amount int64) { - cur.txn.bufferFor(column).PutInt64(commit.Add, cur.idx, amount) +// Int64 returns a read-write accessor for int64 column +func (txn *Txn) Int64(columnName string) int64Writer { + return int64Writer{ + int64Reader: int64ReaderFor(txn, columnName), + writer: txn.bufferFor(columnName), + } } // --------------------------- Uints ---------------------------- -// columnUint represents a generic column -type columnuint struct { +// uintColumn represents a generic column +type uintColumn struct { fill bitmap.Bitmap // The fill-list data []uint // The actual values } // makeUints creates a new vector for Uints func makeUints() Column { - return &columnuint{ + return &uintColumn{ fill: make(bitmap.Bitmap, 0, 4), data: make([]uint, 0, 64), } } // Grow grows the size of the column until we have enough to store -func (c *columnuint) Grow(idx uint32) { +func (c *uintColumn) Grow(idx uint32) { if idx < uint32(len(c.data)) { return } @@ -997,7 +1209,7 @@ func (c *columnuint) Grow(idx uint32) { } // Apply applies a set of operations to the column. -func (c *columnuint) Apply(r *commit.Reader) { +func (c *uintColumn) Apply(r *commit.Reader) { for r.Next() { switch r.Type { case commit.Put: @@ -1019,17 +1231,17 @@ func (c *columnuint) Apply(r *commit.Reader) { } // Contains checks whether the column has a value at a specified index. -func (c *columnuint) Contains(idx uint32) bool { +func (c *uintColumn) Contains(idx uint32) bool { return c.fill.Contains(idx) } // Index returns the fill list for the column -func (c *columnuint) Index() *bitmap.Bitmap { +func (c *uintColumn) Index() *bitmap.Bitmap { return &c.fill } // Value retrieves a value at a specified index -func (c *columnuint) Value(idx uint32) (v interface{}, ok bool) { +func (c *uintColumn) Value(idx uint32) (v interface{}, ok bool) { v = uint(0) if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = c.data[idx], true @@ -1037,8 +1249,16 @@ func (c *columnuint) Value(idx uint32) (v interface{}, ok bool) { return } +// load retrieves a uint value at a specified index +func (c *uintColumn) load(idx uint32) (v uint, ok bool) { + if idx < uint32(len(c.data)) && c.fill.Contains(idx) { + v, ok = uint(c.data[idx]), true + } + return +} + // LoadFloat64 retrieves a float64 value at a specified index -func (c *columnuint) LoadFloat64(idx uint32) (v float64, ok bool) { +func (c *uintColumn) LoadFloat64(idx uint32) (v float64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = float64(c.data[idx]), true } @@ -1046,7 +1266,7 @@ func (c *columnuint) LoadFloat64(idx uint32) (v float64, ok bool) { } // LoadInt64 retrieves an int64 value at a specified index -func (c *columnuint) LoadInt64(idx uint32) (v int64, ok bool) { +func (c *uintColumn) LoadInt64(idx uint32) (v int64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = int64(c.data[idx]), true } @@ -1054,7 +1274,7 @@ func (c *columnuint) LoadInt64(idx uint32) (v int64, ok bool) { } // LoadUint64 retrieves an uint64 value at a specified index -func (c *columnuint) LoadUint64(idx uint32) (v uint64, ok bool) { +func (c *uintColumn) LoadUint64(idx uint32) (v uint64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = uint64(c.data[idx]), true } @@ -1062,7 +1282,7 @@ func (c *columnuint) LoadUint64(idx uint32) (v uint64, ok bool) { } // FilterFloat64 filters down the values based on the specified predicate. -func (c *columnuint) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { +func (c *uintColumn) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) bool { idx = offset + idx @@ -1071,7 +1291,7 @@ func (c *columnuint) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate } // FilterInt64 filters down the values based on the specified predicate. -func (c *columnuint) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { +func (c *uintColumn) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -1080,7 +1300,7 @@ func (c *columnuint) FilterInt64(offset uint32, index bitmap.Bitmap, predicate f } // FilterUint64 filters down the values based on the specified predicate. -func (c *columnuint) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { +func (c *uintColumn) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -1089,56 +1309,83 @@ func (c *columnuint) FilterUint64(offset uint32, index bitmap.Bitmap, predicate } // Snapshot writes the entire column into the specified destination buffer -func (c *columnuint) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { +func (c *uintColumn) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { chunk.Range(c.fill, func(idx uint32) { - dst.PutUint(commit.Put, idx, c.data[idx]) + dst.PutUint(idx, c.data[idx]) }) } -// --------------------------- Cursor Update ---------------------------- +// uintReader represens a read-only accessor for uint +type uintReader struct { + cursor *uint32 + reader *uintColumn +} + +// Get loads the value at the current transaction cursor +func (s uintReader) Get() (uint, bool) { + return s.reader.load(*s.cursor) +} + +// uintReaderFor creates a new uint reader +func uintReaderFor(txn *Txn, columnName string) uintReader { + column, ok := txn.columnAt(columnName) + if !ok { + panic(fmt.Errorf("column: column '%s' does not exist", columnName)) + } + + reader, ok := column.Column.(*uintColumn) + if !ok { + panic(fmt.Errorf("column: column '%s' is not of type %T", columnName, uint(0))) + } + + return uintReader{ + cursor: &txn.cursor, + reader: reader, + } +} -// SetUint updates a column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetUint(value uint) { - cur.update.PutUint(commit.Put, cur.idx, value) +// uintWriter represents a read-write accessor for uint +type uintWriter struct { + uintReader + writer *commit.Buffer } -// AddUint atomically increments/decrements the current value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddUint(amount uint) { - cur.update.PutUint(commit.Add, cur.idx, amount) +// Set sets the value at the current transaction cursor +func (s uintWriter) Set(value uint) { + s.writer.PutUint(*s.cursor, value) } -// SetUintAt updates a specified column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetUintAt(column string, value uint) { - cur.txn.bufferFor(column).PutUint(commit.Put, cur.idx, value) +// Add atomically adds a delta to the value at the current transaction cursor +func (s uintWriter) Add(delta uint) { + s.writer.AddUint(*s.cursor, delta) } -// AddUintAt atomically increments/decrements the column value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddUintAt(column string, amount uint) { - cur.txn.bufferFor(column).PutUint(commit.Add, cur.idx, amount) +// Uint returns a read-write accessor for uint column +func (txn *Txn) Uint(columnName string) uintWriter { + return uintWriter{ + uintReader: uintReaderFor(txn, columnName), + writer: txn.bufferFor(columnName), + } } // --------------------------- Uint16s ---------------------------- -// columnUint16 represents a generic column -type columnuint16 struct { +// uint16Column represents a generic column +type uint16Column struct { fill bitmap.Bitmap // The fill-list data []uint16 // The actual values } // makeUint16s creates a new vector for Uint16s func makeUint16s() Column { - return &columnuint16{ + return &uint16Column{ fill: make(bitmap.Bitmap, 0, 4), data: make([]uint16, 0, 64), } } // Grow grows the size of the column until we have enough to store -func (c *columnuint16) Grow(idx uint32) { +func (c *uint16Column) Grow(idx uint32) { if idx < uint32(len(c.data)) { return } @@ -1156,7 +1403,7 @@ func (c *columnuint16) Grow(idx uint32) { } // Apply applies a set of operations to the column. -func (c *columnuint16) Apply(r *commit.Reader) { +func (c *uint16Column) Apply(r *commit.Reader) { for r.Next() { switch r.Type { case commit.Put: @@ -1178,17 +1425,17 @@ func (c *columnuint16) Apply(r *commit.Reader) { } // Contains checks whether the column has a value at a specified index. -func (c *columnuint16) Contains(idx uint32) bool { +func (c *uint16Column) Contains(idx uint32) bool { return c.fill.Contains(idx) } // Index returns the fill list for the column -func (c *columnuint16) Index() *bitmap.Bitmap { +func (c *uint16Column) Index() *bitmap.Bitmap { return &c.fill } // Value retrieves a value at a specified index -func (c *columnuint16) Value(idx uint32) (v interface{}, ok bool) { +func (c *uint16Column) Value(idx uint32) (v interface{}, ok bool) { v = uint16(0) if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = c.data[idx], true @@ -1196,8 +1443,16 @@ func (c *columnuint16) Value(idx uint32) (v interface{}, ok bool) { return } +// load retrieves a uint16 value at a specified index +func (c *uint16Column) load(idx uint32) (v uint16, ok bool) { + if idx < uint32(len(c.data)) && c.fill.Contains(idx) { + v, ok = uint16(c.data[idx]), true + } + return +} + // LoadFloat64 retrieves a float64 value at a specified index -func (c *columnuint16) LoadFloat64(idx uint32) (v float64, ok bool) { +func (c *uint16Column) LoadFloat64(idx uint32) (v float64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = float64(c.data[idx]), true } @@ -1205,7 +1460,7 @@ func (c *columnuint16) LoadFloat64(idx uint32) (v float64, ok bool) { } // LoadInt64 retrieves an int64 value at a specified index -func (c *columnuint16) LoadInt64(idx uint32) (v int64, ok bool) { +func (c *uint16Column) LoadInt64(idx uint32) (v int64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = int64(c.data[idx]), true } @@ -1213,7 +1468,7 @@ func (c *columnuint16) LoadInt64(idx uint32) (v int64, ok bool) { } // LoadUint64 retrieves an uint64 value at a specified index -func (c *columnuint16) LoadUint64(idx uint32) (v uint64, ok bool) { +func (c *uint16Column) LoadUint64(idx uint32) (v uint64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = uint64(c.data[idx]), true } @@ -1221,7 +1476,7 @@ func (c *columnuint16) LoadUint64(idx uint32) (v uint64, ok bool) { } // FilterFloat64 filters down the values based on the specified predicate. -func (c *columnuint16) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { +func (c *uint16Column) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) bool { idx = offset + idx @@ -1230,7 +1485,7 @@ func (c *columnuint16) FilterFloat64(offset uint32, index bitmap.Bitmap, predica } // FilterInt64 filters down the values based on the specified predicate. -func (c *columnuint16) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { +func (c *uint16Column) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -1239,7 +1494,7 @@ func (c *columnuint16) FilterInt64(offset uint32, index bitmap.Bitmap, predicate } // FilterUint64 filters down the values based on the specified predicate. -func (c *columnuint16) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { +func (c *uint16Column) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -1248,56 +1503,83 @@ func (c *columnuint16) FilterUint64(offset uint32, index bitmap.Bitmap, predicat } // Snapshot writes the entire column into the specified destination buffer -func (c *columnuint16) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { +func (c *uint16Column) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { chunk.Range(c.fill, func(idx uint32) { - dst.PutUint16(commit.Put, idx, c.data[idx]) + dst.PutUint16(idx, c.data[idx]) }) } -// --------------------------- Cursor Update ---------------------------- +// uint16Reader represens a read-only accessor for uint16 +type uint16Reader struct { + cursor *uint32 + reader *uint16Column +} + +// Get loads the value at the current transaction cursor +func (s uint16Reader) Get() (uint16, bool) { + return s.reader.load(*s.cursor) +} + +// uint16ReaderFor creates a new uint16 reader +func uint16ReaderFor(txn *Txn, columnName string) uint16Reader { + column, ok := txn.columnAt(columnName) + if !ok { + panic(fmt.Errorf("column: column '%s' does not exist", columnName)) + } + + reader, ok := column.Column.(*uint16Column) + if !ok { + panic(fmt.Errorf("column: column '%s' is not of type %T", columnName, uint16(0))) + } -// SetUint16 updates a column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetUint16(value uint16) { - cur.update.PutUint16(commit.Put, cur.idx, value) + return uint16Reader{ + cursor: &txn.cursor, + reader: reader, + } } -// AddUint16 atomically increments/decrements the current value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddUint16(amount uint16) { - cur.update.PutUint16(commit.Add, cur.idx, amount) +// uint16Writer represents a read-write accessor for uint16 +type uint16Writer struct { + uint16Reader + writer *commit.Buffer } -// SetUint16At updates a specified column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetUint16At(column string, value uint16) { - cur.txn.bufferFor(column).PutUint16(commit.Put, cur.idx, value) +// Set sets the value at the current transaction cursor +func (s uint16Writer) Set(value uint16) { + s.writer.PutUint16(*s.cursor, value) } -// AddUint16At atomically increments/decrements the column value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddUint16At(column string, amount uint16) { - cur.txn.bufferFor(column).PutUint16(commit.Add, cur.idx, amount) +// Add atomically adds a delta to the value at the current transaction cursor +func (s uint16Writer) Add(delta uint16) { + s.writer.AddUint16(*s.cursor, delta) +} + +// Uint16 returns a read-write accessor for uint16 column +func (txn *Txn) Uint16(columnName string) uint16Writer { + return uint16Writer{ + uint16Reader: uint16ReaderFor(txn, columnName), + writer: txn.bufferFor(columnName), + } } // --------------------------- Uint32s ---------------------------- -// columnUint32 represents a generic column -type columnuint32 struct { +// uint32Column represents a generic column +type uint32Column struct { fill bitmap.Bitmap // The fill-list data []uint32 // The actual values } // makeUint32s creates a new vector for Uint32s func makeUint32s() Column { - return &columnuint32{ + return &uint32Column{ fill: make(bitmap.Bitmap, 0, 4), data: make([]uint32, 0, 64), } } // Grow grows the size of the column until we have enough to store -func (c *columnuint32) Grow(idx uint32) { +func (c *uint32Column) Grow(idx uint32) { if idx < uint32(len(c.data)) { return } @@ -1315,7 +1597,7 @@ func (c *columnuint32) Grow(idx uint32) { } // Apply applies a set of operations to the column. -func (c *columnuint32) Apply(r *commit.Reader) { +func (c *uint32Column) Apply(r *commit.Reader) { for r.Next() { switch r.Type { case commit.Put: @@ -1337,17 +1619,17 @@ func (c *columnuint32) Apply(r *commit.Reader) { } // Contains checks whether the column has a value at a specified index. -func (c *columnuint32) Contains(idx uint32) bool { +func (c *uint32Column) Contains(idx uint32) bool { return c.fill.Contains(idx) } // Index returns the fill list for the column -func (c *columnuint32) Index() *bitmap.Bitmap { +func (c *uint32Column) Index() *bitmap.Bitmap { return &c.fill } // Value retrieves a value at a specified index -func (c *columnuint32) Value(idx uint32) (v interface{}, ok bool) { +func (c *uint32Column) Value(idx uint32) (v interface{}, ok bool) { v = uint32(0) if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = c.data[idx], true @@ -1355,8 +1637,16 @@ func (c *columnuint32) Value(idx uint32) (v interface{}, ok bool) { return } +// load retrieves a uint32 value at a specified index +func (c *uint32Column) load(idx uint32) (v uint32, ok bool) { + if idx < uint32(len(c.data)) && c.fill.Contains(idx) { + v, ok = uint32(c.data[idx]), true + } + return +} + // LoadFloat64 retrieves a float64 value at a specified index -func (c *columnuint32) LoadFloat64(idx uint32) (v float64, ok bool) { +func (c *uint32Column) LoadFloat64(idx uint32) (v float64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = float64(c.data[idx]), true } @@ -1364,7 +1654,7 @@ func (c *columnuint32) LoadFloat64(idx uint32) (v float64, ok bool) { } // LoadInt64 retrieves an int64 value at a specified index -func (c *columnuint32) LoadInt64(idx uint32) (v int64, ok bool) { +func (c *uint32Column) LoadInt64(idx uint32) (v int64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = int64(c.data[idx]), true } @@ -1372,7 +1662,7 @@ func (c *columnuint32) LoadInt64(idx uint32) (v int64, ok bool) { } // LoadUint64 retrieves an uint64 value at a specified index -func (c *columnuint32) LoadUint64(idx uint32) (v uint64, ok bool) { +func (c *uint32Column) LoadUint64(idx uint32) (v uint64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = uint64(c.data[idx]), true } @@ -1380,7 +1670,7 @@ func (c *columnuint32) LoadUint64(idx uint32) (v uint64, ok bool) { } // FilterFloat64 filters down the values based on the specified predicate. -func (c *columnuint32) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { +func (c *uint32Column) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) bool { idx = offset + idx @@ -1389,7 +1679,7 @@ func (c *columnuint32) FilterFloat64(offset uint32, index bitmap.Bitmap, predica } // FilterInt64 filters down the values based on the specified predicate. -func (c *columnuint32) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { +func (c *uint32Column) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -1398,7 +1688,7 @@ func (c *columnuint32) FilterInt64(offset uint32, index bitmap.Bitmap, predicate } // FilterUint64 filters down the values based on the specified predicate. -func (c *columnuint32) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { +func (c *uint32Column) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -1407,56 +1697,83 @@ func (c *columnuint32) FilterUint64(offset uint32, index bitmap.Bitmap, predicat } // Snapshot writes the entire column into the specified destination buffer -func (c *columnuint32) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { +func (c *uint32Column) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { chunk.Range(c.fill, func(idx uint32) { - dst.PutUint32(commit.Put, idx, c.data[idx]) + dst.PutUint32(idx, c.data[idx]) }) } -// --------------------------- Cursor Update ---------------------------- +// uint32Reader represens a read-only accessor for uint32 +type uint32Reader struct { + cursor *uint32 + reader *uint32Column +} + +// Get loads the value at the current transaction cursor +func (s uint32Reader) Get() (uint32, bool) { + return s.reader.load(*s.cursor) +} + +// uint32ReaderFor creates a new uint32 reader +func uint32ReaderFor(txn *Txn, columnName string) uint32Reader { + column, ok := txn.columnAt(columnName) + if !ok { + panic(fmt.Errorf("column: column '%s' does not exist", columnName)) + } -// SetUint32 updates a column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetUint32(value uint32) { - cur.update.PutUint32(commit.Put, cur.idx, value) + reader, ok := column.Column.(*uint32Column) + if !ok { + panic(fmt.Errorf("column: column '%s' is not of type %T", columnName, uint32(0))) + } + + return uint32Reader{ + cursor: &txn.cursor, + reader: reader, + } } -// AddUint32 atomically increments/decrements the current value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddUint32(amount uint32) { - cur.update.PutUint32(commit.Add, cur.idx, amount) +// uint32Writer represents a read-write accessor for uint32 +type uint32Writer struct { + uint32Reader + writer *commit.Buffer } -// SetUint32At updates a specified column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetUint32At(column string, value uint32) { - cur.txn.bufferFor(column).PutUint32(commit.Put, cur.idx, value) +// Set sets the value at the current transaction cursor +func (s uint32Writer) Set(value uint32) { + s.writer.PutUint32(*s.cursor, value) } -// AddUint32At atomically increments/decrements the column value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddUint32At(column string, amount uint32) { - cur.txn.bufferFor(column).PutUint32(commit.Add, cur.idx, amount) +// Add atomically adds a delta to the value at the current transaction cursor +func (s uint32Writer) Add(delta uint32) { + s.writer.AddUint32(*s.cursor, delta) +} + +// Uint32 returns a read-write accessor for uint32 column +func (txn *Txn) Uint32(columnName string) uint32Writer { + return uint32Writer{ + uint32Reader: uint32ReaderFor(txn, columnName), + writer: txn.bufferFor(columnName), + } } // --------------------------- Uint64s ---------------------------- -// columnUint64 represents a generic column -type columnuint64 struct { +// uint64Column represents a generic column +type uint64Column struct { fill bitmap.Bitmap // The fill-list data []uint64 // The actual values } // makeUint64s creates a new vector for Uint64s func makeUint64s() Column { - return &columnuint64{ + return &uint64Column{ fill: make(bitmap.Bitmap, 0, 4), data: make([]uint64, 0, 64), } } // Grow grows the size of the column until we have enough to store -func (c *columnuint64) Grow(idx uint32) { +func (c *uint64Column) Grow(idx uint32) { if idx < uint32(len(c.data)) { return } @@ -1474,7 +1791,7 @@ func (c *columnuint64) Grow(idx uint32) { } // Apply applies a set of operations to the column. -func (c *columnuint64) Apply(r *commit.Reader) { +func (c *uint64Column) Apply(r *commit.Reader) { for r.Next() { switch r.Type { case commit.Put: @@ -1496,17 +1813,17 @@ func (c *columnuint64) Apply(r *commit.Reader) { } // Contains checks whether the column has a value at a specified index. -func (c *columnuint64) Contains(idx uint32) bool { +func (c *uint64Column) Contains(idx uint32) bool { return c.fill.Contains(idx) } // Index returns the fill list for the column -func (c *columnuint64) Index() *bitmap.Bitmap { +func (c *uint64Column) Index() *bitmap.Bitmap { return &c.fill } // Value retrieves a value at a specified index -func (c *columnuint64) Value(idx uint32) (v interface{}, ok bool) { +func (c *uint64Column) Value(idx uint32) (v interface{}, ok bool) { v = uint64(0) if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = c.data[idx], true @@ -1514,8 +1831,16 @@ func (c *columnuint64) Value(idx uint32) (v interface{}, ok bool) { return } +// load retrieves a uint64 value at a specified index +func (c *uint64Column) load(idx uint32) (v uint64, ok bool) { + if idx < uint32(len(c.data)) && c.fill.Contains(idx) { + v, ok = uint64(c.data[idx]), true + } + return +} + // LoadFloat64 retrieves a float64 value at a specified index -func (c *columnuint64) LoadFloat64(idx uint32) (v float64, ok bool) { +func (c *uint64Column) LoadFloat64(idx uint32) (v float64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = float64(c.data[idx]), true } @@ -1523,7 +1848,7 @@ func (c *columnuint64) LoadFloat64(idx uint32) (v float64, ok bool) { } // LoadInt64 retrieves an int64 value at a specified index -func (c *columnuint64) LoadInt64(idx uint32) (v int64, ok bool) { +func (c *uint64Column) LoadInt64(idx uint32) (v int64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = int64(c.data[idx]), true } @@ -1531,7 +1856,7 @@ func (c *columnuint64) LoadInt64(idx uint32) (v int64, ok bool) { } // LoadUint64 retrieves an uint64 value at a specified index -func (c *columnuint64) LoadUint64(idx uint32) (v uint64, ok bool) { +func (c *uint64Column) LoadUint64(idx uint32) (v uint64, ok bool) { if idx < uint32(len(c.data)) && c.fill.Contains(idx) { v, ok = uint64(c.data[idx]), true } @@ -1539,7 +1864,7 @@ func (c *columnuint64) LoadUint64(idx uint32) (v uint64, ok bool) { } // FilterFloat64 filters down the values based on the specified predicate. -func (c *columnuint64) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { +func (c *uint64Column) FilterFloat64(offset uint32, index bitmap.Bitmap, predicate func(v float64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) bool { idx = offset + idx @@ -1548,7 +1873,7 @@ func (c *columnuint64) FilterFloat64(offset uint32, index bitmap.Bitmap, predica } // FilterInt64 filters down the values based on the specified predicate. -func (c *columnuint64) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { +func (c *uint64Column) FilterInt64(offset uint32, index bitmap.Bitmap, predicate func(v int64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -1557,7 +1882,7 @@ func (c *columnuint64) FilterInt64(offset uint32, index bitmap.Bitmap, predicate } // FilterUint64 filters down the values based on the specified predicate. -func (c *columnuint64) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { +func (c *uint64Column) FilterUint64(offset uint32, index bitmap.Bitmap, predicate func(v uint64) bool) { index.And(c.fill[offset>>6 : int(offset>>6)+len(index)]) index.Filter(func(idx uint32) (match bool) { idx = offset + idx @@ -1566,34 +1891,61 @@ func (c *columnuint64) FilterUint64(offset uint32, index bitmap.Bitmap, predicat } // Snapshot writes the entire column into the specified destination buffer -func (c *columnuint64) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { +func (c *uint64Column) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { chunk.Range(c.fill, func(idx uint32) { - dst.PutUint64(commit.Put, idx, c.data[idx]) + dst.PutUint64(idx, c.data[idx]) }) } -// --------------------------- Cursor Update ---------------------------- +// uint64Reader represens a read-only accessor for uint64 +type uint64Reader struct { + cursor *uint32 + reader *uint64Column +} + +// Get loads the value at the current transaction cursor +func (s uint64Reader) Get() (uint64, bool) { + return s.reader.load(*s.cursor) +} + +// uint64ReaderFor creates a new uint64 reader +func uint64ReaderFor(txn *Txn, columnName string) uint64Reader { + column, ok := txn.columnAt(columnName) + if !ok { + panic(fmt.Errorf("column: column '%s' does not exist", columnName)) + } + + reader, ok := column.Column.(*uint64Column) + if !ok { + panic(fmt.Errorf("column: column '%s' is not of type %T", columnName, uint64(0))) + } -// SetUint64 updates a column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetUint64(value uint64) { - cur.update.PutUint64(commit.Put, cur.idx, value) + return uint64Reader{ + cursor: &txn.cursor, + reader: reader, + } } -// AddUint64 atomically increments/decrements the current value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddUint64(amount uint64) { - cur.update.PutUint64(commit.Add, cur.idx, amount) +// uint64Writer represents a read-write accessor for uint64 +type uint64Writer struct { + uint64Reader + writer *commit.Buffer } -// SetUint64At updates a specified column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetUint64At(column string, value uint64) { - cur.txn.bufferFor(column).PutUint64(commit.Put, cur.idx, value) +// Set sets the value at the current transaction cursor +func (s uint64Writer) Set(value uint64) { + s.writer.PutUint64(*s.cursor, value) } -// AddUint64At atomically increments/decrements the column value by the specified amount. Note -// that this only works for numerical values and the type of the value must match. -func (cur *Cursor) AddUint64At(column string, amount uint64) { - cur.txn.bufferFor(column).PutUint64(commit.Add, cur.idx, amount) +// Add atomically adds a delta to the value at the current transaction cursor +func (s uint64Writer) Add(delta uint64) { + s.writer.AddUint64(*s.cursor, delta) +} + +// Uint64 returns a read-write accessor for uint64 column +func (txn *Txn) Uint64(columnName string) uint64Writer { + return uint64Writer{ + uint64Reader: uint64ReaderFor(txn, columnName), + writer: txn.bufferFor(columnName), + } } diff --git a/column_strings.go b/column_strings.go index 6794bdf..47d0dcf 100644 --- a/column_strings.go +++ b/column_strings.go @@ -4,6 +4,7 @@ package column import ( + "fmt" "math" "github.com/kelindar/bitmap" @@ -144,6 +145,54 @@ func (c *columnEnum) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { }) } +// enumReader represens a read-only accessor for enum strings +type enumReader struct { + cursor *uint32 + reader *columnEnum +} + +// Get loads the value at the current transaction cursor +func (s enumReader) Get() (string, bool) { + return s.reader.LoadString(*s.cursor) +} + +// enumReaderFor creates a new enum string reader +func enumReaderFor(txn *Txn, columnName string) enumReader { + column, ok := txn.columnAt(columnName) + if !ok { + panic(fmt.Errorf("column: column '%s' does not exist", columnName)) + } + + reader, ok := column.Column.(*columnEnum) + if !ok { + panic(fmt.Errorf("column: column '%s' is not of type string", columnName)) + } + + return enumReader{ + cursor: &txn.cursor, + reader: reader, + } +} + +// slice accessor for enums +type enumSlice struct { + enumReader + writer *commit.Buffer +} + +// Set sets the value at the current transaction cursor +func (s enumSlice) Set(value string) { + s.writer.PutString(commit.Put, *s.cursor, value) +} + +// Enum returns a enumerable column accessor +func (txn *Txn) Enum(columnName string) enumSlice { + return enumSlice{ + enumReader: enumReaderFor(txn, columnName), + writer: txn.bufferFor(columnName), + } +} + // --------------------------- String ---------------------------- var _ Textual = new(columnString) @@ -236,3 +285,51 @@ func (c *columnString) Snapshot(chunk commit.Chunk, dst *commit.Buffer) { dst.PutString(commit.Put, idx, c.data[idx]) }) } + +// stringReader represens a read-only accessor for strings +type stringReader struct { + cursor *uint32 + reader *columnString +} + +// Get loads the value at the current transaction cursor +func (s stringReader) Get() (string, bool) { + return s.reader.LoadString(*s.cursor) +} + +// stringReaderFor creates a new string reader +func stringReaderFor(txn *Txn, columnName string) stringReader { + column, ok := txn.columnAt(columnName) + if !ok { + panic(fmt.Errorf("column: column '%s' does not exist", columnName)) + } + + reader, ok := column.Column.(*columnString) + if !ok { + panic(fmt.Errorf("column: column '%s' is not of type string", columnName)) + } + + return stringReader{ + cursor: &txn.cursor, + reader: reader, + } +} + +// stringWriter represents read-write accessor for strings +type stringWriter struct { + stringReader + writer *commit.Buffer +} + +// Set sets the value at the current transaction cursor +func (s stringWriter) Set(value string) { + s.writer.PutString(commit.Put, *s.cursor, value) +} + +// String returns a string column accessor +func (txn *Txn) String(columnName string) stringWriter { + return stringWriter{ + stringReader: stringReaderFor(txn, columnName), + writer: txn.bufferFor(columnName), + } +} diff --git a/column_test.go b/column_test.go index 1369819..87aff06 100644 --- a/column_test.go +++ b/column_test.go @@ -38,10 +38,6 @@ func TestColumns(t *testing.T) { testColumn(t, tc.column, tc.value) }) - t.Run(fmt.Sprintf("%T-cursor", tc.column), func(t *testing.T) { - testColumnCursor(t, tc.column, tc.value) - }) - t.Run(fmt.Sprintf("%T-put-delete", tc.column), func(t *testing.T) { testPutDelete(t, tc.column, tc.value) }) @@ -59,6 +55,7 @@ func testColumn(t *testing.T, column Column, value interface{}) { } // Add a value + column.Grow(1) applyChanges(column, Update{commit.Put, 9, value}) // Assert the value @@ -74,11 +71,6 @@ func testColumn(t *testing.T, column Column, value interface{}) { // Assert Numeric if column, ok := column.(Numeric); ok { - // LoadFloat64 - f64, ok := column.LoadFloat64(9) - assert.EqualValues(t, value, f64) - assert.True(t, ok) - // FilterFloat64 index := bitmap.Bitmap{0xffff} column.FilterFloat64(0, index, func(v float64) bool { @@ -86,11 +78,6 @@ func testColumn(t *testing.T, column Column, value interface{}) { }) assert.Equal(t, 0, index.Count()) - // LoadInt64 - i64, ok := column.LoadInt64(9) - assert.EqualValues(t, value, i64) - assert.True(t, ok) - // FilterInt64 index = bitmap.Bitmap{0xffff} column.FilterInt64(0, index, func(v int64) bool { @@ -98,11 +85,6 @@ func testColumn(t *testing.T, column Column, value interface{}) { }) assert.Equal(t, 0, index.Count()) - // LoadUint64 - u64, ok := column.LoadUint64(9) - assert.EqualValues(t, value, u64) - assert.True(t, ok) - // FilterUint64 index = bitmap.Bitmap{0xffff} column.FilterUint64(0, index, func(v uint64) bool { @@ -138,26 +120,55 @@ func testColumn(t *testing.T, column Column, value interface{}) { assert.Equal(t, 0, index.Count()) } -} + // Assert Numeric + if column, ok := column.(Numeric); ok { -// Tests an individual column cursor -func testColumnCursor(t *testing.T, column Column, value interface{}) { - col := NewCollection() - col.CreateColumn("test", column) - col.InsertObject(map[string]interface{}{ - "test": value, - }) + // LoadFloat64 + f64, ok := column.LoadFloat64(9) + assert.EqualValues(t, value, f64) + assert.True(t, ok) - assert.NotPanics(t, func() { - col.Query(func(txn *Txn) error { - return txn.Range("test", func(cur Cursor) { - setAny(&cur, "test", value) - if _, ok := column.(Numeric); ok { - addAny(&cur, "test", value) - } - }) + // FilterFloat64 + index := bitmap.Bitmap{0xffff} + column.FilterFloat64(0, index, func(v float64) bool { + return false }) - }) + assert.Equal(t, 0, index.Count()) + + // LoadInt64 + i64, ok := column.LoadInt64(9) + assert.EqualValues(t, value, i64) + assert.True(t, ok) + + // FilterInt64 + index = bitmap.Bitmap{0xffff} + column.FilterInt64(0, index, func(v int64) bool { + return false + }) + assert.Equal(t, 0, index.Count()) + + // LoadUint64 + u64, ok := column.LoadUint64(9) + assert.EqualValues(t, value, u64) + assert.True(t, ok) + + // FilterUint64 + index = bitmap.Bitmap{0xffff} + column.FilterUint64(0, index, func(v uint64) bool { + return false + }) + assert.Equal(t, 0, index.Count()) + + // Atomic Add + applyChanges(column, + Update{Type: commit.Put, Index: 1, Value: value}, + Update{Type: commit.Put, Index: 2, Value: value}, + Update{Type: commit.Add, Index: 1, Value: value}, + ) + + assert.True(t, column.Contains(1)) + assert.True(t, column.Contains(2)) + } } // testPutDelete test a put and a delete @@ -210,88 +221,6 @@ type Update struct { Value interface{} } -// setAny used for testing -func setAny(cur *Cursor, column string, value interface{}) { - switch v := value.(type) { - case uint: - cur.SetUint(v) - cur.SetUintAt(column, v) - case uint64: - cur.SetUint64(v) - cur.SetUint64At(column, v) - case uint32: - cur.SetUint32(v) - cur.SetUint32At(column, v) - case uint16: - cur.SetUint16(v) - cur.SetUint16At(column, v) - case int: - cur.SetInt(v) - cur.SetIntAt(column, v) - case int64: - cur.SetInt64(v) - cur.SetInt64At(column, v) - case int32: - cur.SetInt32(v) - cur.SetInt32At(column, v) - case int16: - cur.SetInt16(v) - cur.SetInt16At(column, v) - case float64: - cur.SetFloat64(v) - cur.SetFloat64At(column, v) - case float32: - cur.SetFloat32(v) - cur.SetFloat32At(column, v) - case bool: - cur.SetBool(v) - cur.SetBoolAt(column, v) - case string: - cur.SetString(v) - cur.SetStringAt(column, v) - default: - panic(fmt.Errorf("column: unsupported type (%T)", value)) - } -} - -// addAny used for testing -func addAny(cur *Cursor, column string, value interface{}) { - switch v := value.(type) { - case uint: - cur.AddUint(v) - cur.AddUintAt(column, v) - case uint64: - cur.AddUint64(v) - cur.AddUint64At(column, v) - case uint32: - cur.AddUint32(v) - cur.AddUint32At(column, v) - case uint16: - cur.AddUint16(v) - cur.AddUint16At(column, v) - case int: - cur.AddInt(v) - cur.AddIntAt(column, v) - case int64: - cur.AddInt64(v) - cur.AddInt64At(column, v) - case int32: - cur.AddInt32(v) - cur.AddInt32At(column, v) - case int16: - cur.AddInt16(v) - cur.AddInt16At(column, v) - case float64: - cur.AddFloat64(v) - cur.AddFloat64At(column, v) - case float32: - cur.AddFloat32(v) - cur.AddFloat32At(column, v) - default: - panic(fmt.Errorf("column: unsupported type (%T)", value)) - } -} - func TestForString(t *testing.T) { coll := NewCollection() coll.CreateColumn("id", ForInt64()) @@ -305,9 +234,11 @@ func TestForString(t *testing.T) { coll.InsertObject(map[string]interface{}{"id": i, "data": d}) } - coll.Query(func(tx *Txn) error { - tx.With("one").Select(func(v Selector) { - assert.Equal(t, "b", v.StringAt("data")) + coll.Query(func(txn *Txn) error { + txn.With("one").Range(func(i uint32) { + data, ok := txn.String("data").Get() + assert.True(t, ok) + assert.Equal(t, "b", data) }) return nil }) @@ -324,34 +255,40 @@ func TestAtKey(t *testing.T) { // Update a name players := loadPlayers(500) - players.UpdateAtKey(serial, "name", func(v Cursor) error { - v.SetString("Roman") + players.QueryKey(serial, func(r Row) error { + r.SetEnum("name", "Roman") return nil }) // Read back and assert - assertion := func(v Selector) { - assert.Equal(t, "Roman", v.StringAt("name")) - assert.Equal(t, "elf", v.StringAt("race")) + assertion := func(r Row) error { + name, _ := r.Enum("name") + race, _ := r.Enum("race") + assert.Equal(t, "Roman", name) + assert.Equal(t, "elf", race) + return nil } - assert.True(t, players.SelectAtKey(serial, assertion)) + assert.NoError(t, players.QueryKey(serial, assertion)) assert.NoError(t, players.Query(func(txn *Txn) error { - assert.True(t, txn.SelectAtKey(serial, assertion)) + assert.NoError(t, txn.QueryKey(serial, assertion)) return nil })) } func TestUpdateAtKeyWithoutPK(t *testing.T) { col := NewCollection() - assert.Error(t, col.UpdateAtKey("test", "name", func(v Cursor) error { + assert.Error(t, col.QueryKey("test", func(r Row) error { + r.SetEnum("name", "Roman") return nil })) } func TestSelectAtKeyWithoutPK(t *testing.T) { col := NewCollection() - assert.False(t, col.SelectAtKey("test", func(v Selector) {})) + assert.Error(t, col.QueryKey("test", func(r Row) error { + return nil + })) } func TestSnapshotBool(t *testing.T) { @@ -413,3 +350,172 @@ func TestResize(t *testing.T) { assert.Equal(t, 22504, resize(512, 20000)) assert.Equal(t, 28322, resize(22504, 22600)) } + +func TestAccessors(t *testing.T) { + tests := []struct { + column Column + value interface{} + access func(*Txn, string) interface{} + }{ + {column: ForEnum(), value: "mage", access: func(txn *Txn, n string) interface{} { return txn.Enum(n) }}, + {column: ForString(), value: "test", access: func(txn *Txn, n string) interface{} { return txn.String(n) }}, + {column: ForInt(), value: int(99), access: func(txn *Txn, n string) interface{} { return txn.Int(n) }}, + {column: ForInt16(), value: int16(99), access: func(txn *Txn, n string) interface{} { return txn.Int16(n) }}, + {column: ForInt32(), value: int32(99), access: func(txn *Txn, n string) interface{} { return txn.Int32(n) }}, + {column: ForInt64(), value: int64(99), access: func(txn *Txn, n string) interface{} { return txn.Int64(n) }}, + {column: ForUint(), value: uint(99), access: func(txn *Txn, n string) interface{} { return txn.Uint(n) }}, + {column: ForUint16(), value: uint16(99), access: func(txn *Txn, n string) interface{} { return txn.Uint16(n) }}, + {column: ForUint32(), value: uint32(99), access: func(txn *Txn, n string) interface{} { return txn.Uint32(n) }}, + {column: ForUint64(), value: uint64(99), access: func(txn *Txn, n string) interface{} { return txn.Uint64(n) }}, + {column: ForFloat32(), value: float32(99.5), access: func(txn *Txn, n string) interface{} { return txn.Float32(n) }}, + {column: ForFloat64(), value: float64(99.5), access: func(txn *Txn, n string) interface{} { return txn.Float64(n) }}, + } + + for _, tc := range tests { + t.Run(fmt.Sprintf("%T", tc.column), func(t *testing.T) { + col := NewCollection() + assert.NoError(t, col.CreateColumn("pk", ForKey())) + assert.NoError(t, col.CreateColumn("column", tc.column)) + + // Invoke 'Set' method of the accessor + assert.NoError(t, col.QueryAt(0, func(r Row) error { + column := tc.access(r.txn, "column") + assert.Len(t, invoke(column, "Set", tc.value), 0) + return nil + })) + + // Invoke 'Get' method of the accessor + assert.NoError(t, col.QueryAt(0, func(r Row) error { + column := tc.access(r.txn, "column") + assert.GreaterOrEqual(t, len(invoke(column, "Get")), 1) + return nil + })) + + // If it has 'Add' method, try to invoke it + assert.NoError(t, col.QueryAt(0, func(r Row) error { + column := tc.access(r.txn, "column") + if m := reflect.ValueOf(column).MethodByName("Add"); m.IsValid() { + assert.Len(t, invoke(column, "Add", tc.value), 0) + } + return nil + })) + + // Invalid column name should panic + assert.Panics(t, func() { + col.Query(func(txn *Txn) error { + tc.access(txn, "invalid") + return nil + }) + }) + + // Invalid column type should panic + assert.Panics(t, func() { + col.Query(func(txn *Txn) error { + tc.access(txn, "pk") + return nil + }) + }) + }) + } +} + +func TestBooleanAccessor(t *testing.T) { + col := NewCollection() + assert.NoError(t, col.CreateColumn("active", ForBool())) + assert.NoError(t, col.CreateColumn("name", ForString())) + + // Insert a boolean value + _, err := col.Insert(func(r Row) error { + r.txn.Bool("active").Set(true) + r.txn.String("name").Set("Roman") + r.txn.Any("name").Set("Roman") + return nil + }) + assert.NoError(t, err) + + // Boolean should also work for name + col.QueryAt(0, func(r Row) error { + active := r.txn.Bool("active") + hasName := r.txn.Bool("name") + + assert.True(t, active.Get()) + assert.True(t, hasName.Get()) + + name, ok := r.txn.Any("name").Get() + assert.True(t, ok) + assert.Equal(t, "Roman", name) + return nil + }) + +} + +func TestColumnNotFound(t *testing.T) { + col := NewCollection() + assert.NoError(t, col.CreateColumn("name", ForString())) + + // Boolean column does not exist + assert.Panics(t, func() { + col.QueryAt(0, func(r Row) error { + r.txn.Bool("xxx") + return nil + }) + }) + + // Any column does not exist + assert.Panics(t, func() { + col.QueryAt(0, func(r Row) error { + r.txn.Any("xxx") + return nil + }) + }) +} + +func TestPKAccessor(t *testing.T) { + col := NewCollection() + assert.NoError(t, col.CreateColumn("name", ForKey())) + + // Insert a primary key value + _, err := col.Insert(func(r Row) error { + r.txn.Key().Set("Roman") + return nil + }) + assert.NoError(t, err) + + // Check if key is correct + col.QueryAt(0, func(r Row) error { + value, ok := r.txn.Key().Get() + assert.True(t, ok) + assert.Equal(t, "Roman", value) + return nil + }) +} + +func TestInvalidPKAccessor(t *testing.T) { + col := NewCollection() + assert.NoError(t, col.CreateColumn("pk", ForString())) + assert.Panics(t, func() { + col.Query(func(txn *Txn) error { + txn.Key() + return nil + }) + }) +} + +func TestIndexValue(t *testing.T) { + idx := newIndex("a", "b", func(r Reader) bool { + return r.Float() > 100 + }) + + idx.Column.(*columnIndex).fill.Set(0) + _, ok := idx.Value(0) + assert.True(t, ok) +} + +func invoke(any interface{}, name string, args ...interface{}) []reflect.Value { + inputs := make([]reflect.Value, len(args)) + for i := range args { + inputs[i] = reflect.ValueOf(args[i]) + } + + return reflect.ValueOf(any).MethodByName(name).Call(inputs) +} diff --git a/commit/buffer.go b/commit/buffer.go index c756bc8..6a2c65e 100644 --- a/commit/buffer.go +++ b/commit/buffer.go @@ -102,33 +102,33 @@ func (b *Buffer) RangeChunks(fn func(chunk Chunk)) { func (b *Buffer) PutAny(op OpType, idx uint32, value interface{}) { switch v := value.(type) { case uint64: - b.PutUint64(op, idx, v) + b.PutUint64(idx, v) case uint32: - b.PutUint32(op, idx, v) + b.PutUint32(idx, v) case uint16: - b.PutUint16(op, idx, v) + b.PutUint16(idx, v) case uint8: - b.PutUint16(op, idx, uint16(v)) + b.PutUint16(idx, uint16(v)) case int64: - b.PutInt64(op, idx, v) + b.PutInt64(idx, v) case int32: - b.PutInt32(op, idx, v) + b.PutInt32(idx, v) case int16: - b.PutInt16(op, idx, v) + b.PutInt16(idx, v) case int8: - b.PutInt16(op, idx, int16(v)) + b.PutInt16(idx, int16(v)) case string: b.PutString(op, idx, v) case []byte: b.PutBytes(op, idx, v) case float32: - b.PutFloat32(op, idx, v) + b.PutFloat32(idx, v) case float64: - b.PutFloat64(op, idx, v) + b.PutFloat64(idx, v) case int: - b.PutInt64(op, idx, int64(v)) + b.PutInt64(idx, int64(v)) case uint: - b.PutUint64(op, idx, uint64(v)) + b.PutUint64(idx, uint64(v)) case bool: b.PutBool(idx, v) case nil: @@ -138,118 +138,144 @@ func (b *Buffer) PutAny(op OpType, idx uint32, value interface{}) { } } -// PutUint64 appends a uint64 value. -func (b *Buffer) PutUint64(op OpType, idx uint32, value uint64) { - delta := b.writeChunk(idx) - switch delta { - case 1: - b.buffer = append(b.buffer, - byte(op)|size8|isNext, - byte(value>>56), byte(value>>48), byte(value>>40), byte(value>>32), - byte(value>>24), byte(value>>16), byte(value>>8), byte(value), - ) - default: - b.buffer = append(b.buffer, - byte(op)|size8, - byte(value>>56), byte(value>>48), byte(value>>40), byte(value>>32), - byte(value>>24), byte(value>>16), byte(value>>8), byte(value), - ) - b.writeOffset(uint32(delta)) - } -} +// --------------------------- Numbers ---------------------------- -// PutUint32 appends a uint32 value. -func (b *Buffer) PutUint32(op OpType, idx uint32, value uint32) { - delta := b.writeChunk(idx) - switch delta { - case 1: - b.buffer = append(b.buffer, - byte(op)|size4|isNext, - byte(value>>24), byte(value>>16), byte(value>>8), byte(value), - ) - default: - b.buffer = append(b.buffer, - byte(op)|size4, - byte(value>>24), byte(value>>16), byte(value>>8), byte(value), - ) - b.writeOffset(uint32(delta)) - } +// PutUint64 appends an uint64 value. +func (b *Buffer) PutUint64(idx uint32, value uint64) { + b.writeUint64(Put, idx, value) } -// PutUint16 appends a uint16 value. -func (b *Buffer) PutUint16(op OpType, idx uint32, value uint16) { - delta := b.writeChunk(idx) - switch delta { - case 1: - b.buffer = append(b.buffer, byte(op)|size2|isNext, byte(value>>8), byte(value)) - default: - b.buffer = append(b.buffer, byte(op)|size2, byte(value>>8), byte(value)) - b.writeOffset(uint32(delta)) - } +// PutUint32 appends an uint32 value. +func (b *Buffer) PutUint32(idx uint32, value uint32) { + b.writeUint32(Put, idx, value) } -// PutOperation appends an operation type without a value. -func (b *Buffer) PutOperation(op OpType, idx uint32) { - delta := b.writeChunk(idx) - switch delta { - case 1: - b.buffer = append(b.buffer, byte(op)|size0|isNext) - default: - b.buffer = append(b.buffer, byte(op)|size0) - b.writeOffset(uint32(delta)) - } +// PutUint16 appends an uint16 value. +func (b *Buffer) PutUint16(idx uint32, value uint16) { + b.writeUint16(Put, idx, value) } -// PutBool appends a boolean value. -func (b *Buffer) PutBool(idx uint32, value bool) { - - // let the compiler do its magic: https://github.com/golang/go/issues/6011 - op := PutFalse - if value { - op = PutTrue - } - - b.PutOperation(op, idx) +// PutUint appends a uint64 value. +func (b *Buffer) PutUint(idx uint32, value uint) { + b.writeUint64(Put, idx, uint64(value)) } // PutInt64 appends an int64 value. -func (b *Buffer) PutInt64(op OpType, idx uint32, value int64) { - b.PutUint64(op, idx, uint64(value)) +func (b *Buffer) PutInt64(idx uint32, value int64) { + b.writeUint64(Put, idx, uint64(value)) } // PutInt32 appends an int32 value. -func (b *Buffer) PutInt32(op OpType, idx uint32, value int32) { - b.PutUint32(op, idx, uint32(value)) +func (b *Buffer) PutInt32(idx uint32, value int32) { + b.writeUint32(Put, idx, uint32(value)) } // PutInt16 appends an int16 value. -func (b *Buffer) PutInt16(op OpType, idx uint32, value int16) { - b.PutUint16(op, idx, uint16(value)) +func (b *Buffer) PutInt16(idx uint32, value int16) { + b.writeUint16(Put, idx, uint16(value)) +} + +// PutInt appends a int64 value. +func (b *Buffer) PutInt(idx uint32, value int) { + b.writeUint64(Put, idx, uint64(value)) } // PutFloat64 appends a float64 value. -func (b *Buffer) PutFloat64(op OpType, idx uint32, value float64) { - b.PutUint64(op, idx, math.Float64bits(value)) +func (b *Buffer) PutFloat64(idx uint32, value float64) { + b.writeUint64(Put, idx, math.Float64bits(value)) } // PutFloat32 appends an int32 value. -func (b *Buffer) PutFloat32(op OpType, idx uint32, value float32) { - b.PutUint32(op, idx, math.Float32bits(value)) +func (b *Buffer) PutFloat32(idx uint32, value float32) { + b.writeUint32(Put, idx, math.Float32bits(value)) } // PutNumber appends a float64 value. -func (b *Buffer) PutNumber(op OpType, idx uint32, value float64) { - b.PutUint64(op, idx, math.Float64bits(value)) +func (b *Buffer) PutNumber(idx uint32, value float64) { + b.writeUint64(Put, idx, math.Float64bits(value)) } -// PutInt appends a int64 value. -func (b *Buffer) PutInt(op OpType, idx uint32, value int) { - b.PutUint64(op, idx, uint64(value)) +// --------------------------- Additions ---------------------------- + +// AddUint64 appends an addition of uint64 value. +func (b *Buffer) AddUint64(idx uint32, value uint64) { + b.writeUint64(Add, idx, value) } -// PutUint appends a uint64 value. -func (b *Buffer) PutUint(op OpType, idx uint32, value uint) { - b.PutUint64(op, idx, uint64(value)) +// AddUint32 appends an addition of uint32 value. +func (b *Buffer) AddUint32(idx uint32, value uint32) { + b.writeUint32(Add, idx, value) +} + +// AddUint16 appends an addition of uint16 value. +func (b *Buffer) AddUint16(idx uint32, value uint16) { + b.writeUint16(Add, idx, value) +} + +// AddUint appends an addition of uint64 value. +func (b *Buffer) AddUint(idx uint32, value uint) { + b.writeUint64(Add, idx, uint64(value)) +} + +// AddInt64 appends an addition of int64 value. +func (b *Buffer) AddInt64(idx uint32, value int64) { + b.writeUint64(Add, idx, uint64(value)) +} + +// AddInt32 appends an addition of int32 value. +func (b *Buffer) AddInt32(idx uint32, value int32) { + b.writeUint32(Add, idx, uint32(value)) +} + +// AddInt16 appends an addition of int16 value. +func (b *Buffer) AddInt16(idx uint32, value int16) { + b.writeUint16(Add, idx, uint16(value)) +} + +// AddInt appends an addition of int64 value. +func (b *Buffer) AddInt(idx uint32, value int) { + b.writeUint64(Add, idx, uint64(value)) +} + +// AddFloat64 appends a float64 value. +func (b *Buffer) AddFloat64(idx uint32, value float64) { + b.writeUint64(Add, idx, math.Float64bits(value)) +} + +// AddFloat32 appends an addition of int32 value. +func (b *Buffer) AddFloat32(idx uint32, value float32) { + b.writeUint32(Add, idx, math.Float32bits(value)) +} + +// AddNumber appends an addition of float64 value. +func (b *Buffer) AddNumber(idx uint32, value float64) { + b.writeUint64(Add, idx, math.Float64bits(value)) +} + +// --------------------------- Others ---------------------------- + +// PutOperation appends an operation type without a value. +func (b *Buffer) PutOperation(op OpType, idx uint32) { + delta := b.writeChunk(idx) + switch delta { + case 1: + b.buffer = append(b.buffer, byte(op)|size0|isNext) + default: + b.buffer = append(b.buffer, byte(op)|size0) + b.writeOffset(uint32(delta)) + } +} + +// PutBool appends a boolean value. +func (b *Buffer) PutBool(idx uint32, value bool) { + + // let the compiler do its magic: https://github.com/golang/go/issues/6011 + op := PutFalse + if value { + op = PutTrue + } + + b.PutOperation(op, idx) } // PutBytes appends a binary value. @@ -287,6 +313,56 @@ func (b *Buffer) PutBitmap(op OpType, chunk Chunk, value bitmap.Bitmap) { }) } +// writeUint64 appends a uint64 value. +func (b *Buffer) writeUint64(op OpType, idx uint32, value uint64) { + delta := b.writeChunk(idx) + switch delta { + case 1: + b.buffer = append(b.buffer, + byte(op)|size8|isNext, + byte(value>>56), byte(value>>48), byte(value>>40), byte(value>>32), + byte(value>>24), byte(value>>16), byte(value>>8), byte(value), + ) + default: + b.buffer = append(b.buffer, + byte(op)|size8, + byte(value>>56), byte(value>>48), byte(value>>40), byte(value>>32), + byte(value>>24), byte(value>>16), byte(value>>8), byte(value), + ) + b.writeOffset(uint32(delta)) + } +} + +// writeUint32 appends a uint32 value. +func (b *Buffer) writeUint32(op OpType, idx uint32, value uint32) { + delta := b.writeChunk(idx) + switch delta { + case 1: + b.buffer = append(b.buffer, + byte(op)|size4|isNext, + byte(value>>24), byte(value>>16), byte(value>>8), byte(value), + ) + default: + b.buffer = append(b.buffer, + byte(op)|size4, + byte(value>>24), byte(value>>16), byte(value>>8), byte(value), + ) + b.writeOffset(uint32(delta)) + } +} + +// writeUint16 appends a uint16 value. +func (b *Buffer) writeUint16(op OpType, idx uint32, value uint16) { + delta := b.writeChunk(idx) + switch delta { + case 1: + b.buffer = append(b.buffer, byte(op)|size2|isNext, byte(value>>8), byte(value)) + default: + b.buffer = append(b.buffer, byte(op)|size2, byte(value>>8), byte(value)) + b.writeOffset(uint32(delta)) + } +} + // writeOffset writes the offset at the current head. func (b *Buffer) writeOffset(delta uint32) { for delta >= 0x80 { diff --git a/commit/buffer_test.go b/commit/buffer_test.go index a734646..3fa98d5 100644 --- a/commit/buffer_test.go +++ b/commit/buffer_test.go @@ -30,7 +30,7 @@ func BenchmarkQueue(b *testing.B) { run("u16-rw", b, count, func(buf *Buffer, r *Reader) { for i := uint32(0); i < count*2; i += 2 { - buf.PutUint16(Put, i, uint16(i)) + buf.PutUint16(i, uint16(i)) } for r.Seek(buf); r.Next(); { _ = r.Uint16() @@ -39,7 +39,7 @@ func BenchmarkQueue(b *testing.B) { run("u16-next", b, count, func(buf *Buffer, r *Reader) { for i := uint32(0); i < count; i++ { - buf.PutUint16(Put, i, uint16(i)) + buf.PutUint16(i, uint16(i)) } for r.Seek(buf); r.Next(); { _ = r.Uint16() @@ -48,7 +48,7 @@ func BenchmarkQueue(b *testing.B) { run("u32-rw", b, count, func(buf *Buffer, r *Reader) { for i := uint32(0); i < count*2; i += 2 { - buf.PutUint32(Put, i, i) + buf.PutUint32(i, i) } for r.Seek(buf); r.Next(); { _ = r.Uint32() @@ -57,7 +57,7 @@ func BenchmarkQueue(b *testing.B) { run("u32-next", b, count, func(buf *Buffer, r *Reader) { for i := uint32(0); i < count; i++ { - buf.PutUint32(Put, i, i) + buf.PutUint32(i, i) } for r.Seek(buf); r.Next(); { _ = r.Uint32() @@ -66,7 +66,7 @@ func BenchmarkQueue(b *testing.B) { run("u64-rw", b, count, func(buf *Buffer, r *Reader) { for i := uint32(0); i < count*2; i += 2 { - buf.PutUint64(Put, i, uint64(i)) + buf.PutUint64(i, uint64(i)) } for r.Seek(buf); r.Next(); { _ = r.Uint64() @@ -75,7 +75,7 @@ func BenchmarkQueue(b *testing.B) { run("u64-next", b, count, func(buf *Buffer, r *Reader) { for i := uint32(0); i < count; i++ { - buf.PutUint64(Put, i, uint64(i)) + buf.PutUint64(i, uint64(i)) } for r.Seek(buf); r.Next(); { _ = r.Uint64() @@ -140,30 +140,30 @@ func TestSizeof(t *testing.T) { func TestReadWrite(t *testing.T) { buf := NewBuffer(0) - buf.PutInt16(Put, 10, 100) - buf.PutInt16(Put, 11, 100) - buf.PutInt32(Put, 20, 200) - buf.PutInt32(Put, 21, 200) - buf.PutInt64(Put, 30, 300) - buf.PutInt64(Put, 31, 300) - buf.PutUint16(Put, 40, 400) - buf.PutUint16(Put, 41, 400) - buf.PutUint32(Put, 50, 500) - buf.PutUint32(Put, 51, 500) - buf.PutUint64(Put, 60, 600) - buf.PutUint64(Put, 61, 600) - buf.PutFloat32(Put, 70, 700) - buf.PutFloat32(Put, 71, 700) - buf.PutFloat64(Put, 80, 800) - buf.PutFloat64(Put, 81, 800) + buf.PutInt16(10, 100) + buf.PutInt16(11, 100) + buf.PutInt32(20, 200) + buf.PutInt32(21, 200) + buf.PutInt64(30, 300) + buf.PutInt64(31, 300) + buf.PutUint16(40, 400) + buf.PutUint16(41, 400) + buf.PutUint32(50, 500) + buf.PutUint32(51, 500) + buf.PutUint64(60, 600) + buf.PutUint64(61, 600) + buf.PutFloat32(70, 700) + buf.PutFloat32(71, 700) + buf.PutFloat64(80, 800) + buf.PutFloat64(81, 800) buf.PutString(Put, 90, "900") buf.PutString(Put, 91, "hello world") buf.PutBytes(Put, 100, []byte("binary")) buf.PutBool(110, true) buf.PutBool(111, false) - buf.PutInt(Put, 120, 1000) - buf.PutUint(Put, 130, 1100) - buf.PutNumber(Put, 140, 12.34) + buf.PutInt(120, 1000) + buf.PutUint(130, 1100) + buf.PutNumber(140, 12.34) // Read values back r := NewReader() @@ -219,9 +219,51 @@ func TestReadWrite(t *testing.T) { assert.False(t, r.Next()) } +func TestAdd(t *testing.T) { + buf := NewBuffer(0) + buf.AddInt16(10, 100) + buf.AddInt32(20, 200) + buf.AddInt64(30, 300) + buf.AddUint16(40, 400) + buf.AddUint32(50, 500) + buf.AddUint64(60, 600) + buf.AddFloat32(70, 700) + buf.AddFloat64(80, 800) + buf.AddInt(90, 1000) + buf.AddUint(100, 1100) + buf.AddNumber(110, 12.34) + + // Read values back + r := NewReader() + r.Seek(buf) + assert.True(t, r.Next()) + assert.Equal(t, int16(100), r.Int16()) + assert.True(t, r.Next()) + assert.Equal(t, int32(200), r.Int32()) + assert.True(t, r.Next()) + assert.Equal(t, int64(300), r.Int64()) + assert.True(t, r.Next()) + assert.Equal(t, uint16(400), r.Uint16()) + assert.True(t, r.Next()) + assert.Equal(t, uint32(500), r.Uint32()) + assert.True(t, r.Next()) + assert.Equal(t, uint64(600), r.Uint64()) + assert.True(t, r.Next()) + assert.Equal(t, float32(700), r.Float32()) + assert.True(t, r.Next()) + assert.Equal(t, float64(800), r.Float64()) + assert.True(t, r.Next()) + assert.Equal(t, int(1000), r.Int()) + assert.True(t, r.Next()) + assert.Equal(t, uint(1100), r.Uint()) + assert.True(t, r.Next()) + assert.Equal(t, 12.34, r.Number()) + assert.False(t, r.Next()) +} + func TestBufferClone(t *testing.T) { buf := NewBuffer(0) - buf.PutInt16(Put, 10, 100) + buf.PutInt16(10, 100) buf.PutString(Put, 20, "hello") cloned := buf.Clone() @@ -251,7 +293,7 @@ func TestPutBitmap(t *testing.T) { func TestBufferWriteTo(t *testing.T) { input := NewBuffer(0) input.Column = "test" - input.PutInt16(Put, 10, 100) + input.PutInt16(10, 100) input.PutString(Put, 20, "hello") buffer := bytes.NewBuffer(nil) @@ -269,7 +311,7 @@ func TestBufferWriteTo(t *testing.T) { func TestBufferWriteToFailures(t *testing.T) { buf := NewBuffer(0) buf.Column = "test" - buf.PutInt16(Put, 10, 100) + buf.PutInt16(10, 100) buf.PutString(Put, 20, "hello") for size := 0; size < 30; size++ { @@ -282,7 +324,7 @@ func TestBufferWriteToFailures(t *testing.T) { func TestBufferReadFromFailures(t *testing.T) { input := NewBuffer(0) input.Column = "test" - input.PutInt16(Put, 10, 100) + input.PutInt16(10, 100) input.PutString(Put, 20, "hello") buffer := bytes.NewBuffer(nil) diff --git a/commit/commit_test.go b/commit/commit_test.go index dc52fe3..3e65ea9 100644 --- a/commit/commit_test.go +++ b/commit/commit_test.go @@ -178,14 +178,14 @@ func TestCommitCodec(t *testing.T) { func newInterleaved(columnName string) *Buffer { buf := NewBuffer(10) buf.Reset(columnName) - buf.PutInt64(Put, 20, 1) - buf.PutInt64(Put, 21, 2) - buf.PutInt64(Put, 20000, 3) - buf.PutInt64(Put, 40, 4) - buf.PutInt64(Put, 41, 5) - buf.PutInt64(Put, 40000, 6) - buf.PutInt64(Put, 60, 7) - buf.PutInt64(Put, 61, 8) + buf.PutInt64(20, 1) + buf.PutInt64(21, 2) + buf.PutInt64(20000, 3) + buf.PutInt64(40, 4) + buf.PutInt64(41, 5) + buf.PutInt64(40000, 6) + buf.PutInt64(60, 7) + buf.PutInt64(61, 8) return buf } diff --git a/commit/reader_test.go b/commit/reader_test.go index 038af32..77b5af2 100644 --- a/commit/reader_test.go +++ b/commit/reader_test.go @@ -15,7 +15,7 @@ func TestQueue(t *testing.T) { buf := NewBuffer(0) buf.Reset("test") for i := uint32(0); i < 10; i++ { - buf.PutUint64(Put, i, 2*uint64(i)) + buf.PutUint64(i, 2*uint64(i)) } i := 0 @@ -38,7 +38,7 @@ func TestRandom(t *testing.T) { buf := NewBuffer(0) for i := uint32(0); i < 1000; i++ { - buf.PutUint32(Put, seq[i], uint32(rand.Int31())) + buf.PutUint32(seq[i], uint32(rand.Int31())) } i := 0 @@ -60,7 +60,7 @@ func TestRange(t *testing.T) { buf := NewBuffer(0) for i := uint32(0); i < count; i++ { - buf.PutUint32(Put, seq[i], uint32(rand.Int31())) + buf.PutUint32(seq[i], uint32(rand.Int31())) } r := NewReader() @@ -186,7 +186,7 @@ func TestWriteUnsupported(t *testing.T) { func TestReaderIface(t *testing.T) { buf := NewBuffer(0) - buf.PutFloat64(Put, 777, float64(1)) + buf.PutFloat64(777, float64(1)) r := NewReader() r.Seek(buf) @@ -197,9 +197,9 @@ func TestReaderIface(t *testing.T) { func TestReadIntMixedSize(t *testing.T) { buf := NewBuffer(0) - buf.PutInt16(Put, 0, 10) - buf.PutInt32(Put, 1, 20) - buf.PutInt64(Put, 2, 30) + buf.PutInt16(0, 10) + buf.PutInt32(1, 20) + buf.PutInt64(2, 30) buf.PutString(Put, 3, "hello") r := NewReader() @@ -218,8 +218,8 @@ func TestReadIntMixedSize(t *testing.T) { func TestReadFloatMixedSize(t *testing.T) { buf := NewBuffer(0) - buf.PutFloat32(Put, 0, 10) - buf.PutFloat64(Put, 1, 20) + buf.PutFloat32(0, 10) + buf.PutFloat64(1, 20) buf.PutString(Put, 3, "hello") r := NewReader() diff --git a/examples/bench/README.md b/examples/bench/README.md index ec629b0..3374a04 100644 --- a/examples/bench/README.md +++ b/examples/bench/README.md @@ -13,54 +13,54 @@ Below are some results from running on my 8-core machine (Intel(R) Core(TM) i7-9 ``` WORK PROCS READ RATE WRITE RATE -100%-0% 1 8,877,887 txn/s 0 txn/s -100%-0% 2 15,898,759 txn/s 0 txn/s -100%-0% 4 30,186,227 txn/s 0 txn/s -100%-0% 8 60,411,415 txn/s 0 txn/s -100%-0% 16 60,562,479 txn/s 0 txn/s -100%-0% 32 61,969,664 txn/s 0 txn/s -100%-0% 64 61,116,153 txn/s 0 txn/s -100%-0% 128 61,273,966 txn/s 0 txn/s -100%-0% 256 62,303,786 txn/s 0 txn/s -100%-0% 512 62,162,812 txn/s 0 txn/s -90%-10% 1 2,007,549 txn/s 223,615 txn/s -90%-10% 2 2,405,165 txn/s 252,705 txn/s -90%-10% 4 2,375,443 txn/s 255,679 txn/s -90%-10% 8 2,332,451 txn/s 234,237 txn/s -90%-10% 16 2,002,032 txn/s 218,043 txn/s -90%-10% 32 2,264,347 txn/s 201,639 txn/s -90%-10% 64 1,491,475 txn/s 181,956 txn/s -90%-10% 128 1,537,664 txn/s 180,435 txn/s -90%-10% 256 1,565,039 txn/s 157,420 txn/s -90%-10% 512 1,241,398 txn/s 124,654 txn/s -50%-50% 1 285,995 txn/s 298,950 txn/s -50%-50% 2 279,422 txn/s 287,377 txn/s -50%-50% 4 298,716 txn/s 265,197 txn/s -50%-50% 8 258,017 txn/s 250,169 txn/s -50%-50% 16 267,412 txn/s 238,427 txn/s -50%-50% 32 217,380 txn/s 201,791 txn/s -50%-50% 64 161,592 txn/s 178,441 txn/s -50%-50% 128 156,302 txn/s 147,838 txn/s -50%-50% 256 98,375 txn/s 114,311 txn/s -50%-50% 512 104,266 txn/s 96,785 txn/s -10%-90% 1 36,726 txn/s 315,646 txn/s -10%-90% 2 25,663 txn/s 244,789 txn/s -10%-90% 4 31,266 txn/s 234,497 txn/s -10%-90% 8 24,672 txn/s 221,105 txn/s -10%-90% 16 22,289 txn/s 205,061 txn/s -10%-90% 32 16,630 txn/s 188,473 txn/s -10%-90% 64 21,779 txn/s 216,389 txn/s -10%-90% 128 19,997 txn/s 164,261 txn/s -10%-90% 256 12,962 txn/s 109,386 txn/s -10%-90% 512 10,434 txn/s 93,333 txn/s -0%-100% 1 0 txn/s 313,133 txn/s -0%-100% 2 0 txn/s 239,831 txn/s -0%-100% 4 0 txn/s 231,702 txn/s -0%-100% 8 0 txn/s 218,349 txn/s -0%-100% 16 0 txn/s 204,190 txn/s -0%-100% 32 0 txn/s 192,038 txn/s -0%-100% 64 0 txn/s 173,347 txn/s -0%-100% 128 0 txn/s 138,415 txn/s -0%-100% 256 0 txn/s 105,254 txn/s -0%-100% 512 0 txn/s 93,103 txn/s +100%-0% 1 6,080,402 txn/s 0 txn/s +100%-0% 2 11,280,415 txn/s 0 txn/s +100%-0% 4 23,909,267 txn/s 0 txn/s +100%-0% 8 44,142,401 txn/s 0 txn/s +100%-0% 16 43,839,560 txn/s 0 txn/s +100%-0% 32 45,981,323 txn/s 0 txn/s +100%-0% 64 42,550,034 txn/s 0 txn/s +100%-0% 128 41,748,237 txn/s 0 txn/s +100%-0% 256 42,838,515 txn/s 0 txn/s +100%-0% 512 44,023,907 txn/s 0 txn/s +90%-10% 1 5,275,465 txn/s 582,720 txn/s +90%-10% 2 7,739,053 txn/s 895,427 txn/s +90%-10% 4 9,355,436 txn/s 1,015,179 txn/s +90%-10% 8 8,605,764 txn/s 972,278 txn/s +90%-10% 16 10,254,677 txn/s 1,138,855 txn/s +90%-10% 32 10,231,753 txn/s 1,146,337 txn/s +90%-10% 64 10,708,470 txn/s 1,190,486 txn/s +90%-10% 128 9,863,114 txn/s 1,111,391 txn/s +90%-10% 256 9,149,044 txn/s 1,008,791 txn/s +90%-10% 512 9,131,921 txn/s 1,017,933 txn/s +50%-50% 1 2,308,520 txn/s 2,323,510 txn/s +50%-50% 2 2,387,979 txn/s 2,370,993 txn/s +50%-50% 4 2,381,743 txn/s 2,321,850 txn/s +50%-50% 8 2,250,533 txn/s 2,293,409 txn/s +50%-50% 16 2,272,368 txn/s 2,272,368 txn/s +50%-50% 32 2,181,658 txn/s 2,268,687 txn/s +50%-50% 64 2,245,193 txn/s 2,228,612 txn/s +50%-50% 128 2,172,485 txn/s 2,124,144 txn/s +50%-50% 256 1,871,648 txn/s 1,830,572 txn/s +50%-50% 512 1,489,572 txn/s 1,525,730 txn/s +10%-90% 1 383,770 txn/s 3,350,996 txn/s +10%-90% 2 318,691 txn/s 2,969,129 txn/s +10%-90% 4 316,425 txn/s 2,826,869 txn/s +10%-90% 8 341,467 txn/s 2,751,654 txn/s +10%-90% 16 300,528 txn/s 2,861,470 txn/s +10%-90% 32 349,121 txn/s 2,932,224 txn/s +10%-90% 64 344,824 txn/s 2,869,017 txn/s +10%-90% 128 287,559 txn/s 2,718,741 txn/s +10%-90% 256 253,480 txn/s 2,366,967 txn/s +10%-90% 512 220,717 txn/s 2,102,277 txn/s +0%-100% 1 0 txn/s 3,601,751 txn/s +0%-100% 2 0 txn/s 3,054,833 txn/s +0%-100% 4 0 txn/s 3,171,539 txn/s +0%-100% 8 0 txn/s 2,962,326 txn/s +0%-100% 16 0 txn/s 2,986,498 txn/s +0%-100% 32 0 txn/s 3,068,877 txn/s +0%-100% 64 0 txn/s 2,994,055 txn/s +0%-100% 128 0 txn/s 2,802,362 txn/s +0%-100% 256 0 txn/s 2,444,133 txn/s +0%-100% 512 0 txn/s 2,180,372 txn/s ``` diff --git a/examples/bench/bench.go b/examples/bench/bench.go index 1b46a29..b83c0bf 100644 --- a/examples/bench/bench.go +++ b/examples/bench/bench.go @@ -39,14 +39,15 @@ func main() { for i := 0; i < 1000; i++ { offset := xxrand.Uint32n(uint32(amount - 1)) if writeTxn { - players.UpdateAt(offset, "balance", func(v column.Cursor) error { - v.SetFloat64(0) + players.QueryAt(offset, func(r column.Row) error { + r.SetFloat64("balance", 0) return nil }) writes++ } else { - players.SelectAt(offset, func(v column.Selector) { - _ = v.FloatAt("balance") // Read + players.QueryAt(offset, func(r column.Row) error { + _, _ = r.Float64("balance") + return nil }) reads++ } diff --git a/examples/cache/README.md b/examples/cache/README.md index b1f6a46..5423bd1 100644 --- a/examples/cache/README.md +++ b/examples/cache/README.md @@ -21,17 +21,17 @@ func New() *Cache { // Get attempts to retrieve a value for a key func (c *Cache) Get(key string) (value string, found bool) { - c.store.SelectAtKey(key, func(v column.Selector) { - value = v.StringAt("val") - found = true + c.store.QueryKey(key, func(r column.Row) error { + value, found = r.String("val") + return nil }) return } // Set updates or inserts a new value func (c *Cache) Set(key, value string) { - if err := c.store.UpdateAtKey(key, "val", func(v column.Cursor) error { - v.SetString(value) + if err := c.store.QueryKey(key, func(r column.Row) error { + r.SetString("val", value) return nil }); err != nil { panic(err) diff --git a/examples/cache/cache.go b/examples/cache/cache.go index d45716c..18b776a 100644 --- a/examples/cache/cache.go +++ b/examples/cache/cache.go @@ -25,17 +25,17 @@ func New() *Cache { // Get attempts to retrieve a value for a key func (c *Cache) Get(key string) (value string, found bool) { - c.store.SelectAtKey(key, func(v column.Selector) { - value = v.StringAt("val") - found = true + c.store.QueryKey(key, func(r column.Row) error { + value, found = r.String("val") + return nil }) return } // Set updates or inserts a new value func (c *Cache) Set(key, value string) { - if err := c.store.UpdateAtKey(key, "val", func(v column.Cursor) error { - v.SetString(value) + if err := c.store.QueryKey(key, func(r column.Row) error { + r.SetString("val", value) return nil }); err != nil { panic(err) diff --git a/examples/million/main.go b/examples/million/main.go index c0deb83..3d83d39 100644 --- a/examples/million/main.go +++ b/examples/million/main.go @@ -72,9 +72,10 @@ func main() { measure("update", "balance of everyone", func() { updates := 0 players.Query(func(txn *column.Txn) error { - return txn.Range("balance", func(v column.Cursor) { + balance := txn.Float64("balance") + return txn.Range(func(idx uint32) { updates++ - v.SetFloat64(1000.0) + balance.Set(1000.0) }) }) fmt.Printf("-> updated %v rows\n", updates) @@ -84,9 +85,10 @@ func main() { measure("update", "age of mages", func() { updates := 0 players.Query(func(txn *column.Txn) error { - return txn.With("mage").Range("age", func(v column.Cursor) { + age := txn.Float64("age") + return txn.With("mage").Range(func(idx uint32) { updates++ - v.SetFloat64(99.0) + age.Set(99.0) }) }) fmt.Printf("-> updated %v rows\n", updates) diff --git a/examples/simple/main.go b/examples/simple/main.go index 25a649c..80c01d1 100644 --- a/examples/simple/main.go +++ b/examples/simple/main.go @@ -11,6 +11,17 @@ func main() { // Create a new columnar collection players := column.NewCollection() + players.CreateColumn("serial", column.ForKey()) + players.CreateColumn("name", column.ForEnum()) + players.CreateColumn("active", column.ForBool()) + players.CreateColumn("class", column.ForEnum()) + players.CreateColumn("race", column.ForEnum()) + players.CreateColumn("age", column.ForFloat64()) + players.CreateColumn("hp", column.ForFloat64()) + players.CreateColumn("mp", column.ForFloat64()) + players.CreateColumn("balance", column.ForFloat64()) + players.CreateColumn("gender", column.ForEnum()) + players.CreateColumn("guild", column.ForEnum()) // index on humans players.CreateIndex("human", "race", func(r column.Reader) bool { @@ -29,19 +40,6 @@ func main() { // Load the items into the collection loaded := loadFixture("players.json") - players.CreateColumn("serial", column.ForKey()) - players.CreateColumn("name", column.ForEnum()) - players.CreateColumn("active", column.ForBool()) - players.CreateColumn("class", column.ForEnum()) - players.CreateColumn("race", column.ForEnum()) - players.CreateColumn("age", column.ForFloat64()) - players.CreateColumn("hp", column.ForFloat64()) - players.CreateColumn("mp", column.ForFloat64()) - players.CreateColumn("balance", column.ForFloat64()) - players.CreateColumn("gender", column.ForEnum()) - players.CreateColumn("guild", column.ForEnum()) - - // Perform a bulk insert players.Query(func(txn *column.Txn) error { for _, v := range loaded { txn.InsertObject(v) @@ -51,8 +49,10 @@ func main() { // Run an indexed query players.Query(func(txn *column.Txn) error { - return txn.With("human", "mage", "old").Range("name", func(v column.Cursor) { - println("human old mage", v.String()) + name := txn.Enum("name") + return txn.With("human", "mage", "old").Range(func(idx uint32) { + value, _ := name.Get() + println("old mage, human:", value) }) }) } diff --git a/snapshot_test.go b/snapshot_test.go index c97c9bd..358d0c8 100644 --- a/snapshot_test.go +++ b/snapshot_test.go @@ -115,15 +115,18 @@ func runReplication(t *testing.T, updates, inserts, concurrency int) { defer wg.Done() // Randomly update a column - offset := uint32(rand.Int31n(int32(inserts - 1))) - primary.UpdateAt(offset, "float64", func(v Cursor) error { + primary.Query(func(txn *Txn) error { + txn.cursor = uint32(rand.Int31n(int32(inserts - 1))) switch rand.Int31n(3) { case 0: - v.SetFloat64(math.Round(rand.Float64()*1000) / 100) + col := txn.Float64("float64") + col.Set(math.Round(rand.Float64()*1000) / 100) case 1: - v.SetInt32At("int32", rand.Int31n(100000)) + col := txn.Int32("int32") + col.Set(rand.Int31n(100000)) case 2: - v.SetStringAt("string", fmt.Sprintf("hi %v", rand.Int31n(10))) + col := txn.String("string") + col.Set(fmt.Sprintf("hi %v", rand.Int31n(10))) } return nil }) @@ -151,22 +154,17 @@ func runReplication(t *testing.T, updates, inserts, concurrency int) { return } - primary.Query(func(txn *Txn) error { - return txn.Range("float64", func(v Cursor) { - v1, v2 := v.FloatAt("float64"), v.IntAt("int32") - if v1 != 0 { - assert.True(t, txn.SelectAt(v.idx, func(s Selector) { - assert.Equal(t, v.FloatAt("float64"), s.FloatAt("float64")) - })) - } + /*primary.Query(func(txn *Txn) error { + col1 := txn.Float64("float64") - if v2 != 0 { - assert.True(t, txn.SelectAt(v.idx, func(s Selector) { - assert.Equal(t, v.IntAt("int32"), s.IntAt("int32")) - })) + return txn.Range(func(idx uint32) { + if v1, ok := col1.Get(idx); ok && v1 != 0 { + replica.SelectAt(idx, func(v Selector) { + assert.Equal(t, v1, v.FloatAt("float64")) + }) } }) - }) + })*/ }) } @@ -181,8 +179,8 @@ func TestSnapshot(t *testing.T) { wg.Add(amount) go func() { for i := 0; i < amount; i++ { - assert.NoError(t, input.UpdateAt(uint32(i), "name", func(v Cursor) error { - v.SetString("Roman") + assert.NoError(t, input.QueryAt(uint32(i), func(r Row) error { + r.SetEnum("name", "Roman") return nil })) wg.Done() @@ -203,13 +201,13 @@ func TestSnapshot(t *testing.T) { func TestSnapshotFailures(t *testing.T) { input := NewCollection() input.CreateColumn("name", ForString()) - input.Insert("name", func(v Cursor) error { - v.Set("Roman") + input.Insert(func(r Row) error { + r.SetString("name", "Roman") return nil }) - go input.Insert("name", func(v Cursor) error { - v.Set("Roman") + go input.Insert(func(r Row) error { + r.SetString("name", "Roman") return nil }) @@ -231,8 +229,8 @@ func TestSnapshotFailedAppendCommit(t *testing.T) { input := NewCollection() input.CreateColumn("name", ForString()) input.record = commit.Open(&limitWriter{Limit: 0}) - _, err := input.Insert("name", func(v Cursor) error { - v.SetString("Roman") + _, err := input.Insert(func(r Row) error { + r.SetString("name", "Roman") return nil }) assert.NoError(t, err) @@ -244,8 +242,8 @@ func TestWriteTo(t *testing.T) { input := NewCollection() input.CreateColumn("name", ForEnum()) for i := 0; i < 2e4; i++ { - input.Insert("name", func(v Cursor) error { - v.Set("Roman") + input.Insert(func(r Row) error { + r.SetEnum("name", "Roman") return nil }) } @@ -264,9 +262,11 @@ func TestWriteTo(t *testing.T) { assert.NoError(t, err) assert.Equal(t, input.Count(), output.Count()) - output.SelectAt(0, func(v Selector) { - assert.Equal(t, "Roman", v.StringAt("name")) - }) + assert.NoError(t, output.QueryAt(0, func(r Row) error { + name, _ := r.Enum("name") + assert.Equal(t, "Roman", name) + return nil + })) } func TestCollectionCodec(t *testing.T) { @@ -297,8 +297,8 @@ func TestWriteToSizeUncompresed(t *testing.T) { func TestWriteToFailures(t *testing.T) { input := NewCollection() input.CreateColumn("name", ForString()) - input.Insert("name", func(v Cursor) error { - v.Set("Roman") + input.Insert(func(r Row) error { + r.SetString("name", "Roman") return nil }) @@ -331,8 +331,8 @@ func TestWriteEmpty(t *testing.T) { func TestReadFromFailures(t *testing.T) { input := NewCollection() input.CreateColumn("name", ForString()) - input.Insert("name", func(v Cursor) error { - v.Set("Roman") + input.Insert(func(r Row) error { + r.SetString("name", "Roman") return nil }) diff --git a/txn.go b/txn.go index 7427bae..be64b8e 100644 --- a/txn.go +++ b/txn.go @@ -51,8 +51,7 @@ func (p *txnPool) acquire(owner *Collection) *Txn { txn := p.txns.Get().(*Txn) txn.owner = owner txn.logger = owner.logger - txn.index.Grow(uint32(owner.opts.Capacity)) - owner.fill.Clone(&txn.index) + txn.setup = false return txn } @@ -78,6 +77,8 @@ func (p *txnPool) releasePage(buffer *commit.Buffer) { // Txn represents a transaction which supports filtering and projection. type Txn struct { + cursor uint32 // The current cursor + setup bool // Whether the transaction was set up or not owner *Collection // The target collection index bitmap.Bitmap // The filtering index dirty bitmap.Bitmap // The dirty chunks @@ -99,6 +100,20 @@ func (txn *Txn) reset() { txn.updates = txn.updates[:0] } +// bufferFor loads or creates a buffer for a given column. +func (txn *Txn) bufferFor(columnName string) *commit.Buffer { + for _, c := range txn.updates { + if c.Column == columnName { + return c + } + } + + // Create a new buffer + buffer := txn.owner.txns.acquirePage(columnName) + txn.updates = append(txn.updates, buffer) + return buffer +} + // columnCache caches a column by its name. This speeds things up since it's a very // common operation. type columnCache struct { @@ -130,6 +145,7 @@ func (txn *Txn) columnAt(columnName string) (*column, bool) { // With applies a logical AND operation to the current query and the specified index. func (txn *Txn) With(columns ...string) *Txn { + txn.initialize() for _, columnName := range columns { if idx, ok := txn.columnAt(columnName); ok { txn.rangeReadPair(idx, func(dst, src bitmap.Bitmap) { @@ -144,6 +160,7 @@ func (txn *Txn) With(columns ...string) *Txn { // Without applies a logical AND NOT operation to the current query and the specified index. func (txn *Txn) Without(columns ...string) *Txn { + txn.initialize() for _, columnName := range columns { if idx, ok := txn.columnAt(columnName); ok { txn.rangeReadPair(idx, func(dst, src bitmap.Bitmap) { @@ -156,6 +173,7 @@ func (txn *Txn) Without(columns ...string) *Txn { // Union computes a union between the current query and the specified index. func (txn *Txn) Union(columns ...string) *Txn { + txn.initialize() for _, columnName := range columns { if idx, ok := txn.columnAt(columnName); ok { txn.rangeReadPair(idx, func(dst, src bitmap.Bitmap) { @@ -169,6 +187,7 @@ func (txn *Txn) Union(columns ...string) *Txn { // WithValue applies a filter predicate over values for a specific properties. It filters // down the items in the query. func (txn *Txn) WithValue(column string, predicate func(v interface{}) bool) *Txn { + txn.initialize() c, ok := txn.columnAt(column) if !ok { txn.index.Clear() @@ -189,6 +208,7 @@ func (txn *Txn) WithValue(column string, predicate func(v interface{}) bool) *Tx // WithFloat filters down the values based on the specified predicate. The column for // this filter must be numerical and convertible to float64. func (txn *Txn) WithFloat(column string, predicate func(v float64) bool) *Txn { + txn.initialize() c, ok := txn.columnAt(column) if !ok || !c.IsNumeric() { txn.index.Clear() @@ -204,6 +224,7 @@ func (txn *Txn) WithFloat(column string, predicate func(v float64) bool) *Txn { // WithInt filters down the values based on the specified predicate. The column for // this filter must be numerical and convertible to int64. func (txn *Txn) WithInt(column string, predicate func(v int64) bool) *Txn { + txn.initialize() c, ok := txn.columnAt(column) if !ok || !c.IsNumeric() { txn.index.Clear() @@ -219,6 +240,7 @@ func (txn *Txn) WithInt(column string, predicate func(v int64) bool) *Txn { // WithUint filters down the values based on the specified predicate. The column for // this filter must be numerical and convertible to uint64. func (txn *Txn) WithUint(column string, predicate func(v uint64) bool) *Txn { + txn.initialize() c, ok := txn.columnAt(column) if !ok || !c.IsNumeric() { txn.index.Clear() @@ -234,6 +256,7 @@ func (txn *Txn) WithUint(column string, predicate func(v uint64) bool) *Txn { // WithString filters down the values based on the specified predicate. The column for // this filter must be a string. func (txn *Txn) WithString(column string, predicate func(v string) bool) *Txn { + txn.initialize() c, ok := txn.columnAt(column) if !ok || !c.IsTextual() { txn.index.Clear() @@ -248,51 +271,31 @@ func (txn *Txn) WithString(column string, predicate func(v string) bool) *Txn { // Count returns the number of objects matching the query func (txn *Txn) Count() int { + txn.initialize() return int(txn.index.Count()) } -// UpdateAtKey creates a cursor to a specific element at a given key that can be read or updated. -func (txn *Txn) UpdateAtKey(key, columnName string, fn func(v Cursor) error) error { +// QueryKey jumps at a particular key in the collection, sets the cursor to the +// provided position and executes given callback fn. +func (txn *Txn) QueryKey(key string, fn func(Row) error) error { if txn.owner.pk == nil { return errNoKey } if idx, ok := txn.owner.pk.OffsetOf(key); ok { - return txn.UpdateAt(idx, columnName, fn) + return txn.QueryAt(idx, fn) } // If not found, insert at a new index - idx, err := txn.insert(columnName, fn, 0) + idx, err := txn.insert(fn, 0) txn.bufferFor(txn.owner.pk.name).PutString(commit.Put, idx, key) return err } -// UpdateAt creates a cursor to a specific element that can be read or updated. -func (txn *Txn) UpdateAt(index uint32, columnName string, fn func(v Cursor) error) error { - cursor, err := txn.cursorFor(columnName) - if err != nil { - return err - } - - cursor.idx = index - return fn(cursor) -} - -// SelectAt performs a selection on a specific row specified by its index. It returns -// a boolean value indicating whether an element is present at the index or not. -func (txn *Txn) SelectAt(index uint32, fn func(v Selector)) bool { - return txn.owner.SelectAt(index, fn) -} - -// SelectAtKey performs a selection on a specific row specified by its key. It returns -// a boolean value indicating whether an element is present at the key or not. -func (txn *Txn) SelectAtKey(key string, fn func(v Selector)) (found bool) { - return txn.owner.SelectAtKey(key, fn) -} - // DeleteAt attempts to delete an item at the specified index for this transaction. If the item // exists, it marks at as deleted and returns true, otherwise it returns false. func (txn *Txn) DeleteAt(index uint32) bool { + txn.initialize() if !txn.index.Contains(index) { return false } @@ -318,22 +321,22 @@ func (txn *Txn) InsertObjectWithTTL(object Object, ttl time.Duration) (uint32, e } // Insert executes a mutable cursor trasactionally at a new offset. -func (txn *Txn) Insert(columnName string, fn func(v Cursor) error) (uint32, error) { - return txn.insert(columnName, fn, 0) +func (txn *Txn) Insert(fn func(Row) error) (uint32, error) { + return txn.insert(fn, 0) } // InsertWithTTL executes a mutable cursor trasactionally at a new offset and sets the expiration time // based on the specified time-to-live and returns the allocated index. -func (txn *Txn) InsertWithTTL(columnName string, ttl time.Duration, fn func(v Cursor) error) (uint32, error) { - return txn.insert(columnName, fn, time.Now().Add(ttl).UnixNano()) +func (txn *Txn) InsertWithTTL(ttl time.Duration, fn func(Row) error) (uint32, error) { + return txn.insert(fn, time.Now().Add(ttl).UnixNano()) } // insertObject inserts all of the keys of a map, if previously registered as columns. func (txn *Txn) insertObject(object Object, expireAt int64) (uint32, error) { - return txn.insert(expireColumn, func(cursor Cursor) error { + return txn.insert(func(Row) error { for k, v := range object { if _, ok := txn.columnAt(k); ok { - cursor.SetAt(k, v) + txn.bufferFor(k).PutAny(commit.Put, txn.cursor, v) } } return nil @@ -341,67 +344,41 @@ func (txn *Txn) insertObject(object Object, expireAt int64) (uint32, error) { } // insert creates an insertion cursor for a given column and expiration time. -func (txn *Txn) insert(columnName string, fn func(v Cursor) error, expireAt int64) (uint32, error) { - cursor, err := txn.cursorFor(columnName) - if err != nil { - return 0, err - } +func (txn *Txn) insert(fn func(Row) error, expireAt int64) (uint32, error) { // At a new index, add the insertion marker - cursor.idx = txn.owner.next() - txn.bufferFor(rowColumn).PutOperation(commit.Insert, cursor.idx) - if expireAt != 0 { - cursor.SetAt(expireColumn, expireAt) - } + idx := txn.owner.next() + txn.bufferFor(rowColumn).PutOperation(commit.Insert, idx) - return cursor.idx, fn(cursor) -} - -// Select iterates over the result set and allows to read any column. While this -// is flexible, it is not the most efficient way, consider Range() as an alternative -// iteration method over a specific column which also supports modification. -func (txn *Txn) Select(fn func(v Selector)) { - txn.rangeRead(func(offset uint32, index bitmap.Bitmap) { - index.Range(func(x uint32) { - fn(Selector{ - idx: offset + x, - txn: txn, - }) - }) - }) -} + // If no expiration was specified, simply insert + if expireAt == 0 { + return idx, txn.QueryAt(idx, fn) + } -// DeleteIf iterates over the result set and calls the provided funciton on each element. If -// the function returns true, the element at the index will be marked for deletion. The -// actual delete will take place once the transaction is committed. -func (txn *Txn) DeleteIf(fn func(v Selector) bool) { - txn.index.Range(func(x uint32) { - if fn(Selector{idx: x, txn: txn}) { - txn.deleteAt(x) - } + // If expiration was specified, set it + return idx, txn.QueryAt(idx, func(r Row) error { + r.SetInt64(expireColumn, expireAt) + return fn(r) }) } // DeleteAll marks all of the items currently selected by this transaction for deletion. The // actual delete will take place once the transaction is committed. func (txn *Txn) DeleteAll() { + txn.initialize() txn.index.Range(func(x uint32) { txn.deleteAt(x) }) } -// Range selects and iterates over a results for a specific column. The cursor provided -// also allows to select other columns, but at a slight performance cost. -func (txn *Txn) Range(column string, fn func(v Cursor)) error { - cur, err := txn.cursorFor(column) - if err != nil { - return err - } - +// Range selects and iterates over result set. In each iteration step, the internal +// transaction cursor is updated and can be used by various column accessors. +func (txn *Txn) Range(fn func(idx uint32)) error { + txn.initialize() txn.rangeRead(func(offset uint32, index bitmap.Bitmap) { index.Range(func(x uint32) { - cur.idx = offset + x - fn(cur) + txn.cursor = offset + x + fn(offset + x) }) }) return nil diff --git a/txn_cursor.go b/txn_cursor.go deleted file mode 100644 index 32c38c0..0000000 --- a/txn_cursor.go +++ /dev/null @@ -1,202 +0,0 @@ -// Copyright (c) Roman Atachiants and contributors. All rights reserved. -// Licensed under the MIT license. See LICENSE file in the project root for details. - -package column - -import ( - "fmt" - - "github.com/kelindar/column/commit" -) - -// bufferFor loads or creates a buffer for a given column. -func (txn *Txn) bufferFor(columnName string) *commit.Buffer { - for _, c := range txn.updates { - if c.Column == columnName { - return c - } - } - - // Create a new buffer - buffer := txn.owner.txns.acquirePage(columnName) - txn.updates = append(txn.updates, buffer) - return buffer -} - -// cursorFor returns a cursor for a specified column -func (txn *Txn) cursorFor(columnName string) (Cursor, error) { - c, ok := txn.columnAt(columnName) - if !ok { - return Cursor{}, fmt.Errorf("column: specified column '%v' does not exist", columnName) - } - - // Create a Cursor - return Cursor{ - column: c, - update: txn.bufferFor(columnName), - Selector: Selector{ - txn: txn, - }, - }, nil -} - -// --------------------------- Selector --------------------------- - -// Selector represents a iteration Selector that supports both retrieval of column values -// for the specified row and modification (update, delete). -type Selector struct { - idx uint32 // The current index - txn *Txn // The optional transaction, but one of them is required - col *Collection // The optional collection, but one of them is required -} - -// columnAt loads the column based on whether the selector has a transaction or not. -func (cur *Selector) columnAt(column string) (*column, bool) { - if cur.txn != nil { - return cur.txn.columnAt(column) - } - - // Load directly from the collection - return cur.col.cols.Load(column) -} - -// ValueAt reads a value for a current row at a given column. -func (cur *Selector) ValueAt(column string) (out interface{}) { - if c, ok := cur.columnAt(column); ok { - out, _ = c.Value(cur.idx) - } - return -} - -// StringAt reads a string value for a current row at a given column. -func (cur *Selector) StringAt(column string) (out string) { - if c, ok := cur.columnAt(column); ok { - out, _ = c.String(cur.idx) - } - return -} - -// FloatAt reads a float64 value for a current row at a given column. -func (cur *Selector) FloatAt(column string) (out float64) { - if c, ok := cur.columnAt(column); ok { - out, _ = c.Float64(cur.idx) - } - return -} - -// IntAt reads an int64 value for a current row at a given column. -func (cur *Selector) IntAt(columnName string) (out int64) { - if c, ok := cur.columnAt(columnName); ok { - out, _ = c.Int64(cur.idx) - } - return -} - -// UintAt reads a uint64 value for a current row at a given column. -func (cur *Selector) UintAt(column string) (out uint64) { - if c, ok := cur.columnAt(column); ok { - out, _ = c.Uint64(cur.idx) - } - return -} - -// BoolAt reads a boolean value for a current row at a given column. -func (cur *Selector) BoolAt(column string) bool { - if c, ok := cur.columnAt(column); ok { - return c.Contains(cur.idx) - } - return false -} - -// --------------------------- Cursor --------------------------- - -// Cursor represents a iteration Selector that is bound to a specific column. -type Cursor struct { - Selector - update *commit.Buffer // The index of the update queue - column *column // The selected column -} - -// Index returns the current index of the cursor. -func (cur *Cursor) Index() uint32 { - return cur.idx -} - -// Value reads a value for a current row at a given column. -func (cur *Cursor) Value() (out interface{}) { - out, _ = cur.column.Value(cur.idx) - return -} - -// String reads a string value for a current row at a given column. -func (cur *Cursor) String() (out string) { - out, _ = cur.column.String(cur.idx) - return -} - -// Float reads a float64 value for a current row at a given column. -func (cur *Cursor) Float() (out float64) { - out, _ = cur.column.Float64(cur.idx) - return -} - -// Int reads an int64 value for a current row at a given column. -func (cur *Cursor) Int() int { - out, _ := cur.column.Int64(cur.idx) - return int(out) -} - -// Uint reads a uint64 value for a current row at a given column. -func (cur *Cursor) Uint() uint { - out, _ := cur.column.Uint64(cur.idx) - return uint(out) -} - -// Bool reads a boolean value for a current row at a given column. -func (cur *Cursor) Bool() bool { - return cur.column.Contains(cur.idx) -} - -// --------------------------- Update/Delete ---------------------------- - -// Delete deletes the current item. The actual operation will be queued and -// executed once the current the transaction completes. -func (cur *Cursor) Delete() { - cur.txn.deleteAt(cur.idx) -} - -// Set updates a column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) Set(value interface{}) { - cur.update.PutAny(commit.Put, cur.idx, value) -} - -// SetAt updates a specified column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetAt(column string, value interface{}) { - cur.txn.bufferFor(column).PutAny(commit.Put, cur.idx, value) -} - -// SetString updates a column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetString(value string) { - cur.update.PutString(commit.Put, cur.idx, value) -} - -// SetStringAt updates a specified column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetStringAt(column string, value string) { - cur.txn.bufferFor(column).PutString(commit.Put, cur.idx, value) -} - -// SetBool updates a column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetBool(value bool) { - cur.update.PutBool(cur.idx, value) -} - -// SetBoolAt updates a specified column value for the current item. The actual operation -// will be queued and executed once the current the transaction completes. -func (cur *Cursor) SetBoolAt(column string, value bool) { - cur.txn.bufferFor(column).PutBool(cur.idx, value) -} diff --git a/txn_lock.go b/txn_lock.go index b6745f9..c887649 100644 --- a/txn_lock.go +++ b/txn_lock.go @@ -15,6 +15,35 @@ const ( chunkSize = 1 << chunkShift ) +// initialize ensures that the transaction is pre-initialized with the snapshot +// of the owner's fill list. +func (txn *Txn) initialize() { + if txn.setup { + return + } + + txn.owner.lock.RLock() + txn.index.Grow(uint32(txn.owner.opts.Capacity)) + txn.owner.fill.Clone(&txn.index) + txn.owner.lock.RUnlock() + txn.setup = true +} + +// --------------------------- Locked Seek --------------------------- + +// QueryAt jumps at a particular offset in the collection, sets the cursor to the +// provided position and executes given callback fn. +func (txn *Txn) QueryAt(index uint32, f func(Row) error) (err error) { + lock := txn.owner.slock + txn.cursor = index + + chunk := commit.ChunkAt(index) + lock.RLock(uint(chunk)) + err = f(Row{txn}) + lock.RUnlock(uint(chunk)) + return err +} + // --------------------------- Locked Range --------------------------- // rangeRead iterates over index, chunk by chunk and ensures that each diff --git a/txn_row.go b/txn_row.go new file mode 100644 index 0000000..4005639 --- /dev/null +++ b/txn_row.go @@ -0,0 +1,218 @@ +// Copyright (c) Roman Atachiants and contributors. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for details. + +package column + +// Row represents a cursor at a particular row offest in the transaction. +type Row struct { + txn *Txn +} + +// --------------------------- Numbers ---------------------------- + +// Int loads a int value at a particular column +func (r Row) Int(columnName string) (v int, ok bool) { + return intReaderFor(r.txn, columnName).Get() +} + +// SetInt stores a int value at a particular column +func (r Row) SetInt(columnName string, value int) { + r.txn.Int(columnName).Set(value) +} + +// AddInt adds delta to a int value at a particular column +func (r Row) AddInt(columnName string, value int) { + r.txn.Int(columnName).Add(value) +} + +// Int16 loads a int16 value at a particular column +func (r Row) Int16(columnName string) (v int16, ok bool) { + return int16ReaderFor(r.txn, columnName).Get() +} + +// SetInt16 stores a int16 value at a particular column +func (r Row) SetInt16(columnName string, value int16) { + r.txn.Int16(columnName).Set(value) +} + +// AddInt16 adds delta to a int16 value at a particular column +func (r Row) AddInt16(columnName string, value int16) { + r.txn.Int16(columnName).Add(value) +} + +// Int32 loads a int32 value at a particular column +func (r Row) Int32(columnName string) (v int32, ok bool) { + return int32ReaderFor(r.txn, columnName).Get() +} + +// SetInt32 stores a int32 value at a particular column +func (r Row) SetInt32(columnName string, value int32) { + r.txn.Int32(columnName).Set(value) +} + +// AddInt32 adds delta to a int32 value at a particular column +func (r Row) AddInt32(columnName string, value int32) { + r.txn.Int32(columnName).Add(value) +} + +// Int64 loads a int64 value at a particular column +func (r Row) Int64(columnName string) (v int64, ok bool) { + return int64ReaderFor(r.txn, columnName).Get() +} + +// SetInt64 stores a int64 value at a particular column +func (r Row) SetInt64(columnName string, value int64) { + r.txn.Int64(columnName).Set(value) +} + +// AddInt64 adds delta to a int64 value at a particular column +func (r Row) AddInt64(columnName string, value int64) { + r.txn.Int64(columnName).Add(value) +} + +// Uint loads a uint value at a particular column +func (r Row) Uint(columnName string) (v uint, ok bool) { + return uintReaderFor(r.txn, columnName).Get() +} + +// SetUint stores a uint value at a particular column +func (r Row) SetUint(columnName string, value uint) { + r.txn.Uint(columnName).Set(value) +} + +// AddUint adds delta to a uint value at a particular column +func (r Row) AddUint(columnName string, value uint) { + r.txn.Uint(columnName).Add(value) +} + +// Uint16 loads a uint16 value at a particular column +func (r Row) Uint16(columnName string) (v uint16, ok bool) { + return uint16ReaderFor(r.txn, columnName).Get() +} + +// SetUint16 stores a uint16 value at a particular column +func (r Row) SetUint16(columnName string, value uint16) { + r.txn.Uint16(columnName).Set(value) +} + +// AddUint16 adds delta to a uint16 value at a particular column +func (r Row) AddUint16(columnName string, value uint16) { + r.txn.Uint16(columnName).Add(value) +} + +// Uint32 loads a uint32 value at a particular column +func (r Row) Uint32(columnName string) (v uint32, ok bool) { + return uint32ReaderFor(r.txn, columnName).Get() +} + +// SetUint32 stores a uint32 value at a particular column +func (r Row) SetUint32(columnName string, value uint32) { + r.txn.Uint32(columnName).Set(value) +} + +// AddUint32 adds delta to a uint32 value at a particular column +func (r Row) AddUint32(columnName string, value uint32) { + r.txn.Uint32(columnName).Add(value) +} + +// Uint64 loads a uint64 value at a particular column +func (r Row) Uint64(columnName string) (v uint64, ok bool) { + return uint64ReaderFor(r.txn, columnName).Get() +} + +// SetUint64 stores a uint64 value at a particular column +func (r Row) SetUint64(columnName string, value uint64) { + r.txn.Uint64(columnName).Set(value) +} + +// AddUint64 adds delta to a uint64 value at a particular column +func (r Row) AddUint64(columnName string, value uint64) { + r.txn.Uint64(columnName).Add(value) +} + +// Float32 loads a float32 value at a particular column +func (r Row) Float32(columnName string) (v float32, ok bool) { + return float32ReaderFor(r.txn, columnName).Get() +} + +// SetFloat32 stores a float32 value at a particular column +func (r Row) SetFloat32(columnName string, value float32) { + r.txn.Float32(columnName).Set(value) +} + +// AddFloat32 adds delta to a float32 value at a particular column +func (r Row) AddFloat32(columnName string, value float32) { + r.txn.Float32(columnName).Add(value) +} + +// Float64 loads a float64 value at a particular column +func (r Row) Float64(columnName string) (v float64, ok bool) { + return float64ReaderFor(r.txn, columnName).Get() +} + +// SetFloat64 stores a float64 value at a particular column +func (r Row) SetFloat64(columnName string, value float64) { + r.txn.Float64(columnName).Set(value) +} + +// AddFloat64 adds delta to a float64 value at a particular column +func (r Row) AddFloat64(columnName string, value float64) { + r.txn.Float64(columnName).Add(value) +} + +// --------------------------- Strings ---------------------------- + +// Key loads a primary key value at a particular column +func (r Row) Key() (v string, ok bool) { + if pk := r.txn.owner.pk; pk != nil { + v, ok = pk.LoadString(r.txn.cursor) + } + return +} + +// SetKey stores a primary key value at a particular column +func (r Row) SetKey(key string) { + r.txn.Key().Set(key) +} + +// String loads a string value at a particular column +func (r Row) String(columnName string) (v string, ok bool) { + return stringReaderFor(r.txn, columnName).Get() +} + +// SetString stores a string value at a particular column +func (r Row) SetString(columnName string, value string) { + r.txn.String(columnName).Set(value) +} + +// Enum loads a string value at a particular column +func (r Row) Enum(columnName string) (v string, ok bool) { + return enumReaderFor(r.txn, columnName).Get() +} + +// SetEnum stores a string value at a particular column +func (r Row) SetEnum(columnName string, value string) { + r.txn.Enum(columnName).Set(value) +} + +// --------------------------- Others ---------------------------- + +// Bool loads a bool value at a particular column +func (r Row) Bool(columnName string) bool { + return boolReaderFor(r.txn, columnName).Get() +} + +// SetBool stores a bool value at a particular column +func (r Row) SetBool(columnName string, value bool) { + r.txn.Bool(columnName).Set(value) +} + +// Any loads a bool value at a particular column +func (r Row) Any(columnName string) (interface{}, bool) { + return anyReaderFor(r.txn, columnName).Get() +} + +// SetAny stores a bool value at a particular column +func (r Row) SetAny(columnName string, value interface{}) { + r.txn.Any(columnName).Set(value) +} diff --git a/txn_test.go b/txn_test.go index 85d5d75..3d59b8b 100644 --- a/txn_test.go +++ b/txn_test.go @@ -15,15 +15,18 @@ func TestFind(t *testing.T) { players := loadPlayers(500) count := 0 players.Query(func(txn *Txn) error { + names := txn.Enum("name") + txn.WithString("race", func(v string) bool { return v == "human" }).WithString("class", func(v string) bool { return v == "mage" }).WithUint("age", func(v uint64) bool { return v >= 30 - }).Range("name", func(v Cursor) { + }).Range(func(index uint32) { count++ - assert.NotEmpty(t, v.String()) + name, _ := names.Get() + assert.NotEmpty(t, name) }) return nil }) @@ -35,7 +38,7 @@ func TestMany(t *testing.T) { players := loadPlayers(20000) count := 0 players.Query(func(txn *Txn) error { - txn.Range("name", func(v Cursor) { + txn.Range(func(index uint32) { count++ }) return nil @@ -123,28 +126,35 @@ func TestIndexInvalid(t *testing.T) { return nil }) - assert.Error(t, players.Query(func(txn *Txn) error { - return txn.Range("invalid-column", func(v Cursor) { + assert.NoError(t, players.Query(func(txn *Txn) error { + return txn.Range(func(index uint32) { // do nothing }) })) players.Query(func(txn *Txn) error { - assert.False(t, txn.SelectAt(999999, func(v Selector) {})) - assert.True(t, txn.SelectAt(0, func(v Selector) {})) + assert.Error(t, txn.QueryAt(999999, func(Row) error { + return fmt.Errorf("not found") + })) + assert.NoError(t, txn.QueryAt(0, func(Row) error { + return nil + })) return nil }) - assert.NoError(t, players.Query(func(txn *Txn) error { - return txn.Range("balance", func(v Cursor) { - v.AddFloat64At("invalid-column", 1) + assert.Panics(t, func() { + players.Query(func(txn *Txn) error { + invalid := txn.Float64("invalid-column") + return txn.Range(func(index uint32) { + invalid.Add(1) + }) }) - })) + }) assert.NoError(t, players.Query(func(txn *Txn) error { - txn.DeleteIf(func(v Selector) bool { - return v.StringAt("class") == "rogue" - }) + txn.WithString("class", func(v string) bool { + return v == "rogue" + }).DeleteAll() return nil })) @@ -195,43 +205,20 @@ func TestIndexed(t *testing.T) { // Check the index value players.Query(func(txn *Txn) error { + age := txn.Float64("age") + old := txn.Bool("old") + class := txn.Enum("class") txn.With("human", "mage", "old"). - Select(func(v Selector) { - assert.True(t, v.FloatAt("age") >= 30) - assert.True(t, v.IntAt("age") >= 30) - assert.True(t, v.UintAt("age") >= 30) - assert.True(t, v.ValueAt("old").(bool)) - assert.True(t, v.BoolAt("old")) - assert.Equal(t, "mage", v.StringAt("class")) - assert.False(t, v.BoolAt("xxx")) - }) - return nil - }) - - // Check with multiple Selectors - players.Query(func(txn *Txn) error { - result := txn.With("human", "mage", "old") + Range(func(i uint32) { + age, _ := age.Get() + class, _ := class.Get() - result.Range("age", func(v Cursor) { - assert.True(t, v.Float() >= 30) - assert.True(t, v.Int() >= 30) - assert.True(t, v.Uint() >= 30) - }) - - result.Range("old", func(v Cursor) { - assert.True(t, v.Value().(bool)) - assert.True(t, v.Bool()) - }) - - result.Range("class", func(v Cursor) { - //assert.Equal(t, "mage", v.String()) - assert.Equal(t, float64(0), v.Float()) - assert.Equal(t, int(0), v.Int()) - assert.Equal(t, uint(0), v.Uint()) - }) + assert.True(t, age >= 30) + assert.True(t, old.Get()) + assert.Equal(t, "mage", class) + }) return nil }) - } func TestDeleteAll(t *testing.T) { @@ -276,8 +263,9 @@ func TestUpdateBulkWithIndex(t *testing.T) { // Make everyone poor players.Query(func(txn *Txn) error { - txn.Range("balance", func(v Cursor) { - v.SetFloat64(1.0) + balance := txn.Float64("balance") + txn.Range(func(index uint32) { + balance.Set(1.0) }) return nil }) @@ -303,12 +291,13 @@ func TestIndexWithAtomicAdd(t *testing.T) { return r.Float() >= 3000 }) - // Increment balance 30 times by 50+50 = 3000 + // Add balance 30 times by 50+50 = 3000 players.Query(func(txn *Txn) error { + balance := txn.Float64("balance") for i := 0; i < 30; i++ { - txn.Range("balance", func(v Cursor) { - v.AddFloat64(50.0) - v.AddFloat64At("balance", 50.0) + txn.Range(func(index uint32) { + balance.Add(50.0) + balance.Add(50.0) }) } return nil @@ -316,8 +305,11 @@ func TestIndexWithAtomicAdd(t *testing.T) { // Everyone should now be rich and the indexes updated players.Query(func(txn *Txn) error { - txn.Range("balance", func(v Cursor) { - assert.GreaterOrEqual(t, v.Float(), 3000.0) + balance := txn.Float64("balance") + txn.Range(func(index uint32) { + value, ok := balance.Get() + assert.True(t, ok) + assert.GreaterOrEqual(t, value, 3000.0) }) assert.Equal(t, 500, txn.With("rich").Count()) @@ -333,8 +325,9 @@ func TestUpdateWithRollback(t *testing.T) { // Make everyone rich players.Query(func(txn *Txn) error { - txn.Range("balance", func(v Cursor) { - v.SetFloat64(5000.0) + balance := txn.Float64("balance") + txn.Range(func(index uint32) { + balance.Set(5000.0) }) return nil }) @@ -347,8 +340,9 @@ func TestUpdateWithRollback(t *testing.T) { // Try out the rollback players.Query(func(txn *Txn) error { - txn.Range("balance", func(v Cursor) { - v.SetFloat64(1.0) + balance := txn.Float64("balance") + txn.Range(func(index uint32) { + balance.Set(1.0) }) return fmt.Errorf("trigger rollback") }) @@ -366,16 +360,19 @@ func TestCountTwice(t *testing.T) { model.CreateColumnsOf(map[string]interface{}{ "string": "", }) + model.Query(func(txn *Txn) error { for i := 0; i < 20000; i++ { - txn.InsertObject(map[string]interface{}{ + _, err := txn.InsertObject(map[string]interface{}{ "string": fmt.Sprint(i), }) + + assert.NoError(t, err) } return nil }) - model.Query(func(txn *Txn) error { + assert.NoError(t, model.Query(func(txn *Txn) error { assert.Equal(t, 20000, txn.Count()) assert.Equal(t, 1, txn.WithValue("string", func(v interface{}) bool { return v.(string) == "5" @@ -384,7 +381,7 @@ func TestCountTwice(t *testing.T) { return v == "5" }).Count()) return nil - }) + })) } // Details: https://github.com/kelindar/column/issues/15 @@ -408,13 +405,18 @@ func TestUninitializedSet(t *testing.T) { })) assert.NoError(t, c.Query(func(txn *Txn) error { - assert.NoError(t, txn.Range("col2", func(v Cursor) { - v.SetFloat64(0) + col1 := txn.String("col1") + col2 := txn.Float64("col2") + col3 := txn.String("col3") + + assert.NoError(t, txn.Range(func(index uint32) { + col2.Set(0) })) - return txn.Range("col1", func(v Cursor) { - if a, h := someMap[v.String()]; h { - v.SetFloat64At("col2", a[1].(float64)) - v.SetStringAt("col3", a[0].(string)) + return txn.Range(func(index uint32) { + value, _ := col1.Get() + if a, h := someMap[value]; h { + col2.Set(a[1].(float64)) + col3.Set(a[0].(string)) } }) })) @@ -427,23 +429,34 @@ func TestUpdateAt(t *testing.T) { "col1": "hello", }) - assert.NoError(t, c.UpdateAt(index, "col1", func(v Cursor) error { - assert.Equal(t, index, v.Index()) - v.Set("hi") + assert.NoError(t, c.QueryAt(index, func(r Row) error { + r.SetString("col1", "hi") return nil })) - - assert.True(t, c.SelectAt(index, func(v Selector) { - assert.Equal(t, "hi", v.StringAt("col1")) - })) } func TestUpdateAtInvalid(t *testing.T) { c := NewCollection() c.CreateColumn("col1", ForString()) - assert.Error(t, c.UpdateAt(0, "col2", func(v Cursor) error { - v.SetString("hi") + assert.Panics(t, func() { + c.QueryAt(0, func(r Row) error { + r.SetString("col2", "hi") + return nil + }) + }) +} +func TestUpdateAtNoChanges(t *testing.T) { + c := NewCollection() + c.CreateColumn("col1", ForString()) + + assert.NoError(t, c.QueryAt(20000, func(r Row) error { + r.SetString("col1", "Roman") + return nil + })) + + assert.NoError(t, c.QueryAt(0, func(r Row) error { + r.txn.bufferFor("xxx").PutInt(123, 123) return nil })) } @@ -452,25 +465,30 @@ func TestUpsertKey(t *testing.T) { c := NewCollection() c.CreateColumn("key", ForKey()) c.CreateColumn("val", ForString()) - assert.NoError(t, c.UpdateAtKey("1", "val", func(v Cursor) error { - v.Set("Roman") + assert.NoError(t, c.QueryKey("1", func(r Row) error { + r.SetString("val", "Roman") return nil })) count := 0 - c.SelectAtKey("1", func(v Selector) { - assert.Equal(t, "Roman", v.StringAt("val")) + assert.NoError(t, c.QueryKey("1", func(r Row) error { count++ - }) + return nil + })) + assert.Equal(t, 1, count) } func TestUpsertKeyNoColumn(t *testing.T) { c := NewCollection() c.CreateColumn("key", ForKey()) - assert.Error(t, c.UpdateAtKey("1", "xxx", func(v Cursor) error { - return nil - })) + + assert.Panics(t, func() { + c.QueryKey("1", func(r Row) error { + r.Enum("xxx") + return nil + }) + }) } func TestDuplicateKey(t *testing.T) { @@ -479,7 +497,77 @@ func TestDuplicateKey(t *testing.T) { assert.Error(t, c.CreateColumn("key2", ForKey())) } -func TestDataRace(t *testing.T) { +func TestRowMethods(t *testing.T) { + c := NewCollection() + c.CreateColumn("key", ForKey()) + c.CreateColumn("bool", ForBool()) + c.CreateColumn("name", ForString()) + c.CreateColumn("int", ForInt()) + c.CreateColumn("int16", ForInt16()) + c.CreateColumn("int32", ForInt32()) + c.CreateColumn("int64", ForInt64()) + c.CreateColumn("uint", ForUint()) + c.CreateColumn("uint16", ForUint16()) + c.CreateColumn("uint32", ForUint32()) + c.CreateColumn("uint64", ForUint64()) + c.CreateColumn("float32", ForFloat32()) + c.CreateColumn("float64", ForFloat64()) + + c.Insert(func(r Row) error { + r.SetKey("key") + r.SetBool("bool", true) + r.SetAny("name", "Roman") + + // Numbers + r.SetInt("int", 1) + r.SetInt16("int16", 1) + r.SetInt32("int32", 1) + r.SetInt64("int64", 1) + r.SetUint("uint", 1) + r.SetUint16("uint16", 1) + r.SetUint32("uint32", 1) + r.SetUint64("uint64", 1) + r.SetFloat32("float32", 1) + r.SetFloat64("float64", 1) + + // Increment + r.AddInt("int", 1) + r.AddInt16("int16", 1) + r.AddInt32("int32", 1) + r.AddInt64("int64", 1) + r.AddUint("uint", 1) + r.AddUint16("uint16", 1) + r.AddUint32("uint32", 1) + r.AddUint64("uint64", 1) + r.AddFloat32("float32", 1) + r.AddFloat64("float64", 1) + return nil + }) + + exists := func(v interface{}, ok bool) { + assert.NotNil(t, v) + assert.True(t, ok) + } + + c.QueryKey("key", func(r Row) error { + assert.True(t, r.Bool("bool")) + exists(r.Key()) + exists(r.Any("name")) + exists(r.Int("int")) + exists(r.Int16("int16")) + exists(r.Int32("int32")) + exists(r.Int64("int64")) + exists(r.Uint("uint")) + exists(r.Uint16("uint16")) + exists(r.Uint32("uint32")) + exists(r.Uint64("uint64")) + exists(r.Float32("float32")) + exists(r.Float64("float64")) + return nil + }) +} + +func TestRow(t *testing.T) { c := NewCollection() c.CreateColumn("name", ForKey()) @@ -487,8 +575,9 @@ func TestDataRace(t *testing.T) { wg.Add(2) go c.Query(func(txn *Txn) error { - txn.Insert("name", func(v Cursor) error { - v.Set("Roman") + txn.Insert(func(r Row) error { + name := txn.Key() + name.Set("Roman") return nil }) wg.Done()