atomic key-value commands

[pekeler]

I have a document that has a hash as one if its values. The values of that hash are numbers and I need to increment them from remote. Now I wish ArangoDB had Redis' HINCRBY and I could use it from the arangojs adapter.

// usageDocument before
{
  _key: "1234",
  phoneCalls: {
    "110": 5
  }
}

usageCollection.hincrby('1234', 'phoneCalls', '110', 1);
usageCollection.hincrby('1234', 'phoneCalls', '040 2374321', 1);

// usageDocument after
{
  _key: "1234",
  phoneCalls: {
    "110": 6,
    "040 2374321": 1
  }
}

That's just one example. Another example for a command I could use right now would be LPUSH to insert a new value at the beginning of an array in a document.

I think it would be awesome if there were equivalents for almost all of these in ArangoDB and if they were usable from the ReST api, local JavaScript, and maybe even straight in AQL.

http://redis.io/commands#hash
http://redis.io/commands#generic
http://redis.io/commands#list
http://redis.io/commands#set
http://redis.io/commands#sorted_set
http://redis.io/commands#string

[pluma]

Unless I'm missing something, this can easily be done using AQL:

LET doc = DOCUMENT(usageCollection, "1234")
UPDATE doc WITH {
  phoneCalls: {
    "110": (OLD.phoneCalls["110"] || 0) + 1,
    "040 2374321": (OLD.phoneCalls["040 2374321"] || 0) + 1
  }
} IN usageCollection

What's your concern? Performance? Flexibility?

I guess an AQL function along the lines of ...

LET diff = {"110": 1, "040 2374321": 1}
UPDATE doc WITH {
  phoneCalls: HINCR(OLD.phoneCalls, diff)
} IN usageCollection

... could be useful.

[pekeler]

@pluma interesting. I guess your HINCR example could be realized with a user defined AQL function. Still...
https://docs.mongodb.org/v2.6/reference/operator/update/inc/
http://docs.couchbase.com/developer/dotnet-2.0/atomic-operations.html
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/WorkingWithItems.html#WorkingWithItems.AtomicCounters

;)

[pluma]

Based on my example the implementation of HINCR (or MongoDB-style $inc) could be something like this in JS:

function HINCR(obj, diff) {
  for (const key of Object.keys(diff)) {
    if (!obj[key]) {
      obj[key] = diff[key];
    } else {
      obj[key] += diff[key];
    }
  }
  return obj;
}

I'm not sure what the performance implications of this are and whether it would be as easy to implement natively. @jsteemann ?

[dothebart]

@mchacki has been doing all those javascript to c++ conversions.

[pekeler]

FYI: this is what I settled on

function HINCR(obj, key) {
  let result = {};
  result[key] = (obj[key] || 0) + 1;
  return result;
} 

which is sufficient when doing an update. (Btw you can't use a const for the loop's variable.)

Anyways, my main point is that all these actions make Redis useful (http://try.redis.io) and that great artists steal ;)

[pluma]

You can actually use const in a for-of like I did. Its scope is limited to each iteration, it's not reassigned.

[CoDEmanX]

@pluma: Do you know what difference it makes to V8? (var vs. let vs. const vs.)

I wonder why Object.keys() is required in for-of at all. Objects don't implement Symbol.iterator, but why? In Python, you can directly iterate over dictionary keys...

[pekeler]

for (const e of ['a', 'b', 'c']) console.log(e); with node 5.4.1 results in

a
a
a

vs. for (let e of ['a', 'b', 'c']) console.log(e);

a
b
c

[pluma]

@pekeler only outside strict-mode. Prefix the line with 'use strict'; in the REPL and it'll behave the same. ArangoDB actually complains if you try to use let/const outside of strict mode. I would imagine Node follows the pre-ES6 non-standard semantics outside strict mode -- it doesn't throw a ReferenceError when a const variable is referred to (e.g. via typeof) in the same block before it is declared, like var, not like let (or const in strict mode):

// execute each line individually in a fresh node shell
// to avoid raising a TypeError when re-declaring const

'use strict'; console.log(typeof v); var v = 5; // logs "undefined"

'use strict'; console.log(typeof c); const c = 5; // throws ReferenceError

'use strict'; console.log(typeof x); let x = 5; // throws ReferenceError

console.log(typeof v); var v = 5; // logs "undefined"

console.log(typeof c); const c = 5; // logs "undefined"

console.log(typeof x); let x = 5; // throws ReferenceError

@CoDEmanX some people have argued that it could be optimized but even if it were the difference would likely be insignificant. It's mostly about signalling intent: this variable will not be reassigned throughout this block of code.

The reason Object.keys is required is that objects aren't dictionaries. Maps are dictionaries. And maps provide three generators: keys, values and entries (pairs of keys and values). Because Object.keys preceded the new collection types it will continue to return an array instead of a generator (and arrays can be treated as generators, i.e. used in for-of loops). The additional functions Object.values and Object.entries are currently in stage 3 and will likely land in ES2016 or ES2017 but they will behave like Object.keys (i.e. return arrays) rather than like the Map or Set equivalents (which is okay: they're also functions on the Object constructor like Object.keys rather than methods like the Map and Set equivalents).

If any of you want to discuss this further I would suggest taking it to the mailing list or IRC. Let's keep the issue focussed on its original topic: atomic key-value commands, not JS syntax. 😄

EDIT: Another more direct proof that const outside strict mode uses function scope rather than block scope:

// again, remember to run each line a fresh node shell

'use strict'; {const c = 5;} console.log(c); // throws ReferenceError

{const c = 5;} console.log(c); // logs 5

[CoDEmanX]

Thanks @pluma, I wasn't aware of the differences between objects and maps (complete result array vs. iterator returned). BTW: for (entry of m) is equivalent to for (entry of m.entries()) if m is a Map. It also greatly combines with destructuring: for ([key, value] of m). But I digress.

AQL is mostly a declarative language. Mostly, because there are also many functions you can call. If I'm not mistaken, that makes it also a functional programming language. @pekeler's example of collection.hincrby(...) looks like object oriented programming, which can't be directly translated to AQL. Nonetheless, it could very well be a supported method of a collection object in some drivers like JavaScript. An AQL equivalent would be more like this though:

LET doc = DOCUMENT("coll/1234")
UPDATE doc WITH {phoneCalls: INCREMENT(OLD.phoneCalls)}

It would be more convenient, if something like the following was allowed:

UPDATE "coll/1234" INCREMENT OLD.phoneCalls

Or even:

INCREMENT DOCUMENT("coll/1234").phoneCalls

Note the change of INCREMENT from function to keyword to make the instruction shorter, but also less flexible.

INCREMENT would be a lot more useful with the following two optional parameters:

• default value: if target attribute doesn't exist, initialize with given default value. Corner cases: should it also initialize if the attribute exists already, but has a null value? Should INCREMENT not do anything or raise an error if the target attribute exists already and is non-numeric (e.g. string)?
• amount: add amount to current (or default) value. If negative amounts are allowed, should there still be a DECREMENT operation?

As function, the signature with support for both aforementioned options could look like INCREMENT(attr, default, amount) or INCREMENT(attr, amount, default). I'm in favor of the latter, because it's more apparent how to specify a default value even if you don't want to increment by a certain amount (just make it a 1). The other way around, people may wonder what to write for default value if they don't want to specify it (null or undefined?).