Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Collection map_reduce does not handle ObjectId fields #81

Open
baruchoxman opened this Issue · 1 comment

2 participants

@baruchoxman

When running map_reduce on a collection, and using a field that is an ObjectId, the returned results do not preserve the ObjectId values.

This is since the json.dumps converts such values to the following:

doc = json.dumps({'rel_id':ObjectId('52530aae4e950d1fe0669036')}, default=js
on_util.default)
doc
'{"rel_id": {"$oid": "52530aae4e950d1fe0669036"}}'

Then, in the 'emit' function in doMap, the above dictionary value is used as a key, and javascript implicitly converts it into a '[object Object'] string.

I've implemented the following patch, but maybe a better fix can be made...

  1. Replacing 'emit' with the following implementation:
    function emit(key, val) {
    if (key['$oid']) {
    mapped_key = '$oid' + key['$oid'];
    }
    else {
    mapped_key = key;
    }
    if(!mappedDict[mapped_key]) {
    mappedDict[mapped_key] = [];
    }
    mappedDict[mapped_key].push(val);
    }

  2. After receiving reduced_rows, do the conversion back to ObjectId:
    reduced_rows = reduce_ctx.call('doReduce', reduce_func, mapped_rows)[:limit]
    for reduced_row in reduced_rows:
    if reduced_row['_id'].startswith('$oid'):
    reduced_row['_id'] = ObjectId(reduced_row['_id'][4:])

Thanks!

@scottsexton

This seems reasonable to me. Can you open a pull request for your patch?

@baruchoxman baruchoxman referenced this issue from a commit in baruchoxman/mongomock
@baruchoxman baruchoxman Fix + test for #81
Making collection map_reduce properly handle ObjectId fields
1bb7f96
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.