Skip to content

Map fields in persistent-mongoDB #149

burp opened this Issue Aug 28, 2013 · 4 comments

2 participants

burp commented Aug 28, 2013


I think this is not intended behaviour, so I'll report it as a bug: when one stores a Map, for example with Text keys, and the keys contain special characters like dots, which are not allowed in mongodb keys, one ends up with a failure: "WriteFailure 12527 not okForStorage".

I'm not sure what the best solution to this is. Serialize the map into a single string like for SQL databases, or is it up to me to not use keys with dots? Should I pre-serialize the Map and store it as a Text in my model?

Regards, Tobias

Yesod Web Framework member

yeah, the error message could be better.

So by default it stores a Map as an embedded object assuming that although your keys are variable from object to object you do know the set of all possible keys (and they are valid mongoDB keys).
I think we have 2 options for allowing arbitrary keys:

  • escape inavlid keys
  • change the behavior (possibly of a newtyped Map) to something different

I am not sure how useful it is to escape invalid keys: that means you have no idea of what your possible keys are ahead of time so the ability to query the object is going to be limited. But on the other hand I don't want to serialize down to a string with MongoDB.

What is your use case? Do you want to query the stored Map keys?

burp commented Aug 29, 2013

As in my case I have no interest in querying the individual map keys directly, I will probably store the map pre-serialized as a simple ByteString. I just need to access it in whole as a Data.Map for further processing.

I'm not sure what the best general solution is. Maybe a remark on how Maps are stored is sufficient.

Yesod Web Framework member

It seems like it might be easiest for persistent to escape the reserved . character and a leading '$' character, but I don't know of any way to directly escape them. There was a patch for . to automatically create a sub-object, but even with that we would need to escape $, but I don't know how to do that.

I can certainly add documentation stating to avoid $ and . or to serialize to JSON.
I might try creating a newtype PersistField instance that helps serialize automatically.

@gregwebs gregwebs added the mongoDB label Aug 4, 2014
@gregwebs gregwebs added the 2.0 label Aug 28, 2014
Yesod Web Framework member

I think the best solution here should be to use a newtype and customize the serialization behavior. I am willing to accept patches for newtypes into persistent-mongoDB.

@gregwebs gregwebs closed this Sep 13, 2014
@gregwebs gregwebs removed the 2.0 label Sep 13, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.