Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Map fields in persistent-mongoDB #149

Closed
burp opened this Issue · 4 comments

2 participants

@burp

Hey,

I think this is not intended behaviour, so I'll report it as a bug: when one stores a Map, for example with Text keys, and the keys contain special characters like dots, which are not allowed in mongodb keys, one ends up with a failure: "WriteFailure 12527 not okForStorage".

I'm not sure what the best solution to this is. Serialize the map into a single string like for SQL databases, or is it up to me to not use keys with dots? Should I pre-serialize the Map and store it as a Text in my model?

Regards, Tobias

@gregwebs
Owner

yeah, the error message could be better.

So by default it stores a Map as an embedded object assuming that although your keys are variable from object to object you do know the set of all possible keys (and they are valid mongoDB keys).
I think we have 2 options for allowing arbitrary keys:

  • escape inavlid keys
  • change the behavior (possibly of a newtyped Map) to something different

I am not sure how useful it is to escape invalid keys: that means you have no idea of what your possible keys are ahead of time so the ability to query the object is going to be limited. But on the other hand I don't want to serialize down to a string with MongoDB.

What is your use case? Do you want to query the stored Map keys?

@burp

As in my case I have no interest in querying the individual map keys directly, I will probably store the map pre-serialized as a simple ByteString. I just need to access it in whole as a Data.Map for further processing.

I'm not sure what the best general solution is. Maybe a remark on how Maps are stored is sufficient.

@gregwebs
Owner

It seems like it might be easiest for persistent to escape the reserved . character and a leading '$' character, but I don't know of any way to directly escape them. There was a patch for . to automatically create a sub-object, but even with that we would need to escape $, but I don't know how to do that.

I can certainly add documentation stating to avoid $ and . or to serialize to JSON.
I might try creating a newtype PersistField instance that helps serialize automatically.

@gregwebs gregwebs added the mongoDB label
@gregwebs gregwebs added the 2.0 label
@gregwebs
Owner

I think the best solution here should be to use a newtype and customize the serialization behavior. I am willing to accept patches for newtypes into persistent-mongoDB.

@gregwebs gregwebs closed this
@gregwebs gregwebs removed the 2.0 label
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.