Mongo async support #1825

marekzebrowski · 2017-01-24T19:09:08Z

as discussed
sgrouples/rogue-fsqio#15
mongo async support for lift 3.1
pre-requisite for porting rogue-async to lift (separate project):

https://github.com/sgrouples/rogue-fsqio/tree/lift31

Shadowfiend · 2017-01-24T19:14:49Z

! 💯 Don't know that I can review this, but this souuunds pretty sweet :)

eltimn

I just have the one question. I'll test it out locally in the next few days. Thanks for sending this over.

eltimn · 2017-01-24T19:29:34Z

...nce/mongodb-record/src/main/scala/net/liftweb/mongodb/record/field/MongoCaseClassField.scala

+        if (elem.isInstanceOf[org.bson.Document]) {
+          setFromDocumentList(jlist.asInstanceOf[java.util.List[org.bson.Document]])
+        } else {
+          setBox(Full(jlist.asScala.toList.asInstanceOf[MyType]))


Is the toList call here still needed?

eltimn · 2017-01-24T19:29:57Z

persistence/mongodb-record/src/main/scala/net/liftweb/mongodb/record/field/MongoListField.scala

+          if(elem.isInstanceOf[org.bson.Document]) {
+            setFromDocumentList(jlist.asInstanceOf[java.util.List[org.bson.Document]])
+          } else {
+            setBox(Full(jlist.toList.asInstanceOf[MyType]))


Another toList call

unfortunately it is needed to match override type MyType = List[ListType] as jlist is java list and we need Scala List[ListType]. Same thing with previous toList

I thought I saw an asScala call before the toList call. The previous one has an asScala call before the toList call, which, I would think, would already have converted it to a Scala List.

well - maybe it can be done better - I'm open for suggestions. when I tried in scala REPL:

scala> :paste // Entering paste mode (ctrl-D to finish) val l = new java.util.ArrayList[Int](2) l.add(1) l.add(2) import scala.collection.JavaConverters._ l.asScala // Exiting paste mode, now interpreting. l: java.util.ArrayList[Int] = [1, 2] import scala.collection.JavaConverters._ res6: scala.collection.mutable.Buffer[Int] = Buffer(1, 2)

I think I understand now. asScala returns a scala.collection.mutable.Buffer in this instance, which still needs to be converted to List.

Anyway, this should be good then. Thanks.

If I could go through and change every concrete List declaration in Lift to Seq, I would sooo do it <_<

eltimn

I was able to take a closer look. Looking good, just a few minor things and a few questions.

eltimn · 2017-01-28T15:17:26Z

persistence/mongodb-record/src/main/scala/net/liftweb/mongodb/record/MongoMetaRecord.scala

+    useCollAsync { coll =>
+      val p = Promise[BaseRecord]
+      val doc: Document = inst.asDocument
+      val options = new UpdateOptions().upsert(true)


Probably want to use the passed in parameter here.

This looks to be the only thing not addressed. The upsert parameter should be used here.

@marekzebrowski I think this was the only thing left, do you mind if I just go ahead and update the branch?

eltimn · 2017-01-28T15:19:24Z

persistence/mongodb-record/src/main/scala/net/liftweb/mongodb/record/MongoMetaRecord.scala

+    * replaces document with new one with given id. if `upsert` is set to true inserts new document
+    * in similar way as save() from sync api
+    */
+  def replaceOneAsync(inst: BaseRecord, upsert: Boolean, concern: WriteConcern) = {


What do you think of adding default values for upsert (true) and concern (ACKNOWLEDGED)?

Also, please add an explicit return type.

eltimn · 2017-01-28T15:20:13Z

persistence/mongodb-record/src/main/scala/net/liftweb/mongodb/record/MongoMetaRecord.scala

+      val doc: Document = inst.asDocument
+      val options = new UpdateOptions().upsert(true)
+      val filter = new org.bson.Document("_id", doc.get("_id"))
+      coll.replaceOne(filter, doc, options, new SingleResultCallback[UpdateResult] {


Should we be calling before/afterSave and resetDirty here, like the other functions?

eltimn · 2017-01-28T15:31:22Z

persistence/mongodb-record/src/main/scala/net/liftweb/mongodb/record/field/MongoListField.scala

@@ -147,6 +160,11 @@ class MongoListField[OwnerType <: BsonRecord[OwnerType], ListType: Manifest](rec
  // set this field's value using a DBObject returned from Mongo.
  def setFromDBObject(dbo: DBObject): Box[MyType] =
    setBox(Full(dbo.asInstanceOf[BasicDBList].toList.asInstanceOf[MyType]))
+
+  def setFromDocumentList(list: java.util.List[Document]): Box[MyType] = {
+    throw new RuntimeException("Warning, , setting Document as field with no converstion, probably not something you want to do")


Typo: converstion

eltimn · 2017-01-28T15:34:02Z

persistence/mongodb-record/src/test/scala/net/liftweb/mongodb/record/MongoFieldSpec.scala

+      val map = Map("a" -> "4", "b" -> "5", "c" -> "6")
+      val doc = new Document(map)
+      rec.mandatoryStringMapField.setFromDocument(doc)
+      rec.mandatoryStringMapField.get must_== map


Codacy complained about this get call here. It must think it's being called on an Option. Changing it to value should shut it up.

eltimn · 2017-01-28T15:36:28Z

persistence/mongodb/src/main/scala/net/liftweb/mongodb/MongoAsync.scala

+  */
+  private[this] val dbs = new ConcurrentHashMap[ConnectionIdentifier, (MongoClient, String)]
+  val codecRegistry = CodecRegistries.fromRegistries(com.mongodb.MongoClient.getDefaultCodecRegistry(),
+    CodecRegistries.fromCodecs(new LongPrimitiveCodec, new IntegerPrimitiveCodec)


Can you explain why the need to add these 2 codecs? Does it have to do with Java/Scala differences?

Also, should we make the registry customizable?

I'm still wrapping my head around the new java driver and the async support.

without that codecs old-style (2.x with BasicDbObject) fails to work. That is caused because default com.mongodb.async.client.MongoClientImpl.getDatabase does not include the same codec registry as old com.mongodb.MongoClient.getDatabase that leads to runtime errors in serialization/deserialization.
Yes, it could be made customizable - in Mongo driver it is.

limit scope of privates in MongoAsync CR fixes use upsert parameter

marekzebrowski · 2017-02-27T07:34:42Z

please go ahead, I don't mind 2017-02-20 11:52 GMT+01:00 Tim Nelson <notifications@github.com>:

…

***@***.**** commented on this pull request. ------------------------------ In persistence/mongodb-record/src/main/scala/net/liftweb/mongodb/record/ MongoMetaRecord.scala <#1825 (comment)>: > @@ -280,6 +292,33 @@ trait MongoMetaRecord[BaseRecord <: MongoRecord[BaseRecord]] } /** + * replaces document with new one with given id. if `upsert` is set to true inserts new document + * in similar way as save() from sync api + */ + def replaceOneAsync(inst: BaseRecord, upsert: Boolean, concern: WriteConcern) = { + useCollAsync { coll => + val p = Promise[BaseRecord] + val doc: Document = inst.asDocument + val options = new UpdateOptions().upsert(true) @marekzebrowski <https://github.com/marekzebrowski> I think this was the only thing left, do you mind if I just go ahead and update the branch? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1825 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAqHLIQGkmUxyBL1oNNUUqRyOkZ_Aaaoks5reXCGgaJpZM4LsqRn> .

-- Marek Żebrowski

eltimn · 2017-03-05T11:44:16Z

Since this branch is on the sgrouples org and I don't have access to modify it, I'm going to close this PR and open a new one.

eltimn reviewed Jan 24, 2017

View reviewed changes

eltimn reviewed Jan 28, 2017

View reviewed changes

farmdawgnation added this to the 3.1 milestone Feb 2, 2017

Mongo async support

00fd3d2

limit scope of privates in MongoAsync CR fixes use upsert parameter

marekzebrowski force-pushed the mongo-async-31 branch from 0bba74e to 00fd3d2 Compare February 6, 2017 20:34

marekzebrowski mentioned this pull request Feb 6, 2017

non-blocking mongo support for rogue - are you interested ? foursquare/fsqio#13

Open

eltimn closed this Mar 5, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mongo async support #1825

Mongo async support #1825

marekzebrowski commented Jan 24, 2017 •

edited

Loading

Shadowfiend commented Jan 24, 2017

eltimn left a comment

eltimn Jan 24, 2017

eltimn Jan 24, 2017

marekzebrowski Jan 25, 2017

eltimn Jan 25, 2017

marekzebrowski Jan 25, 2017

eltimn Jan 25, 2017

Shadowfiend Jan 25, 2017

eltimn left a comment

eltimn Jan 28, 2017

eltimn Feb 6, 2017

eltimn Feb 20, 2017

eltimn Jan 28, 2017

eltimn Jan 28, 2017

eltimn Jan 28, 2017

eltimn Jan 28, 2017

eltimn Jan 28, 2017

marekzebrowski Feb 5, 2017

marekzebrowski commented Feb 27, 2017 via email

eltimn commented Mar 5, 2017

Mongo async support #1825

Mongo async support #1825

Conversation

marekzebrowski commented Jan 24, 2017 • edited Loading

Shadowfiend commented Jan 24, 2017

eltimn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

eltimn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marekzebrowski commented Feb 27, 2017 via email

eltimn commented Mar 5, 2017

marekzebrowski commented Jan 24, 2017 •

edited

Loading