Embedded Update Error #141

dev-inigmas · 2014-07-12T20:10:27Z

After echoing back the "client_id" that was provided by EPF for the parent document, I discovered a non-deterministic error that occurs on updates. (Haven't been able to replicate it for create yet, but that doesn't mean it can't happen...)

I'm assuming that the problem would resolve itself if I included the client_id for every embedded document as well... I'm just reporting this here in hopes that we can work toward a version where client_id isn't required from the server.

Here's the error:

TypeError: Array.prototype.indexOf called on null or undefined

Adding several sub-documents to the parent document seems to increase how often this occurs. So far, I haven't been able to duplicate this when removing many sub-documents.

The text was updated successfully, but these errors were encountered:

ghempton · 2014-07-12T21:42:52Z

Hmm that seems strange. We have several models that use lots of embedded documents. Do you have more of the stack trace handy?

dev-inigmas · 2014-07-12T21:46:16Z

That's all the reason(?) code said from the session.flush(). I'm guessing that your servers pass-through all of the client_ids, though. Right?

Right now, I'm just doing the ids for the parent documents.

Also, do your models with lots of embedded documents have more that one type of embedded document? I noticed that I'm getting this the most with my "Edge" documents. They have lists of "EdgeRequirements" and "EdgeEffects".

ghempton · 2014-07-12T23:15:14Z

Yes they have lots of nested embedded relationships. There are also some
acceptance tests with multiple levels of embedded relationships.

On Sat, Jul 12, 2014 at 2:46 PM, David Patrick notifications@github.com
wrote:

That's all the reason(?) code said from the session.flush(). I'm guessing
that your servers pass-through all of the client_ids, though. Right?

Right now, I'm just doing the ids for the parent documents.

Also, do you models with lots of embedded documents have more that one
type of embedded document? I noticed that I'm getting this the most with my
"Edge" document. It has list of "EdgeRequirements" and "EdgeEffects".

—
Reply to this email directly or view it on GitHub
#141 (comment).

Gordon L. Hempton
http://codebrief.com
360.460.8098

dev-inigmas · 2014-07-13T01:27:46Z

Alright; I setup my server to "pass-through" all of the client ids, revs, etc.... But I still occasionally get that error. Just now, I actually got it while removing a lot of documents. I'm going to see if I can get a stack trace on it. Until then, I'll post my latest request/response/hash.

PUT /api/v1/edges/53c1d7bf97000054aa57aa41:

{
  "edge": {
    "id": "53c1d7bf97000054aa57aa41",
    "client_id": "edge540",
    "client_rev": 2,
    "name": "qwe",
    "rank": "nov",
    "for_ab": false,
    "description": "qwe",
    "module_id": "53c050319700005e2157aa20",
    "reqs": [{
      "id": "edge_req-53c1d7bf97000054aa57aa41-0",
      "client_id": "edge_req541",
      "client_rev": 1,
      "req": "qwe",
      "script": null,
      "edge_id": "53c1d7bf97000054aa57aa41"
    }],
    "effects": [{
      "id": "edge_effect-53c1d7bf97000054aa57aa41-0",
      "client_id": "edge_effect542",
      "client_rev": 1,
      "effect": "qwe",
      "script": null,
      "edge_id": "53c1d7bf97000054aa57aa41"
    }]
  }
}

Response 200:

{
  "edge": {
    "doc_meta": {
      "created": 1405212607575,
      "updated": 1405213787685,
      "created_by": "53bd768f970000004057a9f4"
    },
    "id": "53c1d7bf97000054aa57aa41",
    "name": "qwe",
    "rank": "nov",
    "for_ab": false,
    "effects": [{
      "id": "edge_effect-53c1d7bf97000054aa57aa41-0",
      "client_id": "edge_effect542",
      "client_rev": 1,
      "effect": "qwe",
      "script": null,
      "edge_id": "53c1d7bf97000054aa57aa41"
    }],
    "reqs": [{
      "id": "edge_req-53c1d7bf97000054aa57aa41-0",
      "client_id": "edge_req541",
      "client_rev": 1,
      "req": "qwe",
      "script": null,
      "edge_id": "53c1d7bf97000054aa57aa41"
    }],
    "description": "qwe",
    "module_id": "53c050319700005e2157aa20",
    "client_id": "edge540",
    "client_rev": 2
  }
}

Hash after extractProperty():

{
  "doc_meta": {
    "created": 1405212607575,
    "updated": 1405213787685,
    "created_by": "53bd768f970000004057a9f4"
  },
  "id": "53c1d7bf97000054aa57aa41",
  "name": "qwe",
  "rank": "nov",
  "for_ab": false,
  "effects": [{
    "id": "edge_effect-53c1d7bf97000054aa57aa41-0",
    "client_id": "edge_effect542",
    "client_rev": 1,
    "effect": "qwe",
    "script": null,
    "edge_id": "53c1d7bf97000054aa57aa41"
  }],
  "reqs": [{
    "id": "edge_req-53c1d7bf97000054aa57aa41-0",
    "client_id": "edge_req541",
    "client_rev": 1,
    "req": "qwe",
    "script": null,
    "edge_id": "53c1d7bf97000054aa57aa41"
  }],
  "description": "qwe",
  "module_id": "53c050319700005e2157aa20",
  "client_id": "edge540",
  "client_rev": 2
}

dev-inigmas · 2014-07-13T01:36:21Z

I removed my error handler from the session.flush().then() call. Then, after several edits, I got this:

TypeError: undefined is not a function
    at Ep.Session.reopen.merge (http://bebop.local:9494/dist/deps.js:71930:23)
    at Ep.ChildSession.Ep.Session.extend.merge (http://bebop.local:9494/dist/deps.js:71866:47)
    at apply (http://bebop.local:9494/dist/deps.js:31391:27)
    at superWrapper [as merge] (http://bebop.local:9494/dist/deps.js:30969:15)
    at http://bebop.local:9494/dist/deps.js:70533:40
    at Array.map (native)
    at http://bebop.local:9494/dist/deps.js:70532:31
    at invokeCallback (http://bebop.local:9494/dist/deps.js:33957:19)
    at publish (http://bebop.local:9494/dist/deps.js:33627:9)
    at publishRejection (http://bebop.local:9494/dist/deps.js:34055:7) 

Uncaught Error: Assertion Failed: TypeError: undefined is not a function deps.js:23517
Ember.assert deps.js:23517
RSVP.onerrorDefault deps.js:70077
__exports__.default.trigger deps.js:32921
Promise._onerror deps.js:33645
publishRejection

Any ideas?

dev-inigmas · 2014-07-13T01:41:16Z

Found the source and line number for you, in case that helps:

lib/session/merge.js:49

dev-inigmas · 2014-07-16T19:15:09Z

So.... I'm just about done converting my app over to EPF; but this remains a blocking issue. Please let me know if there's anything I can provide that would help us to find a fix/workaround.

Thanks.

ghempton · 2014-07-16T21:05:48Z

Hmm I would like to get to the bottom of this. You said that this happens
after delete operations, what do the responses look like for those? Seems
weird that model would be null inside of session.merge unless something
weird was being sent down from the serializer.d

On Wed, Jul 16, 2014 at 12:15 PM, David Patrick notifications@github.com
wrote:

So.... I'm just about done converting my app over to EPF; but this remains
a blocking issue. Please let me know if there's anything I can provide that
would help us to find a fix/workaround.

Thanks.

—
Reply to this email directly or view it on GitHub
#141 (comment).

Gordon L. Hempton
http://codebrief.com
360.460.8098

dev-inigmas · 2014-07-16T21:11:06Z

In the original ticket, I thought that this only happened after adding many documents. Lately, I've just been getting this any time there's a change to a list. My long comment from four days ago shows the request/response/afterDeserialization.

ghempton · 2014-07-16T21:15:28Z

What do you mean a change to a list?

dev-inigmas · 2014-07-16T21:29:04Z

I think that the following code examples should illustrate what I mean. In the controller, I sometimes create the embedded models right there and add them. For cases where items are being removed from a list, I'll use removeObject.

Models:

App.Foo = Ep.Model.extend
  bars: Ep.hasMany('bar')

App.Bar = Ep.Model.extend
  foo: Ep.belongsTo 'foo', inverse: 'bars'

App.FooSerializer = App.MyCrazySerializer.extend
  bars: embedded: 'always', inverse: 'foo'

Controller:

App.FooController = Em.ObjectController.extend
  actions:
    addBar: (barData) ->
      foo = @get('content')
      session = foo.get('session')
      bar = session.create 'bar', barData

      foo.get('bars').pushObject(bar)

Route:

App.FooRoute = Em.Route.extend
  actions:
    save: ->
      @get('currentModel.session').flush().then
        ()  => @send('success')
        (r) => @send('failure', r)

ghempton · 2014-07-16T21:55:02Z

Can you try:

foo = @get('content')
      session = foo.get('session')
      bar = session.create 'bar', barData

      foo.get('bars').pushObject(bar)
      bar.set('foo', foo) #added this line

I'm thinking there might be an issue updating the inverse for some reason.

dev-inigmas · 2014-07-16T22:44:56Z

Doesn't seem to help.

So, in another place in my app I'm able to replicate this error much more easily. Note that I only ever edit documents within a child session.

Start with an empty child array in the primary document.
Add an embedded document and flush()
Remove the embedded document then add a new one.
flush() again.

My current theory about this, is that the parent session is being told:

Remove the child document whose id was XXX
Add a new child document whose id is XXX, where XXX is the same as the old child document's id.

I also think that I may be getting this error more deterministically from this section in my app, because the embedded child has a relational Ep.belongsTo relationship defined within it.

ghempton · 2014-07-16T22:56:57Z

Ah so you are re-adding a child with the same id? I think semantically
there are parts of EPF that assume that the removal of an embedded child
means that the child is deleted.

On Wed, Jul 16, 2014 at 3:44 PM, David Patrick notifications@github.com
wrote:

Doesn't seem to help.

So, in another place in my app I'm able to replicate this error much more
easily. Note that I only ever edit documents within a child session.

Start with an empty child array in the primary document.

Add an embedded document and flush()

Remove the embedded document then add a new one.

flush() again.

My current theory about this, is that the parent session is being told:

Remove the child document whose id was XXX

Add a new child document whose id is XXX, where XXX is the same as
the old child document's id.

I also think that I may be getting this error more deterministically from
this section in my app, because the embedded child has a relational
Ep.belongsTo relationship defined within it.

—
Reply to this email directly or view it on GitHub
#141 (comment).

Gordon L. Hempton
http://codebrief.com
360.460.8098

dev-inigmas · 2014-07-16T22:59:05Z

Yes. Removing a Child should delete it. But I've been generating IDs for my embedded objects. These generated id's all follow a similar pattern. Removing an embedded document, then adding a new embedded document caused the new document to have the same ID as the old one.

I'm working on changing that now to see if that resolves things.

ghempton · 2014-07-16T22:59:48Z

That makes sense. I hope this resolves it as well.

On Wed, Jul 16, 2014 at 3:59 PM, David Patrick notifications@github.com
wrote:

Yes. Removing a Child should delete it. But I've been generating IDs for
my embedded objects. These generated id's all follow a similar pattern.
Removing an embedded document, then adding a new embedded document caused
the new document to have the same ID as the old one.

I'm working on changing that now to see if that resolves things.

—
Reply to this email directly or view it on GitHub
#141 (comment).

Gordon L. Hempton
http://codebrief.com
360.460.8098

dev-inigmas · 2014-07-16T23:43:38Z

Hmm..... At this point, I'm not seeing any way around having to update every single sub-document in my database to have it's own id.

I was really hoping to avoid needing to do that.

ghempton · 2014-07-17T00:50:02Z

Yeah unfortunately I think that is going to be required for the short term. This could be avoided in the future with additional logic on EPF's end. In our app we have unique id's on our embedded sub-documents.

dev-inigmas · 2014-07-17T03:47:40Z

Well... I'm storing the embedded document ids for one of my Models now. But I'm still getting random errors:

TypeError: undefined is not a function
    at Ep.Session.reopen.merge (http://bebop.local:9494/dist/deps.js:71930:23)
    at Ep.ChildSession.Ep.Session.extend.merge (http://bebop.local:9494/dist/deps.js:71866:47)
    at apply (http://bebop.local:9494/dist/deps.js:31391:27)
    at superWrapper [as merge] (http://bebop.local:9494/dist/deps.js:30969:15)
    at http://bebop.local:9494/dist/deps.js:70533:40
    at Array.map (native)
    at http://bebop.local:9494/dist/deps.js:70532:31
    at invokeCallback (http://bebop.local:9494/dist/deps.js:33957:19)
    at publish (http://bebop.local:9494/dist/deps.js:33627:9)
    at publishRejection (http://bebop.local:9494/dist/deps.js:34055:7)

Not sure what to do now....

ghempton · 2014-07-17T05:09:23Z

Sorry you are having issues. We use embedded records quite extensively, but the embeddedness stops at the rest level (as our backend is relational). I'd be happy to skype/pair etc. with you to get this to work. You are pioneering using a document store with epf.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Embedded Update Error #141

Embedded Update Error #141

dev-inigmas commented Jul 12, 2014

ghempton commented Jul 12, 2014

dev-inigmas commented Jul 12, 2014

ghempton commented Jul 12, 2014

dev-inigmas commented Jul 13, 2014

dev-inigmas commented Jul 13, 2014

dev-inigmas commented Jul 13, 2014

dev-inigmas commented Jul 16, 2014

ghempton commented Jul 16, 2014

dev-inigmas commented Jul 16, 2014

ghempton commented Jul 16, 2014

dev-inigmas commented Jul 16, 2014

ghempton commented Jul 16, 2014

dev-inigmas commented Jul 16, 2014

ghempton commented Jul 16, 2014

dev-inigmas commented Jul 16, 2014

ghempton commented Jul 16, 2014

dev-inigmas commented Jul 16, 2014

ghempton commented Jul 17, 2014

dev-inigmas commented Jul 17, 2014

ghempton commented Jul 17, 2014

Embedded Update Error #141

Embedded Update Error #141

Comments

dev-inigmas commented Jul 12, 2014

ghempton commented Jul 12, 2014

dev-inigmas commented Jul 12, 2014

ghempton commented Jul 12, 2014

dev-inigmas commented Jul 13, 2014

dev-inigmas commented Jul 13, 2014

dev-inigmas commented Jul 13, 2014

dev-inigmas commented Jul 16, 2014

ghempton commented Jul 16, 2014

dev-inigmas commented Jul 16, 2014

ghempton commented Jul 16, 2014

dev-inigmas commented Jul 16, 2014

ghempton commented Jul 16, 2014

dev-inigmas commented Jul 16, 2014

ghempton commented Jul 16, 2014

dev-inigmas commented Jul 16, 2014

ghempton commented Jul 16, 2014

dev-inigmas commented Jul 16, 2014

ghempton commented Jul 17, 2014

dev-inigmas commented Jul 17, 2014

ghempton commented Jul 17, 2014