Does spraypaint support incremental loading?

[k1w1]

If I define a hasOne relationship like:

const Employee = ApplicationRecord.extend({
  static: {
     jsonapiType: 'employees',
  },
  attrs: {
     name: attr(),
    manager: hasOne(),
  },
});

I want to be able to fetch an employee, and the later, optionally fetch the manger, e.g.

var {e} = await Employee.first()
var {m} = await e.manager()
console.log(m.name)

Spraypaint doesn't seem to support following links that JSON:API returns in the original request. The ability to load sub-records incrementally is one of the great promises of JSON:API, but it seems that in the current implementation I need to fetch all the data in my original request, using include('manager').

Am I missing how to do incremental loading?

[avioli]

You are correct - you cannot fetch incrementally and have to use .includes().

I do it by adding a method on the Model to ensure the relationship is there:

const Employee = ApplicationRecord.extend({
  static: {
     jsonapiType: 'employees',
  },
  attrs: {
     name: attr(),
     manager: hasOne(),
  },
  methods: {
    async ensureManager() {
      // Will throw if this.manager is unset, like when creating a new Employee()!
      // Ensure the manager has id and return if there are any attributes
      if (!this.manager.id || Object.keys(this.manager.attributes).length > 0) return;
      this.manager = await Manager.find(this.manager.id).data;
    },
  },
});

So you do:

const e = (await Employee.first()).data;
await e.ensureManager();
console.log(e.manager.name)

You could batch create these helper methods, iterating over your relationships and creating a method for each, but it is quite abstract and less readable (too DRY):

const rels = {
  manager: hasOne(),
};

const Employee = ...

Object.keys(rels).forEach((key) => {
  const klass = Employee.attributeList[key].klass;
  Object.defineProperty(Employee.prototype, `ensure${key[0].toUpperCase()}${key.slice(1)}`, {
    value: async function() {
      if (!this[key].id || Object.keys(this[key].attributes).length > 0) return;
      this[key] = (await klass.find(this[key].id)).data;
    },
  });
});

[k1w1]

Thanks for the code.

I realized later than having to await every time a relationship is accessed has a performance impact that is probably not desirable. Your approach allows you to accept that cost when you know the relationship is probably not already loaded.

[avioli]

I think it is not the await, but the network request itself that is slow, which will happen only once, but your milage might vary.

[k1w1]

AFAIK simply using await will add a meaningful delay which matters if the code is being called frequently - even if the related record (manager in this case) is actually already loaded. This is because it yields to the micro-task scheduler. So it is best to avoid using await in a case where most of the time no asynchronous action is necessary.

[richmolj]

Thanks for the good conversation here!

I agree we should support eager AND lazy-loading, the latter with links. I previously wrote a (internal) python client that did this, where you could follow the links but also add additional parameters:

const post = await Post.find(1)
// fetch 3 active comments via the link
await post.commentsLink().merge(Comment.where({ active: true }).per(3)).all()
// access as usual
post.comments // [<Comment>, <Comment>, ...]

I would LOVE to see this in Spraypaint but it's not an immediate need for me so I'm not sure when I'd get to it. If anyone would like to PR I'd be happy to help and share some of the python code.

[lucianholt97]

+1 for lazy loading