Metadata for documents for filtering

[peterpeterparker]

A document is a blob in memory, therefore it is not easy to filter and list documents based on their content. Most of the work has to be done in the frontend or by tweaking keys or duplicating data.

To overcome the issue, this weekend idea discussed on Discord, is the addition of a hashmap/hashset or plaintext field for the documents.

export interface Doc<D> {
  key: string;
  data: D;
  owner?: string;
  created_at?: bigint;
  updated_at?: bigint;
  metadata?: ???? // <----- here something new
}

[dennyweiss]

Instead of adding a new artificial metadata field why do we not apply the regex filter to the whole data field?
The only fact that we have to consider is that we have to write regex'es that are applied to json strings. To make this doable we require a query where we can receive the raw data of the data field.

[peterpeterparker]

Instead of adding a new artificial metadata field why do we not apply the regex filter to the whole data field?

This is not possible, I think, because the data field is persisted as a blob or, we would have to decode the blob for each entries on filtering and that would not be that performant I am guessing.

[peterpeterparker]

I've added an optional field called description to documents and assets which accepts string limited to maximal 1024 characters (because it should not be used for other purpose than description - i.e. to avoid developer stringify their objects in this fields instead of using the data).

The description field as also been added to the list functions. The matcher used to be a string, it will now be an object.

matcher {
  key?: string;
  description?: string;
}

Both regex.

[peterpeterparker]

The related PR in the library: junobuild/juno-js#29