[Ivory Coast] Additional tables/fields/relations to sync

[josh-griffin]

Epic: sussol/org-issues#18
Related: sussol/msupply#2544 - sussol/msupply#2523 - sussol/org-issues#6

UPDATE: Summary comment here

Description:

We need to add additional tables/fields/relations for the new features being adding in sussol/org-issues#18

Requirements

Tables to add:

• Period/Schedule [ Still up in the air currently - issue: sussol/msupply#2523
• Reason - added in this issue: sussol/msupply#2544

Fields to add:

• masterList.programSettings sussol/org-issues#6
• masterList.isProgram sussol/org-issues#6

Relations

• Requisition.programId -> masterlist.ID
• requisition.periodId -> period.ID [ see above, may need to wait on this? ]
• stocktake.programId -> masterlist.ID

Comments:

I think these all amount to relatively small changes, not needing to be broken down further? Once someone is assigned that can evaluate if it needs to be.

[josh-griffin]

@andreievg @wlthomson @Chris-Petty

To create a requisition by program we need:

• A program: Determined by masterList.isProgram=True. If the program is to be used for a certain store - a store has a tag in [store]tags. if (masterList.programSettings.storeTags['store tag'] - then this store uses this program.
• An order type: Within masterList.programSettings.storeTags["Store tag"] is an orderTypesfield, an array oforderType` objects.
• A period, determined by the storeTag objects field periodScheduleName which points to a periodSchedule object, which is related to many period objects.

Questions

• Syncing [store]tags: Are we able to just send the mobile site it's own tags and store them in settings? Or assign to a name? This will still have to be updated during sync cycles.
• Handling programSettings/storeTag/orderType objects for sync:
  • They will only ever be pulled, never modified.
  • programSettings: Essentially a join table between a masterList and many storeTags.
  • storeTags:
    • Has a name (The key in programSettings.storeTags) which needs to be unique only for a certain masterList
    • Can either use a compound PK of (masterList.ID, storeTag.name) or can just determine which storeTag object to update when syncing, by iterating through storeTag objects related to the masterList and update the one with the correct name. I think composing a PK (not exactly supported by realm as far as I can tell, but just concat a string).
  • orderType:
    • Should be unique for a store tag. Currently enforced in UI on desktop (pending PR), but as these objects are stored in an array: masterList.programSettings.storeTags.someTag.orderTypes they're not strictly enforced. Not a requirement, but makes life easier [ better user experience.. selecting an order type for a requisition in a drop down with non-distinct names, or disappearing order types ]. Also updating would be a nightmare if they're not unique.
    • Again I think composing a PK of (storeTag.ID,orderType.name) is the best option, but just assigning an arbitrary PK and updating the orderType with a name match for a certain storeTag.orderTypes would work also.

.... or we just store programSettings as a string..? The only issue is finding which masterList objects are usable by the current store as a program. The query would take a bit, having to parse each programSettings, unless we set isProgram to true only if a masterList.programSettings.storeTags contains a key which matches a [store]tag of the current store..? Then we can skip storeTag and programSettings all together and only have a orderType related to a particular masterList

[Chris-Petty]

@andreievg @wlthomson @Chris-Petty

To create a requisition by program we need:

• A program: Determined by masterList.isProgram=True. If the program is to be used for a certain store - a store has a tag in [store]tags. if (masterList.programSettings.storeTags['store tag'] - then this store uses this program.
• An order type: Within masterList.programSettings.storeTags["Store tag"] is an orderTypesfield, an array oforderType` objects.
• A period, determined by the storeTag objects field periodScheduleName which points to a periodSchedule object, which is related to many period objects.

Questions

• Syncing [store]tags: Are we able to just send the mobile site it's own tags and store them in settings? Or assign to a name? This will still have to be updated during sync cycles.

In hindsight I think we should have just done a store table in mobile. Next iteration of mobile we can address 😀. So without doing a store table, saving them into settings is probably best/only way. Unless tags were on [name] instead.

• Handling programSettings/storeTag/orderType objects for sync:

  • They will only ever be pulled, never modified.

  • programSettings: Essentially a join table between a masterList and many storeTags.

  • storeTags:

    • Has a name (The key in programSettings.storeTags) which needs to be unique only for a certain masterList
    • Can either use a compound PK of (masterList.ID, storeTag.name) or can just determine which storeTag object to update when syncing, by iterating through storeTag objects related to the masterList and update the one with the correct name. I think composing a PK (not exactly supported by realm as far as I can tell, but just concat a string).

The above as I understand will come in as an object and be saved as such in the masterList record, so not much other than tweaking that into incomingSyncUtils.js. If you want convenience, adding some getters to the schema would be a good way. e.g. get isProgramActive() { // check if storeTags match my store's tags }

• orderType:

  • Should be unique for a store tag. Currently enforced in UI on desktop (pending PR), but as these objects are stored in an array: masterList.programSettings.storeTags.someTag.orderTypes they're not strictly enforced. Not a requirement, but makes life easier [ better user experience.. selecting an order type for a requisition in a drop down with non-distinct names, or disappearing order types ]. Also updating would be a nightmare if they're not unique.
  • Again I think composing a PK of (storeTag.ID,orderType.name) is the best option, but just assigning an arbitrary PK and updating the orderType with a name match for a certain storeTag.orderTypes would work also.

Confused, shouldn't all this be happening server side and just syncing (i.e. already decided by desktop implementation)?

.... or we just store programSettings as a string..? The only issue is finding which masterList objects are usable by the current store as a program. The query would take a bit, having to parse each programSettings, unless we set isProgram to true only if a masterList.programSettings.storeTags contains a key which matches a [store]tag of the current store..? Then we can skip storeTag and programSettings all together and only have a orderType related to a particular masterList

String we should probably avoid. As mentioned earlier it'll come in as JSON and be parsed to a JS object which is way more useful and I don't see much difficulty in it.
isProgram being set based on store tag matchup I don't like, having the same records on mobile different from the server isn't good. A similar approach would be adding a boolean on the masterListNameJoin. In both cases we've sort of denormalised, though.

masterList.orderTypes would be a fun getter to write. Maybe something like:

  get orderTypes() {
    if (!this.isProgram) return [];
    const storeTags = Object.entries(this.programSettings.storeTags);
    const orderTypes = storeTags.reduce(
      // Use JS Sets to get distinct values
      (types, currentTag) => Array.from(new Set([...types, ...currentTag.orderTypes])),
      [],
    );

    return orderTypes;
  }

[josh-griffin]

The above as I understand will come in as an object and be saved as such in the masterList record, so not much other than tweaking that into incomingSyncUtils.js. If you want convenience, adding some getters to the schema would be a good way. e.g. get isProgramActive() { // check if storeTags match my store's tags }

From my understanding all types must resolve to some scalar with realm and there is no ability to use an 'object' type. As program settings has many nested objects we need to resolve the way in which it is saved. I could be totally wrong? But even if realm has introduced this recently, we're still using an old realm version as far as I know (Also, it's probably behind a paywall, right?)

Confused, shouldn't all this be happening server side and just syncing (i.e. already decided by desktop implementation)?

Obviously my ramblings weren't very clear, sorry! What I mean is, if we use storeTag/orderType etc objects in realm, they'll need a PK, which they don't have as they're stored in an object field in 4D

[andreievg]

@Chris-Petty I am pretty sure that we can't store JSON in realm.

I am fine with storing it as string.

As for extracting program info from master list program settings, seem pretty simple to me

For each store tag, see if key exists in programSettings, return programSettings.storeTag[tag] (first matching)

[Chris-Petty]

Wasn't suggesting to store JSON in Realm sorry, meant that we should pass the JSON and actually store it as JS object in Realm... which you can't do either. Realm just no objects, sadness. Gee I guess store it as a string but you'll have to parse it every time you want something like masterList.programSettings.storeTags[tag].

Where's mah JSONB 😢

[Chris-Petty]

One of the very first issues on realm-js haha realm/realm-js#8

They basically suggest doing a string too.

[josh-griffin]

yeah.. you could potentially make a 'generic object' type. Possibly worth investing time into, but probably not. Just string it for now.

[Chris-Petty]

Sweet. So that eliminates all the PK stuff; updates from central server will just send a whole new string for whatever changes were made?

For any getters what not, just parsing it and writing code using the object should be fine. Will only ever be a few programs so hopefully won't be much overhead.

[josh-griffin]

@wlthomson @Chris-Petty @andreievg

So to summarize, after me wasting everyone's time:

Stores will be sent to mobile: If the store ID matches the mobiles ID, store the tags in settings. Throw everything else out

Tables to add:

• Period (ID, startDate, endDate, name, periodScheduleID
• PeriodSchedule (ID, name)
• Options (ID, title, type) [ Don't store cashOutReasons - only types stockTakeLineAdjustment & requisitionLineVariance ]

Fields to add:

• masterList.programSettings (Object stored as a string - should have some nice getters/setters) sussol/org-issues#6
• masterList.isProgram sussol/org-issues#6
• StocktakeItem.optionsId
• requisition.optionsId

Relations

• Requisition.programId -> masterlist.ID
• requisition.periodId -> period.ID [ see above, may need to wait on this? ]

Question ? :
stocktake.programId -> masterlist.ID
Will have to implement this on desktop also