[RCP-42] Model and Field Resources

[darnjo]

During the RESO Developer Conference, the group approved moving the following items forward for the proposal phase.

---

Background

In order to make server metadata easier to work with, as well as more Data Dictionary friendly and transport agnostic, this proposal establishes requirements on the existing Field Resource and adds a new resource called "Model" to represent models of type "Resource" or "ComplexType."

The term "Resource" is used the same way the Data Dictionary uses it now, but there can be other kinds of models as well so the terminology has been generalized.

This proposal works with both OData Web APIs and RESO Common Format, and in the latter case a non-OData server could advertise their metadata in standard format using the Model, Field, and Lookup Resources.

Field Resource

The following fields will be removed from the Field Resource:

• ResourceName will be deprecated in favor of ModelKey.
• DisplayName will be deprecated in favor of internationalization and locale support being added in [RCP-44] Supporting Multiple Metadata Locales #67 with the DisplayNames field.

Providers MAY still use these fields for backwards compatibility, if needed, but the standard versions of these items MUST also be present in that case. Deprecated items will be treated as local in RESO Analytics.

The following fields will be added to the Field Resource:

• LookupName - Add LookupName to the Field Resource as a nullable string that describes which sets of standard or local lookups are referenced in the Lookup Resource.
  • The Data Dictionary currently defines lookup names, such as StandardStatus, but LookupName will not be validated against the Data Dictionary Lookup Name.
  • If LookupName is present in the Field Resource, there MUST be a corresponding LookupName entry in the Lookup Resource.
• Type - Edm.String, non-nullable. Examples: org.yourOrgName.CustomTypeABC123, org.reso.metadata.StandardStatus, Property.
• CollectionYN - Edm.Boolean, default false.
• ExpandableYN - Edm.Boolean, default false.
• NullableYN - Edm.Boolean, default true.
• Length - Edm.Int32, Length of Edm.String data types. Nullable.
• Precision and Scale - Edm.Int32, supports both Edm.Decimal, Edm.Double types and follow the IEEE 754 definitions. Nullable.
• ReadableYN, OrderableYN, UpdatableYN, SearchableYN - Edm.Boolean, default false.
  • All Keys MUST have SearchableYN = true
  • ModificationTimestamp fields MUST have SearchableYN = true, OrderableYN = true, and UpdatableYN = false
• Definition - Edm.String, default null.
• DisplayNames - Collection(I18nString), default []. Human-friendly representation of the field name. See [RCP-44] Supporting Multiple Metadata Locales #67.

Model Resource

A new resource called "Model" will be created with the following fields.

• ModelKey - Edm.String, not nullable.
• ModelName - Edm.String, not nullable.
• ModelType - ModelType, Single enumeration of either "EntityType" or "ComplexType".
• DisplayNames - Collection(I18nString), default []. Human-friendly representation of the model name. See [RCP-44] Supporting Multiple Metadata Locales #67.
• Definition - Edm.String, nullable.
• PrimaryKeyFieldKey - Edm.String, nullable.
• ModificationTimestampFieldKey - Edm.String, nullable.
• ReadableYN, InsertableYN, UpdatableYN, DeletableYN - Edm.Boolean, default false.
• ModificationTimestamp - Edm.DateTimeOffset, not nullable.

---

Usage Requirements for RESO Web API Providers

• The Lookup Resource may be supported even if the server is not using the Model and Field Resources, but if so, all three will be present.
• If a server does not support the Model, Field, or Lookup Resources, they should respond with a non-2XX HTTP status. The client would then know to query the OData XML Metadata instead.
• It is expected that servers using the Model and Field Resources will provide the minimum metadata required in OData XML Metadata, i.e. EntityType and Property definitions.
  • If "Property" exists in the Model Resource with Type "Resource," for example, the OData XML Metadata MUST have an EntityType of Property in its default entity container.
  • For those using the Model Resource, if the OData XML Metadata has an EntityType of Property in its default entity container, the provider MUST have a corresponding entry in the Model Resource.
  • All Resource/EntityType definitions MUST also have at least one PrimaryKey, but composite keys are possible. The key(s) MUST be present in the Model Resource and match what's in the EntityType definition.
  • All models of type "Resource" MUST have at least one standard or local field, meaning there will also be entries in the Field Resource and corresponding Property definitions in the OData XML Metadata. This follows from the PrimaryKey requirement of EntityType definitions.
  • RESO requires an exact match between the field names and data types used in the Data Dictionary and OData names and types.
  • For local fields, there MUST be parity between the Field Resource definition and the OData XML Metadata. Data type mappings may be found here. These items also map to JSON types and are part of the RESO Common Format.

---

Discussed in #70

^{Originally posted by SergioDelRioT4Bi January 19, 2023}
I would love to finalize the fields in the Fields Resource and get this fully integrated into the specification.

Currently, we have a set of fields defined in the Data Dictionary: Field Resource

The initial proposal is documented here: Initial Proposal

So, let's discuss what fields need to be added, here are my initial thoughts:

• LookupName (a reference to the Lookup Resource LookupName field)
• DataType (EDMX Data Type)
• Updateable
• Queryable
• Orderable

[SergioDelRioT4Bi]

Hi everyone, I have build out the requested E/R model which can be seen here: https://downloads.t4bi.com/RESO/RESO-Meta-Model.html

Hopefully this will help everyone visualize what we are proposing more easily.

Thank You Ivaan for helping me out in simplifying it!

[cobogeri]

Thanks for the E/R model!

I see a type/naming conflict in "Field" with the E/R Model and the RCP - where the RCP has "Definition" the E/R model has "Definitions"

[SergioDelRioT4Bi]

Well, I did that on purpose as I think the RCP is no longer correct after our discussions.

[ThoughtsFromSLC]

I would like to add to the discussion that MLS Grid requires that OriginatingSystemName is submitted with requests for metadata and Lookups in order to specifically return the results that pertain to that specific MLS the request is based for. The MLS Grid provides data for 25 MLS through one end point and using one bearer token. Requiring OriginatingSystemName ensures the data consumer receives only the metadata and lookups that are valid for that specific MLS. RESO supports the use of OriginatingSystemName for the other resources but not Model, Field, and Lookup. I would like to suggest that support for OriginatingSystemName should be added, or at least made optional.

[darnjo]

Interop requested that we also add Definition to the Lookup Resource.

[SergioDelRioT4Bi]

Has anyone reviewed the model: MetaModel? I would like us to all agree that this is what the resources both already look like and will look like for the new ones and Field.

[alifemove]

Question because this has been the result of very poor query performance to get it to work 'correctly'.
ModificationTimestamp is required orderable, but it's not a unique field, requests that order by a non unique field don't guarantee the same order on subsequent requests so during pagination you could end up with the same record twice or some not at all. Any suggestions on how to deal with this / should it be dealt with?

[grispin]

To support media uploads, we would like to request a boolean field be added to the Field and Model resources to represent the HasStream attribute from the EDMX. This is going to be a required attribute for Add/Edit Media using the OData stream handling. Should likely be 'HasStreamYN' to conform to other boolean fields

[SergioDelRioT4Bi]

To support media uploads, we would like to request a boolean field be added to the Field and Model resources to represent the HasStream attribute from the EDMX. This is going to be a required attribute for Add/Edit Media using the OData stream handling. Should likely be 'HasStreamYN' to conform to other boolean fields

@grispin Is this still required based on the latest proposal? I don't think so, right?

[grispin]

We should add that attribute to keep the Model/Field/Lookup in sync with the EDMX.

[AlexAtBadgersRock]

Regarding ExpandableYN for Field: ideally, the proposal would qualify the suitable values for ExpandableYN in similar fashion to what has been done for ReadableYN, OrderableYN, UpdatableYN, and SearchableYN. And 'expandable field' (really, a navigation property') should be ReadableYN=false (as we are returning the expanded object(s)) and also, OrderableYN=false and UpdatableYN=false.