Skip to content

Feature: on the fly HMIS version mapping

Jump to bottom
Eric Jahn edited this page Apr 13, 2018 · 9 revisions

issue id: https://github.com/servinglynk/hmis-lynk-open-source-docs/issues/474 migration architecture

General Description

For the purposes of longitudinal reporting and normalizing data for upload or real-time use into other systems, it is often necessary to translate data values to a previous or later version of the HMIS Data Standard. For example, someone may need to have a 2015 data standard's client gender data "upconverted" to the equivalent 2016 version of gender (which has different gender categories). HUD publishes the year-to-year version mapping for us to use in this conversion. Since HMISLynk stores data in its original version specific format, to maintain exact fidelity to the originally collected data, on-demand conversion of any data element stored for a homeless client migration should be provided, both in the APIs and in the bulk download formats. This also has the side effect of turning HMISLynk into a conversion utility, which is always in great demand in the HMIS practitioner field.

Implementation

An API flag (perhaps a url query parameter) for "map-to-year" with a version specific year value. So: GET .../v2015/clients/{client_id}"?map-to-rear=2014" requested from the v2015 APIs would "downconvert" that particular element to v2014 in the response.

  • migrations would not be persisted in any way: they are simply represented within API responses or serialized output formats (HMIS CSV or XML)

Additional thoughts:

  • Daisy-chaining conversions. Since versions are typically convertible up or down one major year version, the process would convert (under the hood) in two steps to go from v2014 -> v2016
  • There would be a standard error code in an unconvertible situation, say, where the data element no longer exists in the target version.
  • we already have a reporting solution for longitudinal studies: the Big Data Warehouse has all the schema versions in a single table, with a column for the HMIS version (2010, 2014, etc.). It is possible we could also store migrated values, but that would be for a specific customer view, once requested. If such a view were to become popular, we could continue working on it for all customers.
  • The migration code may be a good use of Amazon Lamda serverless processing, since it would be mostly latent.
Clone this wiki locally