In upgrade_metadata index primary keys should be converted to text

[npatki]

Environment Details

• SDV version: 1.0.0b0
• Python version: 3.7
• Operating System: Linux (Colab Notebook)

Background

I have the following metadata (written in the old format, pre-SDV 1.0):

{
  "primary_key": "user_id",
  "fields": {
    "user_id": { "type": "id", "subtype": "integer" },
    ...
  }
}

This conveys that the user id is an index that increments 0, 1, 2, ...

Observed

When I call updgrade_metadata, I get the following:

{
  "primary_key": "user_id",
  "columns": {
    "user_id": { "sdtype": "numerical" },
    ...
  }
}

Technically this works, but it is weird to make this a "numerical" sdtype because it's not actually a statistically valid numerical column. (taking an average, correlation, etc. of this column does not make sense).

Expected

Instead, I propose the upgrade_metadata script should instead make this a "text" sdtype with a Regex that generates indices. This follows our expectation: ID columns will have structured text.

{
  "primary_key": "user_id",
  "columns": {
    "user_id": { "sdtype": "text", "regex_format": "\d{30}" },
    ...
  }
}

When applied to an int column, the synthesizer will return integer values that increase with every row: 0, 1, 2, ... (up to a max of 10^30 rows, which is enough for most purposes).