Add the cron-project-hierarchy DMS task definition.

[ashercodeorg]

As the table has value values that exceed the column length imposed by Redshift, we are unable to import the column into Redshift in a straightforward manner via a DMS task. Thus, we remove it.

Given our removal of value, we remove updated_ip so as to allow placing the table in pegasus rather than pegasus_pii (this column is presumably not useful in Redshift).

[davidsbailey]

How are you planning to determine whether a row represents a channel-backed level (as opposed to a standalone project)? The hidden field in the value column is where this bit lives today. The two ways I could see are (1) do a migration and add a new column, or (2) have redshift key off of whether "hidden": true or "hidden": "true" appear in the value field. Removing this column suggests the former, just wanting to confirm.

[ashercodeorg]

It seems that (1) is the path of least resistance. Especially as I envision it might be useful to have an index on this (new) column in the production DB.

[ashercodeorg]

Related: Do we have plans to back-populate storage_apps.project_type? Should I own doing so?

[davidsbailey]

That would be great. @poorvasingal told me she thought you were doing the backfill of project_type, but if that's news to you then it sounds like we may be under-communicating on who's doing what on this project. The only thing I've been asked to do is populate project_type for new projects.

[davidsbailey]

Yes, it seems like some DB query time could be saved by adding an index on that new column too.

[ashercodeorg]

SGTM.

[poorvasingal]

@ashercodeorg - If you could own backfilling the data, that'd be great. I'm trying to minimize the work Dave needs to put in before June 1, so we can meet the tight deadline for shipping the project gallery.

[ashercodeorg]

As this PR is independent of ongoing schema changes to the storage_apps table, it is being merged.