Validate the yml file

[joseph-robertson]

Should we "validate" the yml file before starting compute nodes? As part of the sampling portion? This could potentially identify "sync" issues between your yml file and your checkout of OpenStudio-BuildStock.

Things we can check for:

• measure argument names against those in the measure (e.g., timeseries_csv_export arguments)
• upgrade options (Parameter|Option) against options lookup entries (e.g., Elaina had the older "HVAC System Combined" instead of the newer "HVAC System Heat Pump")

@nmerket @ekpresent

[nmerket]

This is a great idea. I think a more general validation would be useful as well. We should check every field and verify that it has a reasonable value (i.e. files exist, has a proper enumeration value), we could set certain fields as required or mutually exclusive.

As I think about this, the best place to run it would be as early as possible. So, that would be on the login node on Eagle, on the local machine on AWS, and before spinning off docker containers for the the local docker version.

I know @rHorsey is interested in this too.

[rHorsey]

I've got a version of it working in the far-behind commercial branch that we froze for LA100, although FYI I don't really plan to merge that back in so much as cherry pick features manually. You can validate ASAP to reduce the need for obnoxious manual checks and I was using a pretty basic library called yamale - see an example schema for eagle. I'm happy to implement this in a couple weeks on a feature branch off of master.

[rHorsey]

This could potentially identify "sync" issues between your yml file and your checkout of OpenStudio-BuildStock.

That's certainly possible to do, but I think that might need capabilities outside of yamale; worthwhile regardless.

[nmerket]

Yeah, I think a schema validation would be a good start. Thanks for the library recommendation. However, we're going to want to validate with some more application specific logic as well.

[nmerket]

@rHorsey This issue keeps coming up as something we'll need probably sooner than later. I know you have some prior work on this that we should at least reference. Will you be able to get to this or should I have someone else take it on?

[rHorsey]

@nmerket Yep - probably later this week, maybe Friday.

[nmerket]

Can we meet for a few minutes about this? I'd like to discuss the different levels of validation we'd like and map it out.

[nmerket]

@rHorsey @joseph-robertson I just sent you a meeting invite to discuss how we're going to tackle this. If anyone else is interested in contributing to that design discussion, let me know and I'll add you to the invite.

[nmerket]

#68 is a good start. Ry will get it ready to merge 7/2. Then for the rest of the features below we will create our own branches from that stub.

To Do List:

• Yamale validation. Generalize to base class. @rHorsey
• Check if directories exist
  • output_directory should not exist @rHorsey
  • buildstock_directory should exist @nmerket
  • project_directory should exist within buildstock_directory @nmerket
• Check for data in S3 and check database and tables. All should not exist. @rajeee
• Verify options exist in options_lookup.tsv. @rajeee
• Run integrity checks right before sampling as part of sampling job (later). @nmerket
• Check measure arguments exist for @joseph-robertson
  • TimeseriesCSVExport
  • ApplyUpgrade verify that there are enough measure arguments in the measure.
• Warn on extra measures in folder (commercial, at least at first) @rHorsey

[rHorsey]

Also adding yaml version - starting with 0.1 - no patches a la docker.

[nmerket]

I think we're substantially done here.