Expand on Airport File Validation

[erikquinn]

From @erikquinn on December 14, 2016 0:38

A method exists somewhere that I think is called checkFixes() that verifies all fixes used in an airport file are in fact defined. More checks like this to prevent issues like that uncovered in zlsa/atc#760 would be a great step toward guaranteeing airports created by new contributors are done correctly.

Some things to check include:

• that no fix is listed twice in the fixes section
• that all possible keys exist in file (if not used, value can be left as null)
• that elevations are defined where applicable (eg. runways)
• that all instrument procedures:
  • contain all the appropriate sections
  • icao properties match the key of the entry within the collection (see screenshot)
  • contain at least two fixes (one is not sufficient to point the aircraft anywhere) for all possible exitPoint/runway combinations.

^(not allowable)

(feel free to edit and expand on the above list)

Copied from original issue: n8rzz/atc#199

[erikquinn]

From @n8rzz on December 14, 2016 4:16

is the wiki doc still accurate? https://github.com/zlsa/atc/wiki/Airport-format-description

Any validation should start from a validated contract. ie, these are the things we need and the things that need to always be there. First step, verify and/or update this doc to make sure it is complete and accurate.

From that we can extrapolate rules and build a validator(s) of some kind.

[erikquinn]

From @n8rzz on December 14, 2016 4:25

@erikquinn one other thing is when should we validate? should they be validated at run time, every time an airport loads? Should they be validated on PR, with a special airport test suite of some kind? My thought is for the latter. Once an airport is in the project it shouldn't change, much. Its getting it in that is the big hurdle.

[erikquinn]

Ideally, it would be done at airport load. That way, contributors can simply check the console and fix the issues the validator calls out. If they don't get any errors (and our validation is sufficiently thorough), they should have a perfectly good airport. A test set that can be run is useful, but airports might need a different approach, because airport creators/editors should be alerted of issues without having to install npm and run npm run test or something to figure out if they've made a mistake or not.

[erikquinn]

From @n8rzz on December 14, 2016 22:52

@erikquinn good point on npm. I could argue travis ci, but most users aren't going to have access to that. Or wouldn't have access to it until PR time. I like the idea of immediate feedback.

I have some thought on how this might be achieved and I think this would add immediate value to airport contributors. We just need to hammer down requirements. Should we pull in @indianbhaji and some of the other airport contributors for feedback on this one? Or do you already know exactly(ish) what you want?

[erikquinn]

From @n8rzz on December 14, 2016 23:0

also, how much and what kind of validation would you like to see? should we validation that specific fields exist? or should we validate that specific fields exist and are of the correct type and formats?

Obviously the latter will be much more involved but could add more value to a contributor. Perhaps we start with a basic validator that checks shape and required properties (and get that out within a sprint, and then add to it to me more specific in its validations in future sprints?

[erikquinn]

@n8rzz Just the typical hangups that break airports. I agree that @Fechulo and @indianbhaji would be the ones to talk to in that regard; I tagged them in slack to start the conversation. In general, the more the merrier, but all we really need is to point out the not-so-obvious mistakes (like forgetting to define a fix, forgetting to define the type on an arrival stream, etc).

[erikquinn]

From @indianbhaji on December 20, 2016 18:37

The errors that the game produces aren't very helpful occasionally. For example, accidently making a typo in the STAR value of an arrival (ie. typing blagA.blag.EDDM instead of blagB.blag.EDDM) creates an error, which is only useful in one instance. It'd be nice if such an error wasn't as obscure, as all that you know from the error is that there's something wrong with a STAR.

STARs have many aspects - entryPoints, the route, the name etc. Sometimes an error could relate to a STAR but also relate to a typo in the routeing in the arrivals section.

If it's possible, it'd be nice to see which section of an airport file the error falls into, arrivals, SIDs, departures etc.

The fixes debugging to check if a fix is there is perfectly fine - don't do anything to it! (I always add fixes after creating SIDs and STARs!)

[erikquinn]

@indianbhaji the latter example about the how missing fixes will show up as a console warning as soon as the airport loads is what we're talking about adding on to. Because if something is wrong with a star, it'd be great to have a warning tell you up front, rather than having to find out by typing the command and getting a red response. Are there any items you always check when taking a last look through the file that could be automated? For example, "do all of my SIDs have exit points?" / "Do all of my arrival and departure streams use a valid route?" / etc?

[erikquinn]

From @n8rzz on December 20, 2016 22:40

yes, that ^^

Basically, I'd like to take everything you do manually and codify it so that it happens automatically. 😄

[erikquinn]

From @indianbhaji on December 20, 2016 23:3

One simple thing is checking for elevations - not required, but I still
normally check if I have them anyways.

Otherwise, I always double check speed/altitude restrictions to see that
they aren't actually inserted as a fix in a SID or a STAR, but this
normally comes up in the fix warning anyways.

Arrivals routing is always something I check - normally if the entryPoint
is a VOR etc, I always name it after its proper name. (ie.
FIRENZE.FRZ2M.LIPE instead of FRZ.FRZ2M.LIPE) or it's just a case of double
checking the actual STAR name (FRZ2M in the example above)

Otherwise, there's nothing else that can be probably sorted through JS or
any other script.

Unless you can actually do this with JS, the only thing that's a pain is
checking airspace coordinates. Sometimes the airspace border intercepts
each other or isn't defined properly.

[n8rzz]

Due to the scope of this, and all the things that need to be in place for this to be valuable, we've created a separate repo and created an npm package for the validator.

https://github.com/openscope/validator/issues
https://www.npmjs.com/package/@openscope/validator

If there are any questions, issues, feature requests, etc please log them in that repo.