New Validate Database tool fails databases that previously passed validation

[alwunder]

I am testing the new Validate Database tool and have run into a few snags. In particular, databses hat previously passed validation are now having problems with "Missing or mis-defined fields" on annotation classes that should be ignored by the tool:

2.1 Missing required elements
    Feature class GMContactsAndFaults_Anno
2.2 Missing or mis-defined fields
    GMMapUnitPolys_Anno missing field MapUnit
    GMMapUnitPolys_Anno missing field IdentityConfidence
    GMMapUnitPolys_Anno missing field Label
    GMMapUnitPolys_Anno missing field Symbol
    GMMapUnitPolys_Anno missing field DataSourceID
    GMMapUnitPolys_Anno missing field GMMapUnitPolys_Anno_ID

and some other classes at Level 3:

3.1 Missing or mis-defined fields
    CartographicPoints missing field LocationConfidenceMeters
    CartographicPoints missing field PlotAtScale
    CartographicPoints missing field StationsID
    CartographicPoints missing field MapUnit
    CartographicPoints missing field LocationSourceID
    MapUnitOverlayPolys missing field Type
    GMMapUnitOverlayPolys_Anno missing field Type
    GMMapUnitOverlayPolys_Anno missing field IdentityConfidence
    GMMapUnitOverlayPolys_Anno missing field Label
    GMMapUnitOverlayPolys_Anno missing field Symbol
    GMMapUnitOverlayPolys_Anno missing field DataSourceID
    GMMapUnitOverlayPolys_Anno missing field GMMapUnitOverlayPolys_Anno_ID

Some of these are custom classes that follow the spirit of GeMS but don't need all the fields that GeMS prescribes, e.g., the feature class CartographicPoints is modeled after CartographicLines and has no use for the "LocationConfidenceMeters" attribute. And why are some caught at Level 2 and some at Level 3? I wonder if this is due to something missing that I could add to my GeMS_Definition.py file? I think it's more likely that it has something to do with the way the script is reading and inventorying the database and its classes.

[ethoms-usgs]

Yeah, annotation classes should be completely ignored by the tool. There is one step early on in the flow where they should be excluded for consideration, but I will add another screening.

Of course, there should be room for CartographicPoints. I have added an exception.

[alwunder]

I have tested the latest tool (v2.11.2) on the same database and produced the following errors in the report:

Level 2 errors
2.1 Missing required elements
Feature class GMContactsAndFaults_Anno
2.2 Missing or mis-defined fields
GMMapUnitPolys_Anno missing field MapUnit
GMMapUnitPolys_Anno missing field IdentityConfidence
GMMapUnitPolys_Anno missing field Label
GMMapUnitPolys_Anno missing field Symbol
GMMapUnitPolys_Anno missing field DataSourceID
GMMapUnitPolys_Anno missing field GMMapUnitPolys_Anno_ID
Level 3 errors
3.1 Missing or mis-defined fields
NaturalResourcePoints missing field StationsID
MapUnitOverlayPolys missing field Type
GMMapUnitOverlayPolys_Anno missing field Type
GMMapUnitOverlayPolys_Anno missing field IdentityConfidence
GMMapUnitOverlayPolys_Anno missing field Label
GMMapUnitOverlayPolys_Anno missing field Symbol
GMMapUnitOverlayPolys_Anno missing field DataSourceID
GMMapUnitOverlayPolys_Anno missing field GMMapUnitOverlayPolys_Anno_ID

So it seems the Anno classes are still getting mixed up in the error flagging. Maybe this also has something to do with the way I have named the classes? I see that the Pro tool searches for and treats extensions to the GeMS schema differently than the old ArcMap tool did in the past.

UPDATE: Further study leads me to believe that the error is coming from the way that the tool looks for the required elements in a feature dataset. I have a feature dataset called CartographicElements. It contains the annotation classes that label the features within the GeologicMap feature dataset: GMContactsAndFaults_Anno, GMMapUnitPolys_Anno, and GMMapUnitOverlayPolys_Anno. The script seems to be treating these as if they are the containers for these feature types within the CE feature dataset (thus the error of missing required fields) instead of ignoring them altogether since they are annotation classes. Perhaps it's the order/timing in which the test for whether a class is annotation is conducted? This might also explain why some are flagged at Level 2 and some at Level 3.

UPDATE 2: I am zeroing in on the find_topology_pairs portion of the code in Rule 2.1. The identification of "possible pairs" seems to lead to the first error of the "missing" required elements Feature class GMContactsAndFaults_Anno

[ethoms-usgs]

Try v2.11.3. I reproduced the error with annotation classes and then tracked it down. Should be good now.

[alwunder]

Excellent! The script is now working on Annotation, thanks also to the refresh of the GeMS_utilityFunction.py suggested in Issue #68. But now I have a new problem, which I will raise in a new issue to keep things clean.

Thanks, Evan!