Testing & Validation

[AA-Turner]

Creating an issue with relevant comments from the-scouts/compass-interface-core#1.

Narrative

In Compass, various user roles have access to members in their hierarchy, but different fields are visible, and the structure of the HTML can also vary.

Initially, this can be mitigated with (tightly scoped) try/except blocks, to suppress errors or return None. Longer term, we should consider documenting what we expect to find for a given role, and raising an error or warning if that is not visible (e.g. date of birth for an Administrator role, etc.).

We should also likely add a flag to supress or raise errors for use in production code -- open to dicussion on if it is how to supress errors, and how to communicate which errors have been supressed.

We should also start to test the code -- we can unit test some of the functions that don't integrate with Compass, but for the scraper classes we should try to do some sort of mocking/faking -- perhaps by saving various examples of the returned HTML and parameterising the personal data (invalid phone numbers, missing data etc.). For now, we should gather examples of member IDs where errors occour.

Errors examples:

I ... was encountering exceptions within the xpath queries ... when Compass permissions weren't rendering fields (i.e. Religion).

@rglss, the-scouts/compass-interface-core#1 (comment)

There's been multiple occasions that 'core' fields (email/address etc) are missing within the Compass record.

@rglss, the-scouts/compass-interface-core#1 (comment)

this now errors if the crawler doesn't get a DOB or Join Date (i.e. empty string or None returned from xpath)

@rglss, the-scouts/compass-interface-core#1 (comment)

I've also seen 'Unknown' for join dates

@rglss, the-scouts/compass-interface-core#1 (comment)

Testing comments

any 'proper' testing suite would involve all possible permissions groups etc etc.

I think perhaps useful though where errors have occured to save the response.content str (bytes), and with redaction of data we could use these in some form?

@AA-Turner, the-scouts/compass-interface-core#1 (comment)

Longer term, we should validate permission levels and what we expect to see.

seperating ... errors from Compass and errors caused by the library

@AA-Turner, the-scouts/compass-interface-core#1 (comment)