-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy path06_advanced_validation_2.robot
321 lines (312 loc) · 18 KB
/
06_advanced_validation_2.robot
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
*** Settings ***
Documentation Tests XML validation functionality by demonstrating and
... validating various approaches and use cases.
Library Collections
Library xmlvalidator ${EXECDIR}/test/_data/integration/TC_26/schema.xsd # base_url error_facets
Resource ${EXECDIR}/test/integration/validation_keywords.resource
Suite Teardown Default Test Suite Teardown
Variables teardown_vars.py
*** Variables ***
# ${DELETE_CSV} determines deletion of the csv output files; values can be:
# - bool ${False}: no delete at all
# - bool ${True}: delete the one csv, as generated in current test case run
# - str 'All': delete all csv files present in test case data folder
# ${DELETE_CSV}= all
# ${RESET_SCHEMA}= ${True}
*** Test Cases ***
26_Validation_With_Init_Schema
[Documentation] Validates that the schema as passed during
... library import is correctly loaded.
...
... Subsequently validates various XML files (located
... in a folder) against the loaded schema.
...
... Finally, validates that the keyword has correctly
... identified, collected and reported (returned) all
... errors that are to be expected and that,
... moreover, the errors have also been written to a
... CSV file.
# Setup test variables.
${xml_folder} = Set Variable ${EXECDIR}/test/_data/integration/TC_26
# Validate the init schema is loaded.
${cur_schema}= Get Schema
Should be equal ${cur_schema} schema.xsd
# Validate the XML files against the init XSD.
${errors} ${csv_path} = Validate Xml Files ${xml_folder}
# Define the expected results/errors.
${expected_errors_1} = Create Dictionary
... file_name=invalid_01.xml
... path=/Person/Age
... reason=invalid literal for int() with base 10: 'thirty'
${expected_errors_2} = Create Dictionary
... file_name=invalid_02.xml
... path=/Person
... reason=Unexpected child with tag 'Phone' at position 2. Tag 'Age' expected.
${expected_errors_3} = Create Dictionary
... file_name=invalid_02.xml
... path=/Person
... reason=Unexpected child with tag 'ExtraElement' at position 4.
# Validate the validation results are as expected.
Validate Xml Validation Results ${errors} ${expected_errors_1} ${expected_errors_2} ${expected_errors_3}
# Validate the CSV output.
Validate CSV ${csv_path} ${expected_errors_1} ${expected_errors_2} ${expected_errors_3}
# Teardown.
${xml_validator} = Get Library Instance xmlvalidator
[Teardown] Default Test Case Teardown ${csv_path} ${DELETE_CSV} ${xml_validator} xmlvalidator reset_schema=${RESET_SCHEMA}
27_Various_Ways_To_Load_Schemas_01
[Documentation] Optionally validates that the schema (as active
... at the end of the previous test case) is (still)
... correctly loaded.
...
... Subsequently validates various XML files (located
... in a folder), passing a new schema file (located
... in a seperate/different folder) directly to the
... keyword (replacing the previously active schema).
... Since all XML files are actually valid, the only
... validation applied to the result of the action
... is to ensure that the new schema has, indeed,
... replaced the previously active schema.
...
... Finally, a second set of XML files (located in
... yet another folder) is being validated, by
... passing the XML folder path as xsd_path to the
... keyword. This will load a schema file that is
... located in that folder, provided there is only
... one such schema file present.
... Since all XML files are valid, it is then only
... validated that the new schema has successfully
... replaced the previous schema.
# Validate the init schema is (still) loaded.
IF not ${RESET_SCHEMA}
${cur_schema}= Get Schema
Should be equal ${cur_schema} schema.xsd
END
# Pass an XSD file (path) to the keyword and validate XMLs in a seperate folder.
${errors} ${csv_path} = Validate Xml Files
... ${EXECDIR}/test/_data/integration/TC_27
... xsd_path=${EXECDIR}/test/_data/integration/TC_27/xsd/schema_01.xsd
# Validate the new schema is loaded.
${cur_schema}= Get Schema
Should be equal ${cur_schema} schema_01.xsd
# Pass the XML folder path also as XSD path.
${xml_folder}= Set Variable ${EXECDIR}/test/_data/integration/TC_27/xml_folder
${errors} ${csv_path} = Validate Xml Files
... ${xml_folder}
... xsd_path=${xml_folder}
# Validate the correct schema is loaded.
${cur_schema}= Get Schema
Should be equal ${cur_schema} schema_02.xsd
# Teardown.
${xml_validator} = Get Library Instance xmlvalidator
[Teardown] Default Test Case Teardown ${csv_path} ${DELETE_CSV} ${xml_validator} xmlvalidator reset_schema=${RESET_SCHEMA}
28_Various_Ways_To_Load_Schemas_02
[Documentation] Optionally validates that the schema (as active
... at the end of the previous test case) is (still)
... correctly loaded.
...
... Subsequently validates various XML files (located
... in a folder), without expicitly passing an
... xsd_path. Rather, the XML files are
... mapped to their respective XSD files (located in
... the same folder) dynamically, by file name
... matching. The latter is triggered by passing the
... ``xsd_search_strategy`` argument as
... 'by_filename' to the keyword, next to the XML
... folder path.
... The results of the validation are subsequently
... thoroughly validated in terms of completeness and
... correctness.
...
... Subsequently a second batch of XML files is
... validated in the same fashion. However, this time
... the XSD files reside in their own folder,
... different from the folder where the XML files are
... located.
... Since all XML files are valid, only the correct
... setting schemas is validated.
[Tags] git-exclude
# Validate the schema from the previous test case is (still) loaded.
IF not ${RESET_SCHEMA}
${cur_schema}= Get Schema
Should be equal ${cur_schema} schema_02.xsd
END
# Pass no XSD file path, but provide a matching strategy (XSDs assumed to be in XML folder).
${errors} ${csv_path} = Validate Xml Files
... ${EXECDIR}/test/_data/integration/TC_28
... xsd_search_strategy=by_filename
# Validate the new schema is loaded.
${cur_schema}= Get Schema
Should be equal ${cur_schema} file_4.xsd
# Define the errors that the library should identify, collect and report/return.
${expected_errors_1} = Create Dictionary
... file_name=file_1.xml
... path=/Person/Age
... reason=invalid literal for int() with base 10: 'ABC'
${expected_errors_2} = Create Dictionary
... file_name=file_5.xml
... reason=No matching XSD found for: file_5.
${expected_errors_3} = Create Dictionary
... file_name=file_6.xml
... reason=No matching XSD found for: file_6.
# Validate the validation results are as expected.
Validate Xml Validation Results ${errors} ${expected_errors_1} ${expected_errors_2} ${expected_errors_3}
# Validate the CSV output.
Validate CSV ${csv_path} ${expected_errors_1} ${expected_errors_2} ${expected_errors_3}
# Pass a seperate XSD folder path, but also a search strategy.
${errors} ${csv_path} = Validate Xml Files
... ${EXECDIR}/test/_data/integration/TC_28/xml
... xsd_path=${EXECDIR}/test/_data/integration/TC_28/xsd
... xsd_search_strategy=by_filename
# Validate the new schema is loaded.
${cur_schema}= Get Schema
Should be equal ${cur_schema} file_9.xsd
# Teardown.
${xml_validator} = Get Library Instance xmlvalidator
[Teardown] Default Test Case Teardown ${csv_path} ${DELETE_CSV} ${xml_validator} xmlvalidator reset_schema=${RESET_SCHEMA}
29_Various_Ways_To_Load_Schemas_03
[Documentation] Optionally validates that the schema (as active
... at the end of the previous test case) is (still)
... correctly loaded.
...
... Subsequently validates various XML files (located
... in a folder), without expicitly passing an
... xsd_path. Rather, the XML files are
... mapped to their respective XSD files (located in
... the same folder) dynamically, by namespace
... matching. The latter is triggered by passing the
... ``xsd_search_strategy`` argument as
... 'by_name_space' to the keyword, next to the XML
... folder path.
...
... In total, 21 invalid files will be reported as
... invalid, either because of (1) XSD violation,
... (2) malformedness or (3) because of not matching
... any XSD.
...
... Of the 21 errors that the library will identify,
... collect and report, only three will be validated
... for correctness. since there have been plenty
... validations throughout the various integration
... test suites already. You can inspect the log to
... see the various XSD violations and other types of
... errors that the library reports.
...
... Bascially, there are three XSD files and for each
... XSD file there are three XML files that violate
... the XSD and three XML files that do not violate
... the schema, but are malformed. The latter will be
... catched and reported by the library as well!
... Finally, there are three XML files that do not
... match with any of the three schema files and will
... therefore be reported as such, by the library.
[Tags] git-exclude
# Validate the schema from the previous test case is (still) loaded.
IF not ${RESET_SCHEMA}
${cur_schema}= Get Schema
Should be equal ${cur_schema} schema_02.xsd
END
# Pass no XSD file path, but provide a matching strategy: XSDs assumed to be in XML folder.
${errors} ${csv_path} = Validate Xml Files
... ${EXECDIR}/test/_data/integration/TC_29
... xsd_search_strategy=by_name_space
# Validate the expected schema is loaded.
${cur_schema}= Get Schema
Should Be Equal ${cur_schema} schema3.xsd
# Define the expected validation results.
${expected_errors_1} = Create Dictionary
... file_name=schema1_invalid_1.xml
... path=/Employee/ID
... reason=invalid literal for int() with base 10: 'ABC'
${expected_errors_2} = Create Dictionary
... file_name=schema1_malformed_1.xml
... path=
... reason=Premature end of data in tag Employee line 1, line 1, column 11 (file:/${{ $EXECDIR.replace('\\', '/')}}/test/_data/integration/TC_29/schema1_malformed_1.xml, line 1)
${expected_errors_3} = Create Dictionary
... reason=No matching XSD found for: unmatched_1.
# Validate the validation results are as expected.
Validate Xml Validation Results ${errors} ${expected_errors_1} ${expected_errors_2} ${expected_errors_3}
# Teardown.
${xml_validator} = Get Library Instance xmlvalidator
[Teardown] Default Test Case Teardown ${csv_path} ${DELETE_CSV} ${xml_validator} xmlvalidator reset_schema=${RESET_SCHEMA}
30_Error_Facets
[Documentation] This test case passes custom (non-default) error
... facets to the Validate Xml Files keywords.
...
... Error facets determine which attributes of a
... parse error or an XmlSchema (validation) error
... are collected and reported by the keyword.
...
... Error facets can also be passed during library
... import. Passing error facets to the keyword
... overrules these default facets. However, the next
... time the keyword ios called without passing
... error_facets, the default facets apply again.
...
... The validations errors that are reported/returned
... in this test case will not be validated. Instead,
... each error is logged, including the data type of
... the error facet (e.g. a string or
... xml.etree.ElementTree.Element).
# Specify error facets.
@{new_error_facets}= Create List message elem namespaces
# Pass non-default error facets; pass no XSD file path, but provide a matching strategy (XSDs are assumed to reside in the XML folder).
${errors} ${csv_path} = Validate Xml Files
... ${EXECDIR}/test/_data/integration/TC_30
... xsd_search_strategy=by_name_space
... error_facets=${new_error_facets}
# Let's see what's inside the collected errors/violations.
FOR ${idx} ${error} IN ENUMERATE @{errors}
Log ==================================== Inspecting error #${idx} console=True
FOR ${key} ${value} IN &{error}
Log Error facet '${key}': ${value} (${{ type( $value ) }}) console=True
END
IF ${idx} == 5
BREAK
END
END
# Validate the new schema is loaded.
${cur_schema}= Get Schema
Should Be Equal ${cur_schema} schema3.xsd
# Teardown.
${xml_validator} = Get Library Instance xmlvalidator
[Teardown] Default Test Case Teardown ${csv_path} ${DELETE_CSV} ${xml_validator} xmlvalidator reset_schema=${RESET_SCHEMA}
31_Validation_Passing_XSD_Path_With_Include
[Documentation] Validate multiple XML files against an XSD with
... an included XSD.
...
... This ensures that XSD imports/includes are
... correctly handled.
...
... The Validate Xml Keyword will validate three XML
... files against the loaded schema:
...
... - one file will be deemed valid
... - one file will be deemed invalid on account of
... violating the main XSD schema
... - one file will be deemed invalid on account of
... violating the included XSD schema
# Set up test variables.
${xml_folder} = Set Variable ${EXECDIR}/test/_data/integration/TC_31/xml
${xsd_path} = Set Variable ${EXECDIR}/test/_data/integration/TC_31/main_schema.xsd
${base_url} = Set Variable ${EXECDIR}/test/_data/integration/TC_31
# Validate XML files, passing also a base_url to be used when constructing the schema object.
${result} ${csv_path} = Validate Xml Files ${xml_folder} xsd_path=${xsd_path} base_url=${base_url}
# Validate the schema has been loaded without problems.
${cur_schema}= Get Schema
Should be equal ${cur_schema} main_schema.xsd
# Expected validation results
${expected_result_1} = Create Dictionary
... file_name=invalid_by_include.xml
... path=/Person/PhoneNumber
... reason=value doesn't match any pattern of ['\\\\d{3}-\\\\d{3}-\\\\d{4}']
${expected_result_2} = Create Dictionary
... file_name=invalid_by_main.xml
... path=/Person
... reason=Unexpected child with tag '{http://example.com/my_namespace}PhoneNumber' at position 1. Tag '{http://example.com/my_namespace}Name' expected.
# Validate structured results
Validate Xml Validation Results ${result} ${expected_result_1} ${expected_result_2}
# Validate CSV output
Validate CSV ${csv_path} ${expected_result_1} ${expected_result_2}
# Teardown.
${xml_validator} = Get Library Instance xmlvalidator
[Teardown] Default Test Case Teardown ${csv_path} ${DELETE_CSV} ${xml_validator} xmlvalidator reset_schema=${RESET_SCHEMA}