Skip to content

ACD-4521: Generate the same test ids for indextime tests #166

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 19, 2020
Merged

ACD-4521: Generate the same test ids for indextime tests #166

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a778fdf
ACD-4521: Sorted sample_files in eventgen parser
jayjoshi-crest Aug 18, 2020
6dd5b19
ACD-4521: Updated constants file with the test-id changes
jayjoshi-crest Aug 18, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions pytest_splunk_addon/standard_lib/sample_generation/eventgen_parser.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ def get_sample_stanzas(self):
"""
eventgen_dict = self.get_eventgen_stanzas()
self.check_samples()
for sample_name, stanza_params in eventgen_dict.items():
for sample_name, stanza_params in sorted(eventgen_dict.items()):
sample_path = os.path.join(self.path_to_samples, sample_name)
yield SampleStanza(
sample_path, stanza_params,
Expand Down Expand Up @@ -118,10 +118,9 @@ def get_eventgen_stanzas(self):
Dictionary representing eventgen.conf in the above format.
"""
eventgen_dict = {}
child_dict = {"tokens": {}}
if os.path.exists(self.path_to_samples):
for sample_file in os.listdir(self.path_to_samples):
for stanza in self.eventgen.sects:
for stanza in sorted(self.eventgen.sects):
stanza_match_obj = re.search(stanza, sample_file)
if stanza_match_obj and stanza_match_obj.group(0) == sample_file:
self.match_stanzas.add(stanza)
Expand Down
56 changes: 28 additions & 28 deletions tests/constants.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -445,15 +445,17 @@
"""
TA_FICTION_INDEXTIME_PASSED = [
'*test_splunk_fiction_indextime.py::Test_App::test_events_with_untokenised_values* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:file_monitor_host_prefix::test-host_file_monitor_host_prefix.sample_186_to_test-host_file_monitor_host_prefix.sample_188* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:file_monitor_host_prefix::test-host_file_monitor_host_prefix.sample_2_to_test-host_file_monitor_host_prefix.sample_4* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:file_monitor_time_stamp_plugin::file_monitor_time_stamp_plugin.samples* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:scripted_input_key_fields_fiction::scripted_input_key_fields_fiction.samples* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:scripted_input_line_breaking_fiction::scripted_input_line_breaking_fiction.samples_35* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:scripted_input_line_breaking_fiction::scripted_input_line_breaking_fiction.samples_36* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:sourcetype:file_all_dependent::host_sample_file_three.samples_196_to_host_sample_file_three.samples_202* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:scripted_input_line_breaking_fiction::scripted_input_line_breaking_fiction.samples_33* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:scripted_input_line_breaking_fiction::scripted_input_line_breaking_fiction.samples_34* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:sourcetype:file_all_dependent::host_sample_file_three.samples_190_to_host_sample_file_three.samples_196* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:sourcetype:modinput_host_event_time_plugin::modinput_host_event_time_plugin.samples_1* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:sourcetype:modinput_host_event_time_plugin::modinput_host_event_time_plugin.samples_2* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:sourcetype:user_email_relation_time_plugin::sample_file_two.samples* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:sourcetype_basic::sample_file.samples_13* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:sourcetype_basic::sample_file.samples_14* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:sourcetype_basic::sample_file.samples_15* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:sourcetype_basic::sample_file.samples_16* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:sourcetype_basic::sample_file.samples_17* PASSED*',
Expand All @@ -470,30 +472,30 @@
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:sourcetype_basic::sample_file.samples_28* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:sourcetype_basic::sample_file.samples_29* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:sourcetype_basic::sample_file.samples_30* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:sourcetype_basic::sample_file.samples_13* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:indextime:sourcetype_basic::sample_file.samples_14* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:postfix:regex:stanza::postfix_test_regex_two.samples_1* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test:postfix:regex:stanza::postfix_test_regex_two.samples_2* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test_scripted_input_sourcetype::host_test_scripted_input_one.samples_33* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test_scripted_input_sourcetype::host_test_scripted_input_one.samples_34* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test_scripted_input_sourcetype::host_test_scripted_input_one.samples_35* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_key_fields*test_scripted_input_sourcetype::host_test_scripted_input_one.samples_36* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*mcafee:epo:syslog::sc4s_host_plugin_time.sample_31* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*mcafee:epo:syslog::sc4s_host_plugin_time.sample_32* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:file_monitor_host_prefix::test-host_file_monitor_host_prefix.sample_186_to_test-host_file_monitor_host_prefix.sample_188* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:file_monitor_host_prefix::test-host_file_monitor_host_prefix.sample_2_to_test-host_file_monitor_host_prefix.sample_4* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:modinput_host_prefix::test-modinput_host_prefix.sample_1* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:modinput_host_prefix::test-modinput_host_prefix.sample_2* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:modinput_line_breaker::modinput_line_breaker.samples_1* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:modinput_line_breaker::modinput_line_breaker.samples_2* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:modinput_line_breaker::modinput_line_breaker.samples_3* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:modinput_line_breaker::modinput_line_breaker.samples_4* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:scripted_input_key_fields_fiction::scripted_input_key_fields_fiction.samples* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:scripted_input_line_breaking_fiction::scripted_input_line_breaking_fiction.samples_35* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:scripted_input_line_breaking_fiction::scripted_input_line_breaking_fiction.samples_36* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:sourcetype:file_all_dependent::host_sample_file_three.samples_196_to_host_sample_file_three.samples_202* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:scripted_input_line_breaking_fiction::scripted_input_line_breaking_fiction.samples_33* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:scripted_input_line_breaking_fiction::scripted_input_line_breaking_fiction.samples_34* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:sourcetype:file_all_dependent::host_sample_file_three.samples_190_to_host_sample_file_three.samples_196* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:sourcetype:updated::test_regex_one.samples_1* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:sourcetype:updated::test_regex_one.samples_2* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:sourcetype:updated::test_regex_two.samples_1* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:sourcetype:updated::test_regex_two.samples_2* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:sourcetype:user_email_relation_time_plugin::sample_file_two.samples* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:sourcetype_basic::sample_file.samples_13* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:sourcetype_basic::sample_file.samples_14* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:sourcetype_basic::sample_file.samples_15* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:sourcetype_basic::sample_file.samples_16* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:sourcetype_basic::sample_file.samples_17* PASSED*',
Expand All @@ -510,8 +512,6 @@
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:sourcetype_basic::sample_file.samples_28* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:sourcetype_basic::sample_file.samples_29* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:sourcetype_basic::sample_file.samples_30* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:sourcetype_basic::sample_file.samples_13* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_time*test:indextime:sourcetype_basic::sample_file.samples_14* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_line_breaker*mcafee:epo:syslog::sc4s_host_plugin_time.sample* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_line_breaker*test:indextime:file_monitor_host_prefix::file_monitor_host_prefix.sample* PASSED*',
'*test_splunk_fiction_indextime.py::Test_App::test_indextime_line_breaker*test:indextime:file_monitor_line_breaker::file_monitor_line_breaker_sample.samples* PASSED*',
Expand Down Expand Up @@ -578,8 +578,8 @@
"""
TA_FICTION_INDEXTIME_BROKEN_PASSED = [
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_key_fields*mcafee:epo:syslog::wrong_sc4s_tokens.sample* PASSED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_key_fields*test:indextime:scripted_input_line_breaking::scripted_input_line_breaking.samples_1* PASSED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_key_fields*test:indextime:scripted_input_line_breaking::scripted_input_line_breaking.samples_2* PASSED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_key_fields*test:indextime:scripted_input_line_breaking::scripted_input_line_breaking.samples_3* PASSED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_key_fields*test:indextime:scripted_input_line_breaking::scripted_input_line_breaking.samples_4* PASSED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_key_fields*test:indextime:sourcetype:skip_token_or_empty_line::skip_token_or_empty_line.samples_1* PASSED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_key_fields*test:indextime:sourcetype:skip_token_or_empty_line::skip_token_or_empty_line.samples_2* PASSED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_key_fields*test:indextime:sourcetype:skip_token_or_empty_line::skip_token_or_empty_line.samples_3* PASSED*',
Expand All @@ -606,35 +606,35 @@
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_line_breaker*test:indextime:wrong_timezone::wrong_timezone.samples* PASSED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_cim_fields_not_allowed_in_props*searchtime_cim_fields* PASSED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_eventtype_mapped_multiple_cim_datamodel*mapped_datamodel_tests* PASSED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_splunk_internal_errors PASSED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_splunk_internal_errors PASSED *',
]
"""
Define the TA_fiction_indextime_broken add-on failed test case list.
"""
TA_FICTION_INDEXTIME_BROKEN_FAILED = [
'*test_splunk_fiction_indextime_broken.py::Test_App::test_events_with_untokenised_values FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_key_fields*test:indextime:failing::failing.samples_3* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_key_fields*test:indextime:failing::failing.samples_4* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_key_fields*test:indextime:file_monitor_host_prefix::test1-host_wrong_prefix_filemonitor.sample_212_to_test1-host_wrong_prefix_filemonitor.sample_214* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_key_fields*test:indextime:failing::failing.samples_1* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_key_fields*test:indextime:failing::failing.samples_2* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_key_fields*test:indextime:file_monitor_host_prefix::test1-host_wrong_prefix_filemonitor.sample_218_to_test1-host_wrong_prefix_filemonitor.sample_220* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_key_fields*test:indextime:scripted_input_key_fields::scripted_input_key_fields.samples* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_time*test:indextime:failing::failing.samples_3* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_time*test:indextime:failing::failing.samples_4* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_time*test:indextime:file_monitor_host_prefix::test1-host_wrong_prefix_filemonitor.sample_212_to_test1-host_wrong_prefix_filemonitor.sample_214* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_time*test:indextime:failing::failing.samples_1* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_time*test:indextime:failing::failing.samples_2* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_time*test:indextime:file_monitor_host_prefix::test1-host_wrong_prefix_filemonitor.sample_218_to_test1-host_wrong_prefix_filemonitor.sample_220* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_time*test:indextime:no_host_provided_when_type_event::no_host_provided_when_type_event.samples* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_time*test:indextime:no_prefix_stanza_modinput::no_prefix_stanza_modinput.sample_1* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_time*test:indextime:no_prefix_stanza_modinput::no_prefix_stanza_modinput.sample_2* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_time*test:indextime:no_time_provided_when_type_event::no_time_provided_when_type_event.samples* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_time*test:indextime:scripted_input_key_fields::scripted_input_key_fields.samples* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_time*test:indextime:scripted_input_line_breaking::scripted_input_line_breaking.samples_1* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_time*test:indextime:scripted_input_line_breaking::scripted_input_line_breaking.samples_2* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_time*test:indextime:scripted_input_line_breaking::scripted_input_line_breaking.samples_3* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_time*test:indextime:scripted_input_line_breaking::scripted_input_line_breaking.samples_4* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_time*test:indextime:wrong_line_breaker_modinput::wrong_line_breaker_modinput.samples_1* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_time*test:indextime:wrong_timezone::wrong_timezone.samples* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_line_breaker*test:indextime:failing::failing.samples* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_line_breaker*test:indextime:file_monitor_host_prefix::wrong_prefix_filemonitor.sample* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_line_breaker*test:indextime:scripted_input_line_breaking::scripted_input_line_breaking.samples* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_line_breaker*test:indextime:wrong_line_breaker_file_monitor::wrong_line_breaker_file_monitor.samples* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_line_breaker*test:indextime:no_host_provided_when_type_event::no_host_provided_when_type_event.samples* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_line_breaker*test:indextime:no_prefix_stanza_modinput::no_prefix_stanza_modinput.sample* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_time*test:indextime:wrong_line_breaker_modinput::wrong_line_breaker_modinput.samples_1* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_line_breaker*test:indextime:scripted_input_line_breaking::scripted_input_line_breaking.samples* FAILED*',
'*test_splunk_fiction_indextime_broken.py::Test_App::test_indextime_line_breaker*test:indextime:wrong_line_breaker_file_monitor::wrong_line_breaker_file_monitor.samples* FAILED*',
]
"""
Define the TA_fiction_indextime_broken add-on skipped test case list.
Expand Down