Skip to content

Commit

Permalink
Updates unit testing for MethodsV2
Browse files Browse the repository at this point in the history 
* Updates t/TestUtil.pm to give more explicit information when
  there are mismatches.
* Updates t/TestUtil.pm to support testing of a single MethodsV2
  scenario.
* Updates t/methodsv2.t to utilize testing of a single scenario.
* Correcting scenario GOOD-UNDEL-1, but still breaking.
  • Loading branch information
@matsduf
matsduf committed Jul 9, 2024
1 parent 49f8e00 commit 7b12a5c
Show file tree
Hide file tree
Showing 2 changed files with 45 additions and 27 deletions.
47 changes: 33 additions & 14 deletions t/TestUtil.pm
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,15 @@ unknown to the include path @INC, it can be including using the following code:
=item perform_methodsv2_testing()
perform_methodsv2_testing( %subtests );
perform_methodsv2_testing( $href_subtests, $single_scenario );
This method loads unit test data (test scenarios) and, after some data checks and if the test scenario is testable,
it runs all external L<MethodsV2|Zonemaster::Engine::Test::TestMethodsV2> methods and checks for the presence (or absence) of
specific nameservers data for each specified test scenario.
Takes a hash - the keys of which are scenario names (in all uppercase), and their corresponding values are an array of:
If $single_scenario is set then only that scenario will be run.
Takes a reference to a hash - the keys of which are scenario names (in all uppercase), and their corresponding values are an array of:
=over
Expand Down Expand Up @@ -79,7 +81,8 @@ as the data for the --ns option in I<zonemaster-cli>.
This method loads unit test data (test case name, test module name, array of all message tags and test scenarios) and,
after some data checks and if the test scenario is testable, it runs the specified test case and checks for the presence
(or absence) of specific message tags for each specified test scenario.
(or absence) of specific message tags for each specified test scenario. If $single_scenario has been set in the call
then that scenario will always be run even if it has been set as not testable.
Takes a string (test case name), a string (test module name) and a hash - the keys of which are scenario names
(in all uppercase), and their corresponding values are an array of:
Expand Down Expand Up @@ -181,11 +184,13 @@ sub _check_ds_expressions {
}

sub perform_methodsv2_testing {
my ( %subtests ) = @_;
my ( $href_subtests, $single_scenario ) = @_;
my %subtests = %$href_subtests;

my @untested_scenarios = ();

for my $scenario ( sort ( keys %subtests ) ) {
next if $single_scenario and $scenario ne uc($single_scenario);
if ( ref( $scenario ) ne '' or $scenario ne uc($scenario) ) {
croak "Scenario $scenario: Key must (i) not be a reference and (ii) be in all uppercase";
}
Expand Down Expand Up @@ -218,6 +223,8 @@ sub perform_methodsv2_testing {
croak "Scenario $scenario: Value of testable must be 0 or 1";
}

$testable = 1 if $single_scenario and $scenario eq $single_scenario;

if ( ref( $zone_name ) ne '' ) {
croak "Scenario $scenario: Type of zone name must not be a reference";
}
Expand Down Expand Up @@ -271,9 +278,9 @@ sub perform_methodsv2_testing {
ok( defined $res, "Result is defined" ) or diag "Unexpected undefined result";
foreach my $expected_ip ( @{ $expected_parent_ip } ) {
ok( grep( /^$expected_ip$/, uniq map { $_->address->short } @{ $res } ), "Nameserver IP '$expected_ip' is present" )
or diag "Nameserver IP '$expected_ip' should have been present, but wasn't";
or diag "Expected but missing: $expected_ip";
}
ok( scalar @{ $res } == scalar @{ $expected_parent_ip } ) or diag "Number of nameserver IP addresses in both arrays does not match";
ok( scalar @{ $res } == scalar @{ $expected_parent_ip } ) or diag "Number of nameserver IP addresses in both arrays does not match (found ". scalar @{ $res } . ", expected " . @{ $expected_parent_ip } . ")";
}
else {
ok( ! defined $res, "Result is undefined" ) or diag "Unexpected defined result";
Expand All @@ -292,9 +299,13 @@ sub perform_methodsv2_testing {
ok( defined $res, "Result is defined" ) or diag "Unexpected undefined result";
foreach my $expected_ns ( @{ $expected_res } ) {
ok( grep( /^$expected_ns$/, @{ $res } ), "Nameserver '$expected_ns' is present" )
or diag "Nameserver '$expected_ns' should have been present, but wasn't";
or diag "Expected but missing: $expected_ns";
}
foreach my $ns ( @{ $res } ) {
ok( grep( /^$ns$/, @{ $expected_res } ), "Nameserver '$ns' is expected" )
or diag "Present but not expected: $ns";
}
ok( scalar @{ $res } == scalar @{ $expected_res } ) or diag "Number of nameservers in both arrays does not match";
ok( scalar @{ $res } == scalar @{ $expected_res } ) or diag "Number of nameservers in both arrays does not match (found " . scalar @{ $res } . ", expected " . scalar @{ $expected_res }.")";
}
else {
ok( ! defined $res, "Result is undefined" ) or diag "Unexpected defined result";
Expand All @@ -316,9 +327,13 @@ sub perform_methodsv2_testing {
ok( defined $res, "Result is defined" ) or diag "Unexpected undefined result";
foreach my $expected_name ( @{ $expected_res } ) {
ok( grep( /^$expected_name$/, @{ $res } ), "Nameserver name '$expected_name' is present" )
or diag "Nameserver name '$expected_name' should have been present, but wasn't";
or diag "Expected but missing: $expected_name";
}
ok( scalar @{ $res } == scalar @{ $expected_res } ) or diag "Number of nameserver names in both arrays does not match";
foreach my $name ( @{ $res } ) {
ok( grep( /^$name$/, @{ $expected_res } ), "Nameserver name '$name' is expected" )
or diag "Present but not expected: $name";
}
ok( scalar @{ $res } == scalar @{ $expected_res } ) or diag "Number of nameserver names in both arrays does not match (found " . scalar @{ $res } . ", expected " . scalar @{ $expected_res }.")";
}
else {
ok( ! defined $res, "Result is undefined" ) or diag "Unexpected defined result";
Expand All @@ -328,8 +343,8 @@ sub perform_methodsv2_testing {

# Methods: get_del_ns_ips() and get_zone_ns_ips()
@method_names = qw( get_del_ns_ips get_zone_ns_ips );
my @expected_del_ns_ips = uniq map { (split( m(/), $_ ))[1] } @{ $expected_del_ns };
my @expected_zone_ns_ips = uniq map { (split( m(/), $_ ))[1] } @{ $expected_zone_ns };
my @expected_del_ns_ips = grep defined, uniq map { (split( m(/), $_ ))[1] } @{ $expected_del_ns };
my @expected_zone_ns_ips = grep defined, uniq map { (split( m(/), $_ ))[1] } @{ $expected_zone_ns };
my @expected_ns_ips = ( \@expected_del_ns_ips, \@expected_zone_ns_ips );
foreach my $i ( 0..$#method_names ) {
my $method = $method_names[$i];
Expand All @@ -340,9 +355,13 @@ sub perform_methodsv2_testing {
ok( defined $res, "Result is defined" ) or diag "Unexpected undefined result";
foreach my $expected_ip ( @{ $expected_res } ) {
ok( grep( /^$expected_ip$/, @{ $res } ), "Nameserver IP '$expected_ip' is present" )
or diag "Nameserver IP '$expected_ip' should have been present, but wasn't";
or diag "Expected but missing: $expected_ip";
}
foreach my $ip ( @{ $res } ) {
ok( grep( /^$ip$/, @{ $expected_res } ), "Nameserver IP '$ip' is expected" )
or diag "Present but not expected: $ip";
}
ok( scalar @{ $res } == scalar @{ $expected_res } ) or diag "Number of nameserver IPs in both arrays does not match";
ok( scalar @{ $res } == scalar @{ $expected_res } ) or diag "Number of nameserver IPs in both arrays does not match (found " . scalar @{ $res } . ", expected " . scalar @{ $expected_res }.")";
}
else {
ok( ! defined $res, "Result is undefined" ) or diag "Unexpected defined result";
Expand Down
25 changes: 12 additions & 13 deletions t/methodsv2.t
View file
Original file line number Diff line number Diff line change
Expand Up @@ -92,29 +92,27 @@ my %subtests = (
'GOOD-UNDEL-1' => [
0,
q(child.parent.good-undel-1.methodsv2.xa),
[],
[ qw( ns1-2.child.parent.good-undel-1.methodsv2.xa/127.40.1.42
ns1-2.child.parent.good-undel-1.methodsv2.xa/fda1:b2:c3:0:127:40:1:42
[], # No parent data
[ qw( ns1-2.child.parent.good-undel-1.methodsv2.xa/127.40.1.52
ns1-2.child.parent.good-undel-1.methodsv2.xa/fda1:b2:c3:0:127:40:1:52
ns3.parent.good-undel-1.methodsv2.xa/127.40.1.43
ns3.parent.good-undel-1.methodsv2.xa/fda1:b2:c3:0:127:40:1:43
ns6.good-undel-1.methodsv2.xa/127.40.1.36
ns6.good-undel-1.methodsv2.xa/fda1:b2:c3:0:127:40:1:36 ) ],
[ qw( ns1-2.child.parent.good-undel-1.methodsv2.xa/127.40.1.42
ns1-2.child.parent.good-undel-1.methodsv2.xa/fda1:b2:c3:0:127:40:1:42
ns3.parent.good-undel-1.methodsv2.xa/127.40.1.43
ns3.parent.good-undel-1.methodsv2.xa/fda1:b2:c3:0:127:40:1:43
ns6.good-undel-1.methodsv2.xa/127.40.1.36
ns6.good-undel-1.methodsv2.xa/fda1:b2:c3:0:127:40:1:36 ) ],
[ qw( ns1-2.child.parent.good-undel-1.methodsv2.xa/127.40.1.42
ns1-2.child.parent.good-undel-1.methodsv2.xa/fda1:b2:c3:0:127:40:1:42
[ qw( ns1-2.child.parent.good-undel-1.methodsv2.xa/127.40.1.52
ns1-2.child.parent.good-undel-1.methodsv2.xa/fda1:b2:c3:0:127:40:1:52
ns3.parent.good-undel-1.methodsv2.xa
ns6.good-undel-1.methodsv2.xa ) ],
[ qw( ns1-2.child.parent.good-undel-1.methodsv2.xa/127.40.1.52
ns1-2.child.parent.good-undel-1.methodsv2.xa/fda1:b2:c3:0:127:40:1:52
ns3.parent.good-undel-1.methodsv2.xa/127.40.1.43
ns3.parent.good-undel-1.methodsv2.xa/fda1:b2:c3:0:127:40:1:43
ns6.good-undel-1.methodsv2.xa ) ]
],
'GOOD-UNDEL-2' => [
1,
q(child.parent.good-undel-2.methodsv2.xa),
[],
[], # No parent data
[ qw( ns1.child.parent.good-undel-2.methodsv2.xa/127.40.1.51
ns1.child.parent.good-undel-2.methodsv2.xa/fda1:b2:c3:0:127:40:1:51
ns3.parent.good-undel-2.methodsv2.xa/127.40.1.43
Expand Down Expand Up @@ -145,7 +143,8 @@ if ( not $ENV{ZONEMASTER_RECORD} ) {
Zonemaster::Engine::Profile->effective->set( q{no_network}, 1 );
}

perform_methodsv2_testing( %subtests );
# perform_methodsv2_testing( %subtests );
perform_methodsv2_testing( \%subtests, $ENV{ZONEMASTER_METHODSV2_SCENARIO} ); # For patched TestUtil.pm

if ( $ENV{ZONEMASTER_RECORD} ) {
Zonemaster::Engine::Nameserver->save( $datafile );
Expand Down

0 comments on commit 7b12a5c

Please sign in to comment.