Skip to content

Commit

Permalink
Add a new 'report-junit' command
Browse files Browse the repository at this point in the history
Because of the semantics of the JUnit output, it does not make much sense
to try to shovel this format into the 'report' command.  For example:
there is no reason to honor the --show-context flag, and honoring the
--results-filter flag means that the generated reports are misleading due
to invalid total test counts.  (My original separation of the HTML format
into 'report-html' seems to have proven right...)

Removal of the JUnit support from the 'report' command, as well as other
reorganizations, will come later.
  • Loading branch information
jmmv committed May 30, 2014
1 parent 63faad6 commit ebc43a2
Show file tree
Hide file tree
Showing 14 changed files with 443 additions and 15 deletions.
3 changes: 3 additions & 0 deletions kyua-cli/NEWS
Expand Up @@ -3,6 +3,9 @@ Changes in version 0.9

STILL UNDER DEVELOPMENT; NOT RELEASED YET.

* Added the 'report-junit' command to generate JUnit XML result files.
The output has been verified to work within Jenkins.

* Changed the 'report' command, again, to support outputing various
different formats. The '--output' flag has been modified to support
specifying the format to generate.
Expand Down
2 changes: 2 additions & 0 deletions kyua-cli/cli/Makefile.am.inc
Expand Up @@ -48,6 +48,8 @@ libcli_a_SOURCES += cli/cmd_report.cpp
libcli_a_SOURCES += cli/cmd_report.hpp
libcli_a_SOURCES += cli/cmd_report_html.cpp
libcli_a_SOURCES += cli/cmd_report_html.hpp
libcli_a_SOURCES += cli/cmd_report_junit.cpp
libcli_a_SOURCES += cli/cmd_report_junit.hpp
libcli_a_SOURCES += cli/cmd_test.cpp
libcli_a_SOURCES += cli/cmd_test.hpp
libcli_a_SOURCES += cli/common.cpp
Expand Down
94 changes: 94 additions & 0 deletions kyua-cli/cli/cmd_report_junit.cpp
@@ -0,0 +1,94 @@
// Copyright 2014 Google Inc.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

#include "cli/cmd_report_junit.hpp"

#include <cstddef>
#include <cstdlib>

#include "cli/common.ipp"
#include "cli/report_junit.hpp"
#include "engine/drivers/scan_action.hpp"
#include "engine/test_result.hpp"
#include "utils/cmdline/parser.ipp"
#include "utils/defs.hpp"
#include "utils/optional.ipp"

namespace cmdline = utils::cmdline;
namespace config = utils::config;
namespace fs = utils::fs;
namespace scan_action = engine::drivers::scan_action;

using cli::cmd_report_junit;
using utils::optional;


/// Default constructor for cmd_report.
cmd_report_junit::cmd_report_junit(void) : cli_command(
"report-junit", "", 0, 0,
"Generates a JUnit report with the result of a previous action")
{
add_option(store_option);
add_option(cmdline::int_option(
"action", "The action to report; if not specified, defaults to the "
"latest action in the database", "id"));
add_option(cmdline::path_option("output", "Path to the output file", "path",
"/dev/stdout"));
}


/// Entry point for the "report" subcommand.
///
/// \param ui Object to interact with the I/O of the program.
/// \param cmdline Representation of the command line to the subcommand.
/// \param unused_user_config The runtime configuration of the program.
///
/// \return 0 if everything is OK, 1 if the statement is invalid or if there is
/// any other problem.
int
cmd_report_junit::run(cmdline::ui* ui, const cmdline::parsed_cmdline& cmdline,
const config::tree& UTILS_UNUSED_PARAM(user_config))
{
optional< int64_t > action_id;
if (cmdline.has_option("action"))
action_id = cmdline.get_option< cmdline::int_option >("action");

result_types no_filter;
no_filter.push_back(engine::test_result::passed);
no_filter.push_back(engine::test_result::skipped);
no_filter.push_back(engine::test_result::expected_failure);
no_filter.push_back(engine::test_result::broken);
no_filter.push_back(engine::test_result::failed);

report_junit_hooks hooks(
ui, cmdline.get_option< cmdline::path_option >("output"), true,
no_filter);
scan_action::drive(store_path(cmdline), action_id, hooks);

return EXIT_SUCCESS;
}
54 changes: 54 additions & 0 deletions kyua-cli/cli/cmd_report_junit.hpp
@@ -0,0 +1,54 @@
// Copyright 2014 Google Inc.
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions are
// met:
//
// * Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer.
// * Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
// * Neither the name of Google Inc. nor the names of its contributors
// may be used to endorse or promote products derived from this software
// without specific prior written permission.
//
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

/// \file cli/cmd_report_junit.hpp
/// Provides the cmd_report_junit class.

#if !defined(CLI_CMD_REPORT_JUNIT_HPP)
#define CLI_CMD_REPORT_JUNIT_HPP

#include "cli/common.hpp"

namespace cli {


/// Implementation of the "report-junit" subcommand.
class cmd_report_junit : public cli_command
{
public:
cmd_report_junit(void);

int run(utils::cmdline::ui*, const utils::cmdline::parsed_cmdline&,
const utils::config::tree&);
};


} // namespace cli


#endif // !defined(CLI_CMD_REPORT_JUNIT_HPP)
2 changes: 2 additions & 0 deletions kyua-cli/cli/main.cpp
Expand Up @@ -51,6 +51,7 @@ extern "C" {
#include "cli/cmd_list.hpp"
#include "cli/cmd_report.hpp"
#include "cli/cmd_report_html.hpp"
#include "cli/cmd_report_junit.hpp"
#include "cli/cmd_test.hpp"
#include "cli/common.ipp"
#include "cli/config.hpp"
Expand Down Expand Up @@ -169,6 +170,7 @@ safe_main(cmdline::ui* ui, int argc, const char* const argv[],

commands.insert(new cli::cmd_report(), "Reporting");
commands.insert(new cli::cmd_report_html(), "Reporting");
commands.insert(new cli::cmd_report_junit(), "Reporting");

if (mock_command.get() != NULL)
commands.insert(mock_command);
Expand Down
14 changes: 5 additions & 9 deletions kyua-cli/cli/report_junit.cpp
Expand Up @@ -100,15 +100,6 @@ cli::report_junit_hooks::report_junit_hooks(
}


/// Starts the report.
void
cli::report_junit_hooks::begin(void)
{
_writer("<?xml version=\"1.0\" encoding=\"iso-8859-1\"?>");
_writer("<testsuite>");
}


/// Callback executed when an action is found.
///
/// \param action_id The identifier of the loaded action.
Expand All @@ -117,10 +108,15 @@ void
cli::report_junit_hooks::got_action(const int64_t action_id,
const engine::action& action)
{
_writer("<?xml version=\"1.0\" encoding=\"iso-8859-1\"?>");
_writer("<testsuite>");

_action_id = action_id;
if (_show_context) {
const engine::context& context = action.runtime_context();
_writer("<properties>");
_writer(F("<property name=\"kyua.action_id\" value=\"%s\"/>")
% _action_id);
_writer(F("<property name=\"cwd\" value=\"%s\"/>")
% text::escape_xml(context.cwd().str()));
for (config::properties_map::const_iterator iter =
Expand Down
2 changes: 0 additions & 2 deletions kyua-cli/cli/report_junit.hpp
Expand Up @@ -60,8 +60,6 @@ class report_junit_hooks : public engine::drivers::scan_action::base_hooks {
report_junit_hooks(utils::cmdline::ui*, const utils::fs::path&,
const bool, const cli::result_types&);

void begin(void);

void got_action(const int64_t, const engine::action&);
void got_result(store::results_iterator&);

Expand Down
1 change: 1 addition & 0 deletions kyua-cli/doc/.gitignore
Expand Up @@ -10,6 +10,7 @@ kyua-help.1
kyua-list.1
kyua-plain-interface.7
kyua-report-html.1
kyua-report-junit.1
kyua-report.1
kyua-test.1
kyua-test-filters.7
Expand Down
6 changes: 6 additions & 0 deletions kyua-cli/doc/Makefile.am.inc
Expand Up @@ -94,6 +94,12 @@ EXTRA_DIST += doc/kyua-report-html.1.in
doc/kyua-report-html.1: $(srcdir)/doc/kyua-report-html.1.in
name=kyua-report-html.1; $(BUILD_MANPAGE)

man_MANS += doc/kyua-report-junit.1
CLEANFILES += doc/kyua-report-junit.1
EXTRA_DIST += doc/kyua-report-junit.1.in
doc/kyua-report-junit.1: $(srcdir)/doc/kyua-report-junit.1.in
name=kyua-report-junit.1; $(BUILD_MANPAGE)

man_MANS += doc/kyua-report.1
CLEANFILES += doc/kyua-report.1
EXTRA_DIST += doc/kyua-report.1.in
Expand Down
71 changes: 71 additions & 0 deletions kyua-cli/doc/kyua-report-junit.1.in
@@ -0,0 +1,71 @@
.\" Copyright 2014 Google Inc.
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions are
.\" met:
.\"
.\" * Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" * Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\" * Neither the name of Google Inc. nor the names of its contributors
.\" may be used to endorse or promote products derived from this software
.\" without specific prior written permission.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Dd May 26, 2014
.Dt KYUA-REPORT-JUNIT 1
.Os
.Sh NAME
.Nm report-junit
.Nd Generates a JUnit report with the results of a test action
.Sh SYNOPSIS
.Nm
.Op Fl -action Ar id
.Op Fl -output Ar path
.Op Fl -store Ar file
.Sh DESCRIPTION
The
.Nm
command provides a simple mechanism to generate JUnit reports of the
execution of a test suite. The command processes an action stored in the
database and then generates a single XML file that complies to the JUnit
XSchema.
.Pp
The JUnit output is static and self-contained, so it can easily be plugged
into any continuous integration system, like Jenkins.
.Pp
The following subcommand options are recognized:
.Bl -tag -width XX
.It Fl -action Ar id
Specifies the action for which to generate a report. If not provided,
defaults to the latest action stored in the database.
.It Fl -output Ar directory
Specifies the file into which to store the JUnit report.
.It Fl -store Ar path , Fl s Ar path
Specifies the database to use. Defaults to
.Pa ~/.kyua/store.db .
The database is created if it does not exist.
.El
.Sh EXIT STATUS
The
.Nm
command always returns 0.
.Pp
Additional exit codes may be returned as described in
.Xr kyua 1 .
.Sh SEE ALSO
.Xr kyua 1 ,
.Xr kyua-report 1
12 changes: 8 additions & 4 deletions kyua-cli/doc/kyua.1.in
Expand Up @@ -25,7 +25,7 @@
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.Dd February 16, 2014
.Dd May 26, 2014
.Dt KYUA 1
.Os
.Sh NAME
Expand Down Expand Up @@ -121,7 +121,7 @@ for more details.
.Ss Available commands
The following commands are generic and do not have any relation to the execution
of tests or the inspection of their results:
.Bl -tag -width reportXhtmlXX -offset indent
.Bl -tag -width reportXjunitXX -offset indent
.It Ar about
Shows general program information.
See
Expand All @@ -143,7 +143,7 @@ See
.Pp
The following commands are used to generate reports based on the data previously
stored in the database:
.Bl -tag -width reportXhtmlXX -offset indent
.Bl -tag -width reportXjunitXX -offset indent
.It Ar report
Generates a plain-text report.
See
Expand All @@ -152,10 +152,14 @@ See
Generates an HTML report.
See
.Xr kyua-report-html 1 .
.It Ar report-junit
Generates a JUnit report.
See
.Xr kyua-report-junit 1 .
.El
.Pp
The following commands are used to interact with a test suite:
.Bl -tag -width reportXhtmlXX -offset indent
.Bl -tag -width reportXjunitXX -offset indent
.It Ar debug
Executes a single test case in a controlled environment for debugging purposes.
See
Expand Down
1 change: 1 addition & 0 deletions kyua-cli/integration/Kyuafile
Expand Up @@ -10,6 +10,7 @@ atf_test_program{name="cmd_debug_test"}
atf_test_program{name="cmd_help_test"}
atf_test_program{name="cmd_list_test"}
atf_test_program{name="cmd_report_html_test"}
atf_test_program{name="cmd_report_junit_test"}
atf_test_program{name="cmd_report_test"}
atf_test_program{name="cmd_test_test"}
atf_test_program{name="global_test"}
9 changes: 9 additions & 0 deletions kyua-cli/integration/Makefile.am.inc
Expand Up @@ -124,6 +124,15 @@ integration/cmd_report_html_test: \
@src="$(srcdir)/integration/cmd_report_html_test.sh"; \
dst="integration/cmd_report_html_test"; $(ATF_SH_BUILD)

tests_integration_SCRIPTS += integration/cmd_report_junit_test
CLEANFILES += integration/cmd_report_junit_test
EXTRA_DIST += integration/cmd_report_junit_test.sh
integration/cmd_report_junit_test: \
$(srcdir)/integration/cmd_report_junit_test.sh $(ATF_SH_DEPS)
test -d integration || mkdir -p integration
@src="$(srcdir)/integration/cmd_report_junit_test.sh"; \
dst="integration/cmd_report_junit_test"; $(ATF_SH_BUILD)

tests_integration_SCRIPTS += integration/cmd_test_test
CLEANFILES += integration/cmd_test_test
EXTRA_DIST += integration/cmd_test_test.sh
Expand Down

0 comments on commit ebc43a2

Please sign in to comment.