Skip to content

Commit

Permalink
Add an experimental new way of capturing stdout/stderr
Browse files Browse the repository at this point in the history
Unlike the relatively non-invasive old way of capturing stdout/stderr,
this new way is also able to capture output from C's stdlib functions
such as `printf`. This is done by redirecting stdout and stderr file
descriptors to a file, and then reading this file back.

This approach has two sizeable drawbacks:
1) Performance, obviously. Previously an installed capture made the
program run faster (as long as it was then discarded), because a call
to `std::cout` did not result in text output to the console. This new
capture method in fact forces disk IO. While it is likely that any
modern OS will keep this file in memory-cache and might never actually
issue the IO to the backing storage, it is still a possibility and
calls to the file system are not free.

2) Nonportability. While POSIX is usually assumed portable, and this
implementation relies only on a very common parts of it, it is no
longer standard C++ (or just plain C) and thus might not be available
on some obscure platforms. Different C libs might also implement the
relevant functions in a less-than-useful ways (e.g. MS's `tmpfile`
generates a temp file inside system folder, so it will not work
without elevated privileges and thus is useless).

These two drawbacks mean that, at least for now, the new capture is
opt-in. To opt-in, `CATCH_CONFIG_EXPERIMENTAL_REDIRECT` needs to be
defined in the implementation file.

Closes #1243
  • Loading branch information
horenmar committed Apr 29, 2018
1 parent 88a6ff0 commit e92b9c0
Show file tree
Hide file tree
Showing 5 changed files with 243 additions and 44 deletions.
2 changes: 2 additions & 0 deletions CMakeLists.txt
Expand Up @@ -159,6 +159,7 @@ set(INTERNAL_HEADERS
${HEADER_DIR}/internal/catch_objc.hpp
${HEADER_DIR}/internal/catch_objc_arc.hpp
${HEADER_DIR}/internal/catch_option.hpp
${HEADER_DIR}/internal/catch_output_redirect.h
${HEADER_DIR}/internal/catch_platform.h
${HEADER_DIR}/internal/catch_random_number_generator.h
${HEADER_DIR}/internal/catch_reenable_warnings.h
Expand Down Expand Up @@ -225,6 +226,7 @@ set(IMPL_SOURCES
${HEADER_DIR}/internal/catch_matchers_generic.cpp
${HEADER_DIR}/internal/catch_matchers_string.cpp
${HEADER_DIR}/internal/catch_message.cpp
${HEADER_DIR}/internal/catch_output_redirect.cpp
${HEADER_DIR}/internal/catch_registry_hub.cpp
${HEADER_DIR}/internal/catch_interfaces_reporter.cpp
${HEADER_DIR}/internal/catch_random_number_generator.cpp
Expand Down
5 changes: 4 additions & 1 deletion docs/configuration.md
Expand Up @@ -121,6 +121,7 @@ by using `_NO_` in the macro, e.g. `CATCH_CONFIG_NO_CPP17_UNCAUGHT_EXCEPTIONS`.
CATCH_CONFIG_DISABLE_STRINGIFICATION // Disable stringifying the original expression
CATCH_CONFIG_DISABLE // Disables assertions and test case registration
CATCH_CONFIG_WCHAR // Enables use of wchart_t
CATCH_CONFIG_EXPERIMENTAL_REDIRECT // Enables the new (experimental) way of capturing stdout/stderr

Currently Catch enables `CATCH_CONFIG_WINDOWS_SEH` only when compiled with MSVC, because some versions of MinGW do not have the necessary Win32 API support.

Expand All @@ -131,7 +132,9 @@ Currently Catch enables `CATCH_CONFIG_WINDOWS_SEH` only when compiled with MSVC,
`CATCH_CONFIG_WCHAR` is on by default, but can be disabled. Currently
it is only used in support for DJGPP cross-compiler.

These toggles can be disabled by using `_NO_` form of the toggle, e.g. `CATCH_CONFIG_NO_WINDOWS_SEH`.
With the exception of `CATCH_CONFIG_EXPERIMENTAL_REDIRECT`,
these toggles can be disabled by using `_NO_` form of the toggle,
e.g. `CATCH_CONFIG_NO_WINDOWS_SEH`.

### `CATCH_CONFIG_FAST_COMPILE`
Defining this flag speeds up compilation of test files by ~20%, by making 2 changes:
Expand Down
131 changes: 131 additions & 0 deletions include/internal/catch_output_redirect.cpp
@@ -0,0 +1,131 @@
/*
* Created by Martin on 28/04/2018.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/

#include "catch_output_redirect.h"



#include <cstdio>
#include <cstring>
#include <fstream>
#include <sstream>

#if defined(CATCH_PLATFORM_WINDOWS)
#include <io.h> //_dup and _dup2
#define dup _dup
#define dup2 _dup2
#define fileno _fileno
#else
#include <unistd.h> // dup and dup2
#endif

namespace Catch {

RedirectedStream::RedirectedStream( std::ostream& originalStream, std::ostream& redirectionStream )
: m_originalStream( originalStream ),
m_redirectionStream( redirectionStream ),
m_prevBuf( m_originalStream.rdbuf() )
{
m_originalStream.rdbuf( m_redirectionStream.rdbuf() );
}

RedirectedStream::~RedirectedStream() {
m_originalStream.rdbuf( m_prevBuf );
}

RedirectedStdOut::RedirectedStdOut() : m_cout( Catch::cout(), m_rss.get() ) {}
auto RedirectedStdOut::str() const -> std::string { return m_rss.str(); }

RedirectedStdErr::RedirectedStdErr()
: m_cerr( Catch::cerr(), m_rss.get() ),
m_clog( Catch::clog(), m_rss.get() )
{}
auto RedirectedStdErr::str() const -> std::string { return m_rss.str(); }



#if defined(CATCH_PLATFORM_WINDOWS)
TempFile::TempFile() {
if (tmpnam_s(m_buffer)) {
throw std::runtime_error("Could not get a temp filename");
}
if (fopen_s(&m_file, m_buffer, "w")) {
char buffer[100];
if (strerror_s(buffer, errno)) {
throw std::runtime_error("Could not translate errno to string");
}
throw std::runtime_error("Could not open the temp file: " + std::string(m_buffer) + buffer);
}
}
#else
TempFile::TempFile() {
m_file = std::tmpfile();
if (!m_file) {
throw std::runtime_error("Could not create a temp file.");
}
}

#endif

TempFile::~TempFile() {
// TBD: What to do about errors here?
std::fclose(m_file);
// We manually create the file on Windows only, on Linux
// it will be autodeleted
#if defined(CATCH_PLATFORM_WINDOWS)
std::remove(m_buffer);
#endif
}


FILE* TempFile::getFile() {
return m_file;
}

std::string TempFile::getContents() {
std::stringstream sstr;
char buffer[100] = {};
std::rewind(m_file);
while (std::fgets(buffer, sizeof(buffer), m_file)) {
sstr << buffer;
}
return sstr.str();
}

OutputRedirect::OutputRedirect(std::string& stdout_dest, std::string& stderr_dest) :
m_originalStdout(dup(1)),
m_originalStderr(dup(2)),
m_stdoutDest(stdout_dest),
m_stderrDest(stderr_dest) {
dup2(fileno(m_stdoutFile.getFile()), 1);
dup2(fileno(m_stderrFile.getFile()), 2);
}

OutputRedirect::~OutputRedirect() {
Catch::cout() << std::flush;
fflush(stdout);
// Since we support overriding these streams, we flush cerr
// even though std::cerr is unbuffered
Catch::cerr() << std::flush;
Catch::clog() << std::flush;
fflush(stderr);

dup2(m_originalStdout, 1);
dup2(m_originalStderr, 2);

m_stdoutDest += m_stdoutFile.getContents();
m_stderrDest += m_stderrFile.getContents();
}


} // namespace Catch

#if defined(CATCH_PLATFORM_WINDOWS)
#undef dup
#undef dup2
#undef fileno
#endif
98 changes: 98 additions & 0 deletions include/internal/catch_output_redirect.h
@@ -0,0 +1,98 @@
/*
* Created by Martin on 28/04/2018.
*
* Distributed under the Boost Software License, Version 1.0. (See accompanying
* file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
*/
#ifndef TWOBLUECUBES_CATCH_OUTPUT_REDIRECT_H
#define TWOBLUECUBES_CATCH_OUTPUT_REDIRECT_H

#include "catch_platform.h"
#include "catch_stream.h"

#include <cstdio>
#include <iosfwd>
#include <string>

namespace Catch {

class RedirectedStream {
std::ostream& m_originalStream;
std::ostream& m_redirectionStream;
std::streambuf* m_prevBuf;

public:
RedirectedStream( std::ostream& originalStream, std::ostream& redirectionStream );
~RedirectedStream();
};

class RedirectedStdOut {
ReusableStringStream m_rss;
RedirectedStream m_cout;
public:
RedirectedStdOut();
auto str() const -> std::string;
};

// StdErr has two constituent streams in C++, std::cerr and std::clog
// This means that we need to redirect 2 streams into 1 to keep proper
// order of writes
class RedirectedStdErr {
ReusableStringStream m_rss;
RedirectedStream m_cerr;
RedirectedStream m_clog;
public:
RedirectedStdErr();
auto str() const -> std::string;
};


// Windows's implementation of std::tmpfile is terrible (it tries
// to create a file inside system folder, thus requiring elevated
// privileges for the binary), so we have to use tmpnam(_s) and
// create the file ourselves there.
class TempFile {
public:
TempFile(TempFile const&) = delete;
TempFile& operator=(TempFile const&) = delete;
TempFile(TempFile&&) = delete;
TempFile& operator=(TempFile&&) = delete;

TempFile();
~TempFile();

std::FILE* getFile();
std::string getContents();

private:
std::FILE* m_file = nullptr;
#if defined(CATCH_PLATFORM_WINDOWS)
char m_buffer[L_tmpnam] = { 0 };
#endif
};


class OutputRedirect {
public:
OutputRedirect(OutputRedirect const&) = delete;
OutputRedirect& operator=(OutputRedirect const&) = delete;
OutputRedirect(OutputRedirect&&) = delete;
OutputRedirect& operator=(OutputRedirect&&) = delete;


OutputRedirect(std::string& stdout_dest, std::string& stderr_dest);
~OutputRedirect();

private:
int m_originalStdout = -1;
int m_originalStderr = -1;
TempFile m_stdoutFile;
TempFile m_stderrFile;
std::string& m_stdoutDest;
std::string& m_stderrDest;
};


} // end namespace Catch

#endif // TWOBLUECUBES_CATCH_OUTPUT_REDIRECT_H
51 changes: 8 additions & 43 deletions include/internal/catch_run_context.cpp
Expand Up @@ -3,55 +3,14 @@
#include "catch_enforce.h"
#include "catch_random_number_generator.h"
#include "catch_stream.h"
#include "catch_output_redirect.h"

#include <cassert>
#include <algorithm>
#include <sstream>

namespace Catch {

class RedirectedStream {
std::ostream& m_originalStream;
std::ostream& m_redirectionStream;
std::streambuf* m_prevBuf;

public:
RedirectedStream( std::ostream& originalStream, std::ostream& redirectionStream )
: m_originalStream( originalStream ),
m_redirectionStream( redirectionStream ),
m_prevBuf( m_originalStream.rdbuf() )
{
m_originalStream.rdbuf( m_redirectionStream.rdbuf() );
}
~RedirectedStream() {
m_originalStream.rdbuf( m_prevBuf );
}
};

class RedirectedStdOut {
ReusableStringStream m_rss;
RedirectedStream m_cout;
public:
RedirectedStdOut() : m_cout( Catch::cout(), m_rss.get() ) {}
auto str() const -> std::string { return m_rss.str(); }
};

// StdErr has two constituent streams in C++, std::cerr and std::clog
// This means that we need to redirect 2 streams into 1 to keep proper
// order of writes
class RedirectedStdErr {
ReusableStringStream m_rss;
RedirectedStream m_cerr;
RedirectedStream m_clog;
public:
RedirectedStdErr()
: m_cerr( Catch::cerr(), m_rss.get() ),
m_clog( Catch::clog(), m_rss.get() )
{}
auto str() const -> std::string { return m_rss.str(); }
};


RunContext::RunContext(IConfigPtr const& _config, IStreamingReporterPtr&& reporter)
: m_runInfo(_config->name()),
m_context(getCurrentMutableContext()),
Expand Down Expand Up @@ -299,13 +258,19 @@ namespace Catch {
Timer timer;
try {
if (m_reporter->getPreferences().shouldRedirectStdOut) {
#if !defined(CATCH_CONFIG_EXPERIMENTAL_REDIRECT)
RedirectedStdOut redirectedStdOut;
RedirectedStdErr redirectedStdErr;

timer.start();
invokeActiveTestCase();
redirectedCout += redirectedStdOut.str();
redirectedCerr += redirectedStdErr.str();

#else
OutputRedirect r(redirectedCout, redirectedCerr);
timer.start();
invokeActiveTestCase();
#endif
} else {
timer.start();
invokeActiveTestCase();
Expand Down

0 comments on commit e92b9c0

Please sign in to comment.