Submitted version.

JEREMY_WRIGHT_A1.tex

@@ -0,0 +1,156 @@
+\documentclass[12pt,journal]{article}
+\hyphenation{op-tical net-works semi-conduc-tor}
+
+\usepackage{url}
+\usepackage[hidelinks]{hyperref}
+\usepackage[backend=biber,style=ieee]{biblatex}
+\addbibresource{thesis.bib}
+\usepackage{geometry}
+\usepackage{fancyhdr}
+\usepackage{afterpage}
+\usepackage{graphicx}
+\usepackage{amsmath,amssymb,amsbsy}
+\usepackage{pdflscape}
+\usepackage{tikz}
+\def\checkmark{\tikz\fill[scale=0.4](0,.35) -- (.25,0) -- (1,.7) -- (.25,.15) -- cycle;} 
+\usepackage[activate={true,nocompatibility},final,tracking=true,kerning=true,spacing=true,factor=1100,stretch=10,shrink=10]{microtype}
+% activate={true,nocompatibility} - activate protrusion and expansion
+% final - enable microtype; use "draft" to disable
+% tracking=true, kerning=true, spacing=true - activate these techniques
+% factor=1100 - add 10% to the protrusion amount (default is 1000)
+% stretch=10, shrink=10 - reduce stretchability/shrinkability (default is 20/20)
+\usepackage{dcolumn,array}
+\usepackage{tocloft}
+\usepackage[section]{placeins}
+\usepackage[english]{babel}
+\usepackage{todonotes}
+\usepackage{blindtext}
+\usepackage{amsthm}
+\usepackage{setspace}
+\usepackage[babel=true]{csquotes}
+\blindmathtrue
+\usepackage[acronym]{glossaries}
+\usepackage[section]{algorithm}
+\usepackage{algpseudocode}
+\usepackage{listings}
+\newtheorem{mydef}{Definition}
+
+\begin{document}
+\doublespace
+\title{Unit Test Frameworks \\ CSE 565 Assignment \#1}
+\author{Jeremy Wright - 1000738685}
+
+% make the title area
+\maketitle
+
+\section{Frameworks}
+Unit testing should be part of an overall test strategy to
+ensure product quality. Unit testing, forms the smallest indivisible unit. Test
+frameworks provide organization tools, to help the developer control the growing
+mass of tests.
+%\section{What is unit testing}
+%Wikipedia defines unit testing as testing, ``... a unit as the smallest testable
+%part of an application. \autocite{wikipedia_unit_2014}'' I posed this question
+%to a colleague of mine, and his response was, ``A unit test de-tangles
+%dependencies so one can test a submodules prior to merging it's functionality
+%with another module.'' \autocite{brandenburg}. I like this definition since it
+%attacks the heart of a problem.  There is a block of code that needs to be
+%tested, it could be large or small by you are concerned with its dependencies
+%prior to making the block larger. It speaks to the incremental, or even design
+%motivating potential of unit tests. I feel, when used properly, unit tests can
+%how motivate the design, since it forced one to think about the interface
+%up-front. The unit-test by design will use the interface, hence making it easy
+%to test, tends to make the module easier to understand.
+%
+%However, in practice unit testing can be a more visceral tool. While it does
+%function as a design tool, its greatest return on investment is its ability to
+%test edge conditions, and in a sense embarrassing bugs. Dr. Collofello asked in
+%class, ``What if a tester wrote the unit tests instead of the developer? Would
+%that be a good idea?''   My initial response was, those are my tests. Hence,
+%I now use Definition~\ref{def:person_unit_test}.
+%
+%\begin{mydef} 
+%    \label{def:person_unit_test}
+%A unit test is a test to provide the developer some confidence
+%a region of code works as intended before other people see the code and the
+%developer is open to mockery and scandal.
+%\end{mydef}
+
+A framework is by no means necessary to perform unit testing. One method of unit
+testing is to simply create separate executables for each test, and use simple
+asserts to check for pre and post conditions. This method makes command line
+debugging easier since each test is simply a basic executable. Test frameworks
+however are quite useful when you want integration with an IDE, or continuous
+integration system such as XML output for a build server. 
+
+A unit test framework however can be quite an enabler, since it doesn't forget
+to run all the tests. When not using such a framework, one has to setup some
+form of automation to ensure all tests are run with every build. Build systems
+such as autoconf, and CMake both support such automation, but sometimes it's
+nicer to have a test tool do the work, especially when you want to generates
+reports off the results.
+
+Frameworks are typically build time dependencies, and allow developers to
+annotate, or mark up their code in some manner. The test generation step then
+discovers tests and generates an executable.  Frameworks also provide
+organization tools called test suites which allow one to control SetUp, and
+TearDown of the suite, as well as which tests get grouped together.
+
+A favorite feature of mine I picked up from FPGA design is constrained random
+test generation \autocite{constrained}. Sorting functions are ideal candidates
+since there input domain is large, random test generation can expose test cases
+the developer may not have thought of directly.  Test frameworks which allow for
+generic programming, and runtime test generation are big motivators for me to
+use a test framework in my own designs.
+
+\begin{table}[h!]
+    \centering
+    \begin{tabular}{l | c | c }
+        \hline
+        & JUnit \autocite{junit} & GTest \autocite{gtest}\\
+        \hline \hline
+        Language                 & Java  & C++ \\
+        Test Generation          & \checkmark \autocite{google_java_tools}
+        & \checkmark
+        (Section ~\ref{sec:source_code})\\
+        Build Server Integration & JUnit XML & Creates
+        JUnit compatible XML\\
+        IDE Integration          & Eclipse, IntelliJ  &  None \\
+        Generic Programming      & Java Generics  &  Templates\\
+        Randomization      & Override test runner  &  Test Shuffling\\
+        Test Discovery  & Annotations & Macros \\
+        Verification Technique & Assert Functions & Assert Macros \\
+        License &  Eclipse Public License Version 1.0 & BSD-3-Clause \\
+        Integration & Maven & Manual Compilation \\
+        \hline
+    \end{tabular}
+    \caption{Comparison of GTest and JUnit}
+    \label{tab:comparison}
+\end{table}
+
+\section{Part 2 Demonstration}
+The demonstration code verifies a simple selection sort algorithm.  It does so
+by generating 10,000 random test vectors. Each vector size is uniformly
+distributed between 0 and 1500 elements. Each element in the vector is uniformly
+distributed over the range of a 32-bit integer. Using this generated input data,
+Google Test instantiates 10,000 instances of the RandomizedTest class, and
+executes the tests. 
+\singlespace
+\subsection{Unit under test}
+\label{sec:source_code}
+\lstinputlisting[basicstyle=\ttfamily,language=c++]{sort.hpp}
+\subsection{Unit Test}
+\lstinputlisting[basicstyle=\ttfamily,language=c++]{test.cpp}
+\subsection{Unit Test Result}
+\lstinputlisting[basicstyle=\ttfamily]{test_output.result}
+\subsection{CMake Build Script}
+\lstinputlisting[basicstyle=\ttfamily]{CMakeLists.txt}
+
+\clearpage
+
+\printbibliography
+
+
+
+\end{document}
+

Makefile

@@ -0,0 +1,49 @@
+main_tex=JEREMY_WRIGHT_A1.tex
+
+main_pdf=$(patsubst %.tex,%.pdf,$(main_tex))
+tex_files=$(wildcard *.tex)
+dot_files=$(wildcard *.dot)
+dot_figures=$(patsubst %.dot,%.pdf,$(dot_files))
+svg_files=$(wildcard *.svg)
+svg_figures=$(patsubst %.svg,%.pdf,$(svg_files))
+octave_files=$(wildcard *.m)
+octave_figures=$(patsubst %.m,%.tex,$(octave_files))
+
+.SUFFIXES: .png .dot .pdf .tex .txt .dat .svg .m
+.PHONY: clean all distclean spell
+
+all: $(main_pdf) 
+
+$(main_pdf): $(main_tex) thesis.bib $(tex_files) $(dot_figures) $(svg_figures) $(octave_figures) 
+
+personal.aspell: $(HOME)/.vim/spell/en.utf-8.add
+	aspell --lang=en create master ./personal.aspell < ~/dotfiles/vim/spell/en.utf-8.add
+
+spell: $(tex_files) personal.aspell
+	$(foreach doc,$(tex_files), aspell --personal personal.aspell --lang=en --mode=tex check $(doc);)
+
+.tex.pdf: 
+	latexmk -pdf --halt-on-error $< > compile.log
+
+.pdf.txt:
+	ps2txt $< $@
+
+.txt.dat:
+	style $< > $@
+
+.dot.pdf:
+	dot -Tpdf $< > $@
+
+.svg.pdf:
+	inkscape --export-area-drawing --export-latex -A$@ $<
+
+.m.tex:
+	octave $<
+
+distclean: clean
+	rm -rf *.pdf
+
+clean:
+	rm -rf *.log *.acn *.aux *.bbl *.bcf *.blg *.glo *.glsdefs *.ist *.log *.out *.run.xml *.toc *.acr *.alg *.glg *.gls *.fdb_latexmk *.dvi *.fls *.png *.dat *.pdf_tex $(dot_figures) $(svg_figures) $(octave_figures) *.tdo *.lof *.lot *-inc.eps personal.aspell
+
+

main.cpp

@@ -3,6 +3,8 @@
 #include <random>
 #include <limits>
 #include <cassert>
+#include <iostream>
+
 int main(int argc, const char *argv[])
 {
     std::numeric_limits<int> limits;

sort.hpp

@@ -1,21 +1,17 @@
 #include <iterator>
-#include <iostream>
 #include <algorithm>
 
 template <typename InputIterator>
 void selection_sort(InputIterator b, InputIterator e)
 {
-
     //Selection sort performs the following steps:
-    //1) Starting at index 0, search the entire array to find the
-    //smallest value
-    //2) Swap the smallest value found with the value at index 0
-    //3) Repeat steps 1 & 2 starting from the next index
+    //1) From the current iterator, find the smallest value
+    //2) Swap the smallest value with the current iterator
+    //3) Continue until end of range
 
     for(InputIterator c = b; c != e ; ++c)
     {
-        auto m = std::min_element(c, e);
-        std::swap(*m, *c);
+        std::swap(*(std::min_element(c, e)), *c);
     }
 
 }

test.cpp

@@ -22,22 +22,27 @@ std::vector<std::vector<int>> GenerateTestCases()
     std::vector<TestInput> test_cases;
     std::numeric_limits<int> limits;
 
-    std::mt19937 engine(time(0)); // Fixed seed of 0
+    std::mt19937 engine(time(0));
     std::uniform_int_distribution<int> range_dist(0,1500);
-    std::uniform_int_distribution<int> element_dist(0, limits.max());
+    std::uniform_int_distribution<int> element_dist(
+            limits.min(), 
+            limits.max());
 
     size_t n = 10000;
     auto f = [&](){return element_dist(engine);};
     std::generate_n(std::back_inserter(test_cases), n, 
             [&](){ 
             TestInput vs;
-            std::generate_n(std::back_inserter(vs), range_dist(engine), f);
+            std::generate_n(std::back_inserter(vs), 
+                range_dist(engine), 
+                f);
             return vs;
             });
     return test_cases;
 }
 
 INSTANTIATE_TEST_CASE_P(
-        GeneralAndSpecial,
-        RandomizedTest,
-        testing::ValuesIn(GenerateTestCases()));
+    hw1,
+    RandomizedTest,
+    testing::ValuesIn(GenerateTestCases())
+);

test_output.result

@@ -0,0 +1,18 @@
+Running main() from gtest_main.cc
+[==========] Running 10000 tests from 1 test case.
+[----------] Global test environment set-up.
+[----------] 10000 tests from hw1/RandomizedTest
+[ RUN      ] hw1/RandomizedTest.Sorting/0
+[       OK ] hw1/RandomizedTest.Sorting/0 (2 ms)
+[ RUN      ] hw1/RandomizedTest.Sorting/1
+... Test output truncated for clarity ...
+[       OK ] hw1/RandomizedTest.Sorting/9997 (0 ms)
+[ RUN      ] hw1/RandomizedTest.Sorting/9998
+[       OK ] hw1/RandomizedTest.Sorting/9998 (1 ms)
+[ RUN      ] hw1/RandomizedTest.Sorting/9999
+[       OK ] hw1/RandomizedTest.Sorting/9999 (0 ms)
+[----------] 10000 tests from hw1/RandomizedTest (4576 ms total)
+
+[----------] Global test environment tear-down
+[==========] 10000 tests from 1 test case ran. (4578 ms total)
+[  PASSED  ] 10000 tests.