-
Notifications
You must be signed in to change notification settings - Fork 843
/
Copy pathUnitTesting.html
232 lines (193 loc) · 8.79 KB
/
UnitTesting.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title>Unit Testing</title>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
<style type="text/css">
<!--
body {
font-family: Verdana,Helvetica,Arial,sans-serif;
font-size: small;
}
code,pre {
font-family: "Courier New",Courier,monospace;
font-size: 1em;
}
h3 {
padding: 2px;
border-left: 4px solid #FFCC00;
border-bottom: 1px solid #FFCC00;
}
h4 {
padding: 2px;
border-left: 8px solid #FF9933;
border-bottom: 1px solid #FF9933;
}
h5 {
font-size: small;
}
pre {
margin-left: 25px;
margin-right: 25px;
padding: 5px;
background-color: #EEEEEE;
border-left: 10px solid #CCCCCC;
}
p.note {
padding: 5px;
background-color: #DDDDFF;
border: 1px solid #6666FF;
}
acronym {
cursor: help;
border-bottom: 1px dotted black;
}
-->
</style>
</head>
<body>
<h2>Unit Testing</h2>
<p><i>Unit testing</i> means running automated tests for the certain block of
the code. These tests are run either by scripts or unit testing framework. The
result is a list os passed and/or failed tests.</p>
<p>Unit testing framework used with WinMerge development is the Google Test,
available from <a href="http://code.google.com/p/googletest/">
http://code.google.com/p/googletest/</a>. Other frameworks can be used when
need arises. However we don't want to use every known framework so the reason
for adding new framework needs to be good, like new technology or language
used that current framework does not support.</p>
<p>Earlier we tried to start using CppUnit but unfortunately even compiling
CppUnit seemed to be too hard for many people. And creating tests is just a
mess. So CppUnit usage is discouraged nowadays.</p>
<h3>Policy</h3>
<p><b>Existing unit tests must be run without errors.</b> For every error
there must be a known explanation. It is a good practice to document errors
into unit tests so others know what is going on. It is also recommended to
submit bug reports of failing tests if the fix is not imminent.</p>
<p>When changing the code which has unit tests, test must be always run before
the patch is submitted to patch tracker and before the patch is committed to
SVN. Possible known failures must be explained. Failing unit can be a blocker
preventing committing the patch.</p>
<p><b>Failing unit tests may be (and most likely are) blockers for the beta and
stable releases.</b> Experimental releases can be released with known errors.
<h3>Adding new tests</h3>
<p>Unit tests by nature rarely are and can be complete. Adding more tests and
improving current tests is highly encouraged. It is the easiest way to improve
WinMerge quality!</p>
<h4>Modifying existing tests</h4>
<p>There must be some care when altering existing tests though. It may be that
there was a reason the existing test were done as it was. If that reason is not
obvious, it is better to create new improved test and leave the old test
untouched.</p>
<p>Fixing a failing unit testing by modifying the test is highly questionable.
The test must be <i>buggy</i> for this to be allowed.</p>
<h2>Quick guide to Google Test</h2>
<p><i>Google Test</i> is an external library/framework that must be compiled
before using with unit tests.</p>
<h3>Unzip and compile Google Test library</h3>
<p>Unzip the Google Test sources to some folder. Pick a folder you can keep
around (not some temp folder) because you'll need the header files every time
you compile unit tests.</p>
<p>Open solution file <code>msvc\gtest.sln</code> to the Visual Studio.
Depending on Visual Studio version the project/solution file may need to be
updated for later version before usage. Visual Studio asks about the conversion
when opening the solution file.</p>
<p>Build both <i>Debug</i> and <i>Release</i> targets of <i>gtest</i> project. You
don't need to build other projects. Indeed, <i>rebuild all</i> for the solution
does not work. Some of the projects in the solution are unit tests for the
framework itself.</p>
<p>Copy the compiled <code>gtest.lib</code> and <code>gtestd.lib</code> files
from <code>msvc\Release</code> and <code>msvc\Debug</code> subfolders to
<code>Testing\GoogleTest</code> - folder in WinMerge source tree.
</p>
<h3>Setup Visual Studio for Google Test</h3>
<p>As mentioned earlier Google Test header files are needed to compile the
tests. As the Google Test was unzipped to separate folder we must first tell
Visual Studio where to find those headers.</p>
<p>Add the <code>gtest-[version]\include</code> -folder to Visual Studio's
include file folders:
<ul>
<li>Open Options -dialog in Visual Studio</li>
<li>Select Projects and solutions</li>
<li>Select VC++ Directories</li>
<li>Select Include files</li>
<li>Add <code>gtest-[version]\include</code> -folder as <b>LAST</b> folder in
that list</li>
</ul>
<h3>Open the test project into Visual Studio and compile</h3>
<p>Open the testing project to Visual Studio (VS 2003 and later work) and
compile <i>Debug</i> or <i>Release</i> -target of the project.</p>
<h3>Run the tests</h3>
<p>Open the Command Prompt and CD to folder where executable were build,
usually <code>[projectname]/Debug</code> or <code>[projectname]/Release</code>.
</p>
<h3>Run the executable.</h3>
<p>While the testing runs it prints progress information and info about
<i>passed</i>/<i>failed</i> tests.</p>
<h2>Creating new test project</h2>
<p>Initial creation of new test project takes some effort and time for the
first time. It is still pretty straightforward so nobody should have any real
problems with it. If somebody has, please tell Kimmo about it and we'll
improve the documentation!</p>
<h3>Create new VS project</h3>
<p>Open the Visual Studio and select creating new project. Select <i>Win32
Console Project</i> as the project type. You can create some other type too and
then just fix the properties. But console project selects many properties
correctly for you.</p>
<p><b>Note:</b> If you are creating tests for class depending on MFC, then
remember to include MFC (libraries) to the project.</p>
<p>Add new test projects into subfolders to folder <code>Testing\GoogleTest
</code>. E.g. <code>Testing\GoogleTest\CommandLine</code>. The project (folder)
name should somehow relate the to the code/functionality/class you are testing.
</p>
<h3>Set the project properties</h3>
<p>Several project settings need to be changed to make test project compile
(properly). There might be better choises and practices, but this is the
best setup we currently know. If you have ideas for improving, let us know!</p>
<p>Set the C/C++ properties according list below:</p>
<ul>
<li><b>General/Additional include folders:</b>
"..\..\..\Src";"..\..\..\Src\Common" (note three "dir up"s)</li>
</ul>
<p>Set the Linker properties according list below:</p>
<ul>
<li><b>Input/Additional dependencies:</b> <code>gtest.lib</code> (note: you
may use debug version <code>gtestd.lib</code> in debug target)
</ul>
<h3>Add test file(s)</h3>
<p>Next you need to add new C++ code (<code>.cpp</code>) file to the project.
Name the file as [projectname]<code>_test.cpp</code> and make sure the file
name does not overlap with the C++ files you are testing! <code>_test</code> as
part of the filen name helps.</p>
<p>Copy/paste the beginning of existing test file to the new <code>.cpp
</code> -file. Copy e.g. from <code>Testing\GoogleTest\Paths\paths_test.cpp
</code>. Copy the content to the first test function (<code>TEST_F()</code>).
Don't copy the test functions, but copy the namespace closing brace and <code>
main()</code> function from the end of the file.</p>
<p>Rename the test class to describe your tests (e.g. <code>CommandLineTest</code>
). Remember to rename constructor/destructor methods too!
<h3>Add tests</h3>
<p>Now you can start adding tests. Look Google Test documentation and existing
unit tests for help on that.</p>
<p>The simple test method looks like this:</p>
<pre>
TEST_F(PathTest, Create_abspath1)
{
EXPECT_TRUE(paths_CreateIfNeeded("c:\\Temp"));
}
</pre>
<p>The <code>TEST_F</code> macro defines a new test function, having test class
(<code>PathTest</code>) and test name (<code>Create_abspath1</code>) as
parameters. Then there is a function call done inside test macro. The framework
defines several macros for the tests. The <code>EXPECT_TRUE</code> macro makes
the test succeed if the parameter (function return value) is true, otherwise the
test fails.</p>
<h3>Split tests to several cases/files</h3>
<p>GoogleTests allows splitting tests to several cases and several files.
Adding new test case is easy: just add new file and create new test class
there. Don't copy <code>main()</code> function. Eatch test project should have
only one <code>main()</code>.
</p>
</body>
</html>