/
CHANGES.txt
460 lines (383 loc) · 22.4 KB
/
CHANGES.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
NOTE: A combined description of changes since NUnit 2.6.3 is under development along with a summary statement of major features. Both will be available in the first Beta release.
NUnit 3.0.0 Alpha 4 - December 30, 2014
Console
* The command-line may now include any number of assemblies and/or supported projects.
Engine
* A driver is now included, which allows running NUnit 2.x tests under NUnit 3.0.
* The engine can now load and run tests specified in a number of project formats:
* NUnit (.nunit)
* Visual Studio C# projects (.csproj)
* Visual Studio F# projects (.vjsproj)
* Visual Studio Visual Basic projects (.vbproj)
* Visual Studio solutions (.sln)
* Legacy C++ and Visual JScript projects (.csproj and .vjsproj) are also supported
* Support for the current C++ format (.csxproj) is not yet available
* Creation of output files like TestResult.xml in various formats is now a
service of the engine, available to any runner.
Framework
* ApartmentAttribute has been added, replacing STAAttribute and MTAAttribute.
* Unnecessary overloads of Assert.That and Assume.That have been removed.
* Multiple SetUpFixtures may be specified in a single namespace.
* Improvements to the Pairwise strategy test case generation algorithm.
* The new NUnitLite runner --testlist option, allows a list of tests to be kept in a file.
Issues Resolved
* 37 Multiple SetUpFixtures should be permitted on same namespace
* 210 TestContext.WriteLine in an AppDomain causes an error
* 227 Add support for VS projects and solutions
* 231 Update C# samples to use NUnit 3.0
* 233 Update F# samples to use NUnit 3.0
* 234 Update C++ samples to use NUnit 3.0
* 265 Reorganize console reports for nunit-console and nunitlite
* 299 No full path to assembly in XML file under Compact Framework
* 301 Command-line length
* 363 Make Xml result output an engine service
* 377 CombiningStrategyAttributes don't work correctly on generic methods
* 388 Improvements to NUnitLite runner output
* 390 Specify exactly what happens when a test times out
* 396 ApartmentAttribute
* 397 CF nunitlite runner assembly has the wrong name
* 407 Assert.Pass() with ]]> in message crashes console runner
* 414 Simplify Assert overloads
* 416 NUnit 2.x Framework Driver
* 417 Complete work on NUnit projects
* 420 Create Settings file in proper location
NUnit 3.0.0 Alpha 3 - November 29, 2014
Breaking Changes
* NUnitLite tests must reference both the nunit.framework and nunitlite assemblies.
Framework
* The NUnit and NUnitLite frameworks have now been merged. There is no longer any distinction
between them in terms of features, although some features are not available on all platforms.
* The release includes two new framework builds: compact framework 3.5 and portable. The portable
library is compatible with .NET 4.5, Silverlight 5.0, Windows 8, Windows Phone 8.1,
Windows Phone Silverlight 8, Mono for Android and MonoTouch.
* A number of previously unsupported features are available for the Compact Framework:
- Generic methods as tests
- RegexConstraint
- TimeoutAttribute
- FileAssert, DirectoryAssert and file-related constraints
Engine
* The logic of runtime selection has now changed so that each assembly runs by default
in a separate process using the runtime for which it was built.
* On 64-bit systems, each test process is automatically created as 32-bit or 64-bit,
depending on the platform specified for the test assembly.
Console
* The console runner now runs tests in a separate process per assembly by default. They may
still be run in process or in a single separate process by use of command-line options.
* The console runner now starts in the highest version of the .NET runtime available, making
it simpler to debug tests by specifying that they should run in-process on the command-line.
* The -x86 command-line option is provided to force execution in a 32-bit process on a 64-bit system.
* A writeability check is performed for each output result file before trying to run the tests.
* The -teamcity option is now supported.
Issues Resolved
* 12 Compact framework should support generic methods
* 145 NUnit-console fails if test result message contains invalid xml characters
* 155 Create utility classes for platform-specific code
* 223 Common code for NUnitLite console runner and NUnit-Console
* 225 Compact Framework Support
* 238 Improvements to running 32 bit tests on a 64 bit system
* 261 Add portable nunitlite build
* 284 NUnitLite Unification
* 293 CF does not have a CurrentDirectory
* 306 Assure NUnit can write resultfile
* 308 Early disposal of runners
* 309 NUnit-Console should support incremental output under TeamCity
* 325 Add RegexConstraint to compact framework build
* 326 Add TimeoutAttribute to compact framework build
* 327 Allow generic test methods in the compact framework
* 328 Use .NET Stopwatch class for compact framework builds
* 331 Alpha 2 CF does not build
* 333 Add parallel execution to desktop builds of NUnitLite
* 334 Include File-related constraints and syntax in NUnitLite builds
* 335 Re-introduce 'Classic' NUnit syntax in NUnitLite
* 336 Document use of separate obj directories per build in our projects
* 337 Update Standard Defines page for .NET 3.0
* 341 Move the NUnitLite runners to separate assemblies
* 367 Refactor XML Escaping Tests
* 372 CF Build TestAssemblyRunnerTests
* 373 Minor CF Test Fixes
* 378 Correct documentation for PairwiseAttribute
* 386 Console Output Improvements
NUnit 3.0.0 Alpha 2 - November 2, 2014
Breaking Changes
* The console runner no longer displays test results in the debugger.
* The NUnitLite compact framework 2.0 build has been removed.
* All addin support has been removed from the framework. Documentation of NUnit 3.0 extensibility features will be published in time for the beta release. In the interim, please ask for support on the nunit-discuss list.
General
* A separate solution has been created for Linux
* We now have continuous integration builds under both Travis and Appveyor
* The compact framework 3.5 build is now working and will be supported in future releases.
New Features
* The console runner now automatically detects 32- versus 64-bit test assemblies.
* The NUnitLite report output has been standardized to match that of nunit-console.
* The NUnitLite command-line has been standardized to match that of nunit-console where they share the same options.
* Both nunit-console and NUnitLite now display output in color.
* ActionAttributes now allow specification of multiple targets on the attribute as designed. This didn't work in the first alpha.
* OneTimeSetUp and OneTimeTearDown failures are now shown on the test report. Individual test failures after OneTimeSetUp failure are no longer shown.
* The console runner refuses to run tests build with older versions of NUnit. A plugin will be available to run older tests in the future.
Issues Resolved
* 222 Color console for NUnitLite
* 229 Timing failures in tests
* 241 Remove reference to Microslft BCL packages
* 243 Create solution for Linux
* 245 Multiple targets on action attributes not implemented
* 246 C++ tests do not compile in VS2013
* 247 Eliminate trace display when running tests in debug
* 255 Add new result states for more precision in where failures occur
* 256 ContainsConstraint break when used with AndConstraint
* 264 Stacktrace displays too many entries
* 269 Add manifest to nunit-console and nunit-agent
* 270 OneTimeSetUp failure results in too much output
* 271 Invalid tests should be treated as errors
* 274 Command line options should be case insensitive
* 276 NUnit-console should not reference nunit.framework
* 278 New result states (ChildFailure and SetupFailure) break NUnit2XmlOutputWriter
* 282 Get tests for NUnit2XmlOutputWriter working
* 288 Set up Appveyor CI build
* 290 Stack trace still displays too many items
* 315 NUnit 3.0 alpha: Cannot run in console on my assembly
* 319 CI builds are not treating test failures as failures of the build
* 322 Remove Stopwatch tests where they test the real .NET Stopwatch
NUnit 3.0.0 Alpha 1 - September 22, 2014
Breaking Changes
* Legacy suites are no longer supported
* Assert.NullOrEmpty is no longer supported (Use Is.Null.Or.Empty)
General
* MsBuild is now used for the build rather than NAnt
* The framework test harness has been removed now that nunit-console is at a point where it can run the tests.
New Features
* Action Attributes have been added with the same features as in NUnit 2.6.3.
* TestContext now has a method that allows writing to the XML output.
* TestContext.CurrentContext.Result now provides the error message and stack trace during teardown.
* Does prefix operator supplies several added constraints.
Issues Resolved
* 6 Log4net not working with NUnit
* 13 Standardize commandline options for nunitlite runner
* 17 No allowance is currently made for nullable arguents in TestCase parameter conversions
* 33 TestCaseSource cannot refer to a parameterized test fixture
* 54 Store message and stack trace in TestContext for use in TearDown
* 111 Implement Changes to File, Directory and Path Assertions
* 112 Implement Action Attributes
* 156 Accessing multiple AppDomains within unit tests result in SerializationException
* 163 Add --trace option to NUnitLite
* 167 Create interim documentation for the alpha release
* 169 Design and implement distribution of NUnit packages
* 171 Assert.That should work with any lambda returning bool
* 175 Test Harness should return an error if any tests fail
* 180 Errors in Linux CI build
* 181 Replace NAnt with MsBuild / XBuild
* 183 Standardize commandline options for test harness
* 188 No output from NUnitLite when selected test is not found
* 189 Add string operators to Does prefix
* 193 TestWorkerTests.BusyExecutedIdleEventsCalledInSequence fails occasionally
* 197 Deprecate or remove Assert.NullOrEmpty
* 202 Eliminate legacy suites
* 203 Combine framework, engine and console runner in a single solution and repository
* 209 Make Ignore attribute's reason mandatory
* 215 Running 32-bit tests on a 64-bit OS
* 219 Teardown failures are not reported
Console Issues Resolved (Old nunit-console project, now combined with nunit)
* 2 Failure in TestFixtureSetUp is not reported correctly
* 5 CI Server for nunit-console
* 6 System.NullReferenceException on start nunit-console-x86
* 21 NUnitFrameworkDriverTests fail if not run from same directory
* 24 'Debug' value for /trace option is deprecated in 2.6.3
* 38 Confusing Excluded categories output
NUnit 2.9.7 - August 8, 2014
Breaking Changes
* NUnit no longer supports void async test methods. You should use a Task return Type instead.
* The ExpectedExceptionAttribute is no longer supported. Use Assert.Throws() or Assert.That(..., Throws) instead for a more precise specification of where the exception is expected to be thrown.
New Features
* Parallel test execution is supported down to the Fixture level. Use ParallelizableAttribute to indicate types that may be run in parallel.
* Async tests are supported for .NET 4.0 if the user has installed support for them.
* A new FileExistsConstraint has been added along with FileAssert.Exists and FileAssert.DoesNotExist
* ExpectedResult is now supported on simple (non-TestCase) tests.
* The Ignore attribute now takes a named parameter Until, which allows specifying a date after which the test is no longer ignored.
* The following new values are now recognized by PlatformAttribute: Win7, Win8, Win8.1, Win2012Server, Win2012ServerR2, NT6.1, NT6.2, 32-bit, 64-bit
* TimeoutAttribute is now supported under Silverlight
* ValuesAttribute may be used without any values on an enum or boolean argument. All possible values are used.
* You may now specify a tolerance using Within when testing equality of DateTimeOffset values.
* The XML output now includes a start and end time for each test.
Issues Resolved
* 8 [SetUpFixture] is not working as expected
* 14 CI Server for NUnit Framework
* 21 Is.InRange Constraint Ambiguity
* 27 Values attribute support for enum types
* 29 Specifying a tolerance with "Within" doesn't work for DateTimeOffset data types
* 31 Report start and end time of test execution
* 36 Make RequiresThread, RequiresSTA, RequiresMTA inheritable
* 45 Need of Enddate together with Ignore
* 55 Incorrect XML comments for CollectionAssert.IsSubsetOf
* 62 Matches(Constraint) does not work as expected
* 63 Async support should handle Task return type without state machine
* 64 AsyncStateMachineAttribute should only be checked by name
* 65 Update NUnit Wiki to show the new location of samples
* 66 Parallel Test Execution within test assemblies
* 67 Allow Expected Result on simple tests
* 70 EquivalentTo isn't compatible with IgnoreCase for dictioneries
* 75 Async tests should be supported for projects that target .NET 4.0
* 82 nunit-framework tests are timing out on Linux
* 83 Path-related tests fail on Linux
* 85 Culture-dependent NUnit tests fail on non-English machine
* 88 TestCaseSourceAttribute documentation
* 90 EquivalentTo isn't compatible with IgnoreCase for char
* 100 Changes to Tolerance definitions
* 110 Add new platforms to PlatformAttribute
* 113 Remove ExpectedException
* 118 Workarounds for missing InternalPreserveStackTrace in mono
* 121 Test harness does not honor the --worker option when set to zero
* 129 Standardize Timeout in the Silverlight build
* 130 Add FileAssert.Exists and FileAssert.DoesNotExist
* 132 Drop support for void async methods
* 153 Surprising behavior of DelayedConstraint pollingInterval
* 161 Update API to support stopping an ongoing test run
NOTE: Bug Fixes below this point refer to the number of the bug in Launchpad.
NUnit 2.9.6 - October 4, 2013
Main Features
* Separate projects for nunit-console and nunit.engine
* New builds for .NET 4.5 and Silverlight
* TestContext is now supported
* External API is now stable; internal interfaces are separate from API
* Tests may be run in parallel on separate threads
* Solutions and projects now use VS2012 (except for Compact framework)
Bug Fixes
* 463470 We should encapsulate references to pre-2.0 collections
* 498690 Assert.That() doesn't like properties with scoped setters
* 501784 Theory tests do not work correctly when using null parameters
* 531873 Feature: Extraction of unit tests from NUnit test assembly and calling appropriate one
* 611325 Allow Teardown to detect if last test failed
* 611938 Generic Test Instances disappear
* 655882 Make CategoryAttribute inherited
* 664081 Add Server2008 R2 and Windows 7 to PlatformAttribute
* 671432 Upgrade NAnt to Latest Release
* 676560 Assert.AreEqual does not support IEquatable<T>
* 691129 Add Category parameter to TestFixture
* 697069 Feature request: dynamic location for TestResult.xml
* 708173 NUnit's logic for comparing arrays - use Comparer<T[]> if it is provided
* 709062 "System.ArgumentException : Cannot compare" when the element is a list
* 712156 Tests cannot use AppDomain.SetPrincipalPolicy
* 719184 Platformdependency in src/ClientUtilities/util/Services/DomainManager.cs:40
* 719187 Using Path.GetTempPath() causes conflicts in shared temporary folders
* 735851 Add detection of 3.0, 3.5 and 4.0 frameworks to PlatformAttribute
* 736062 Deadlock when EventListener performs a Trace call + EventPump synchronisation
* 756843 Failing assertion does not show non-linear tolerance mode
* 766749 net-2.0\nunit-console-x86.exe.config should have a <startup /> element and also enable loadFromRemoteSources
* 770471 Assert.IsEmpty does not support IEnumerable
* 785460 Add Category parameter to TestCaseSourceAttribute
* 787106 EqualConstraint provides inadequate failure information for IEnumerables
* 792466 TestContext MethodName
* 794115 HashSet incorrectly reported
* 800089 Assert.Throws() hides details of inner AssertionException
* 848713 Feature request: Add switch for console to break on any test case error
* 878376 Add 'Exactly(n)' to the NUnit constraint syntax
* 882137 When no tests are run, higher level suites display as Inconclusive
* 882517 NUnit 2.5.10 doesn't recognize TestFixture if there are only TestCaseSource inside
* 885173 Tests are still executed after cancellation by user
* 885277 Exception when project calls for a runtime using only 2 digits
* 885604 Feature request: Explicit named parameter to TestCaseAttribute
* 890129 DelayedConstraint doesn't appear to poll properties of objects
* 892844 Not using Mono 4.0 profile under Windows
* 893919 DelayedConstraint fails polling properties on references which are initially null
* 896973 Console output lines are run together under Linux
* 897289 Is.Empty constraint has unclear failure message
* 898192 Feature Request: Is.Negative, Is.Positive
* 898256 IEnumerable<T> for Datapoints doesn't work
* 899178 Wrong failure message for parameterized tests that expect exceptions
* 904841 After exiting for timeout the teardown method is not executed
* 908829 TestCase attribute does not play well with variadic test functions
* 910218 NUnit should add a trailing separator to the ApplicationBase
* 920472 CollectionAssert.IsNotEmpty must dispose Enumerator
* 922455 Add Support for Windows 8 and Windows 2012 Server to PlatformAttribute
* 928246 Use assembly.Location instead of assembly.CodeBase
* 958766 For development work under TeamCity, we need to support nunit2 formatted output under direct-runner
* 1000181 Parameterized TestFixture with System.Type as constructor arguments fails
* 1000213 Inconclusive message Not in report output
* 1023084 Add Enum support to RandomAttribute
* 1028188 Add Support for Silverlight
* 1029785 Test loaded from remote folder failed to run with exception System.IODirectory
* 1037144 Add MonoTouch support to PlatformAttribute
* 1041365 Add MaxOsX and Xbox support to platform attribute
* 1057981 C#5 async tests are not supported
* 1060631 Add .NET 4.5 build
* 1064014 Simple async tests should not return Task<T>
* 1071164 Support async methods in usage scenarios of Throws constraints
* 1071343 Runner.Load fails on CF if the test assembly contains a generic method
* 1071861 Error in Path Constraints
* 1072379 Report test execution time at a higher resolution
* 1074568 Assert/Assume should support an async method for the ActualValueDelegate
* 1082330 Better Exception if SetCulture attribute is applied multiple times
* 1111834 Expose Random Object as part of the test context
* 1111838 Include Random Seed in Test Report
* 1172979 Add Category Support to nunitlite Runner
* 1203361 Randomizer uniqueness tests sometimes fail
* 1221712 When non-existing test method is specified in -test, result is still "Tests run: 1, Passed: 1"
* 1223294 System.NullReferenceException thrown when ExpectedExceptionAttribute is used in a static class
* 1225542 Standardize commandline options for test harness
Bug Fixes in 2.9.6 But Not Listed Here in the Release
* 541699 Silverlight Support
* 1222148 /framework switch does not recognize net-4.5
* 1228979 Theories with all test cases inconclusive are not reported as failures
NUnit 2.9.5 - July 30, 2010
Bug Fixes
* 483836 Allow non-public test fixtures consistently
* 487878 Tests in generic class without proper TestFixture attribute should be invalid
* 498656 TestCase should show array values in GUI
* 513989 Is.Empty should work for directories
* 519912 Thread.CurrentPrincipal Set In TestFixtureSetUp Not Maintained Between Tests
* 532488 constraints from ConstraintExpression/ConstraintBuilder are not reusable
* 590717 categorie contains dash or trail spaces is not selectable
* 590970 static TestFixtureSetUp/TestFixtureTearDown methods in base classes are not run
* 595683 NUnit console runner fails to load assemblies
* 600627 Assertion message formatted poorly by PropertyConstraint
* 601108 Duplicate test using abstract test fixtures
* 601645 Parametered test should try to convert data type from source to parameter
* 605432 ToString not working properly for some properties
* 606548 Deprecate Directory Assert in 2.5 and remove it in 3.0
* 608875 NUnit Equality Comparer incorrectly defines equality for Dictionary objects
NUnit 2.9.4 - May 4, 2010
Bug Fixes
* 419411 Fixture With No Tests Shows as Non-Runnable
* 459219 Changes to thread princpal cause failures under .NET 4.0
* 459224 Culture test failure under .NET 4.0
* 462019 Line endings needs to be better controlled in source
* 462418 Assume.That() fails if I specify a message
* 483845 TestCase expected return value cannot be null
* 488002 Should not report tests in abstract class as invalid
* 490679 Category in TestCaseData clashes with Category on ParameterizedMethodSuite
* 501352 VS2010 projects have not been updated for new directory structure
* 504018 Automatic Values For Theory Test Parameters Not Provided For bool And enum
* 505899 'Description' parameter in both TestAttribute and TestCaseAttribute is not allowed
* 523335 TestFixtureTearDown in static class not executed
* 556971 Datapoint(s)Attribute should work on IEnumerable<T> as well as on Arrays
* 561436 SetCulture broken with 2.5.4
* 563532 DatapointsAttribute should be allowed on properties and methods
NUnit 2.9.3 - October 26, 2009
Main Features
* Created new API for controlling framework
* New builds for .Net 3.5 and 4.0, compact framework 3.5
* Support for old style tests has been removed
* New adhoc runner for testing the framework
Bug Fixes
* 432805 Some Framework Tests don't run on Linux
* 440109 Full Framework does not support "Contains"
NUnit 2.9.2 - September 19, 2009
Main Features
* NUnitLite code is now merged with NUnit
* Added NUnitLite runner to the framework code
* Added Compact framework builds
Bug Fixes
* 430100 Assert.Catch<T> should return T
* 432566 NUnitLite shows empty string as argument
* 432573 Mono test should be at runtime
NUnit 2.9.1 - August 27, 2009
General
* Created a separate project for the framework and framework tests
* Changed license to MIT / X11
* Created Windows installer for the framework
Bug Fixes
* 400502 NUnitEqualityComparer.StreamsEqual fails for same stream
* 400508 TestCaseSource attirbute is not working when Type is given
* 400510 TestCaseData variable length ctor drops values
* 417557 Add SetUICultureAttribute from NUnit 2.5.2
* 417559 Add Ignore to TestFixture, TestCase and TestCaseData
* 417560 Merge Assert.Throws and Assert.Catch changes from NUnit 2.5.2
* 417564 TimeoutAttribute on Assembly