/
CHANGES.txt
533 lines (421 loc) · 22.1 KB
/
CHANGES.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
===========================================================================
5.6
Added: Suite files that only contain other suites do not get reported
Fixed: @BeforeClass methods would incorrectly report cyclic graphs
Added: get/setAttribute to ITestContext
Added: plugging in factory objects to handle the actual instantiation of tests
Added: dataProvider to @Factory
Added: ISuite now gives access to the current XmlSuite
Fixed: TESTNG-139 dependsOnMethods gets confused when dependency is "protected"
Fixed: TESTNG-141 junit attribute set to false in testng-failed.xml when it should be true
Fixed: TESTNG-142 Exceptions in DataProvider are not reported as failed test
Added: Improved behavior for @Before/@AfterClass when using @Factory
(http://forums.opensymphony.com/thread.jspa?threadID=6594&messageID=122294#122294)
Added: Support for concurrent execution for invocationCount=1 threadPoolSize>1 and @DataProvider
(http://forums.opensymphony.com/thread.jspa?threadID=64738&tstart=0)
Eclipse plug-in
Fixed: Bug that made group launch configurations unusable
===========================================================================
5.5
Fixed: @BeforeGroup methods were run twice when in a base class
Fixed: @BeforeGroup methods were run twice with a @Test at class level
Fixed: parallel="tests" didn't work as advertised
Added: Support for thread-count at test level
Added: Method selectors receive a Context and can stop the chain with setStopped()
Fixed: XmlMethodSelector was always run first regardless of its priority
Added: @BeforeGroups/@AfterGroups can live in classes without @Test methods
Added: DataProvider can now take an ITestContext parameter
Fixed: Wasn't parsing <selector-class-name> correctly
Fixed: Annotation Transformers now work on class-level annotations
Fixed: Some class-level @Test attributes were not always honored
Added: Clean separation between @Test invocation events and @Configuration invocation events
(see also TESTNG-111)
Added: Test instances created by @Factory now run in multiple threads in parallel mode
Fixed: @Before/@AfterGroups invocation order
Fixed: TESTNG-27: Parameters are not used on <test> level anymore
Fixed: TESTNG-107 don't create an output directory if "outputDirectory" is null
Fixed: TESTNG-127 UseDefaultListeners in Ant Task does not work
Fixed: TESTNG-119 Running TestNG runner with invalid '-sourcedir' on JDK14 JavaDoc annotated test classes won't fail.
Fixed: TESTNG-113 Dependent methods within the same static inner class are not found
Fixed: TESTNG-125 TestNG failed for test classes under *.java*.* pakages
Eclipse plug-in
Fixed: issue with launch configuration
Fixed: TESTNG-124: setting location of testng reports output
===========================================================================
5.4
Fixed: Ant task issue with paths containing spaces
Added: for @BeforeGroups and @AfterGroups specifying the groups() attribute will auto-include the method
into those groups by default (previously you had to also provide the value() attribute).
Added: the load @Tests (invocationCount + threadPoolSize) are triggered simultaneous
Fixed: reports are correctly displaying the thread info
Added: @DataProvider name defaults to method name
Added: support for remote protocol to pass parameter information
Fixed: TextReporter logs information about the parameters of the test methods
Fixed: concurrency issue in JUnitXMLReporter
Fixed: output of JUnitXMLReporter must be CDATA
Fixed: XML unsupported annotations/parallel attribute values are reported
Eclipse plug-in
Fixed: groups with multi-attribute javadoc annotations
Fixed: consistent behavior for dependsOnMethods
Fixed: consistent behavior for tests with dependsOnGroups (a warning is emitted)
Fixed: consistent merge of configuration arguments when an existing launch configuration exists
===========================================================================
5.3
Fixed: use a single instance of bsh.Interpreter
Added: @Before/@AfterMethod can declare a java.lang.reflect.Method parameter to be informed about the @Test method
Fixed: super classes must not be listed in testng-failures.xml
Fixed: parallel attribute must not appear if empty or null in testng-failures.xml
Fixed: parsing for javadoc annotations is done on request only
Added: improved multiple suite summary page report
Added: -target option deprecated in favor of -annotations javadoc|jdk
Fixed: filesets in the ant task didn't work if the paths have spaces in them
Fixed: Before/After Suite were behaving wrong in parallel execution
Added: A generic/extensible RemoteTestNG was added to the core
Fixed: Before/AfterGroup-s were behaving wrong when using invocationCount, dataProvider and threadPoolSize
Fixed: improved support for running different annotation type tests in the same suite
Fixed: testng-failed.xml was generated even if there were no failures/skipps
Fixed: -usedefaultlisteners was wrongly passed to JVM instead of TestNG options
Added: Attribute dataProviderClass for @Test and @testng.test
Fixed: Forgot to account for cases where both invocationCount and DataProviders are present
Fixed: AfterGroups were invoked out of order with invocationCount and DataProviders
Fixed: Reporter.getOutput() returned an empty array if a timeOut was specified
Added: testng.xml now supports <suite-files>
Added: ant task can receive several listeners
Fixed: TESTNG-109 Skipped tests with expected exceptions are reported as failures
Added: ant task can now select the parallel mode for running tests
Fixed: ant task correctly deals with empty groups and excludedgroups parameters
Added: ant task can override default suite and test names
Added: comand line support for setting parallel mode, suite and test names
Eclipse plug-in
Added: Support for configuring per project usedefaultlisteners
Added: Contextual drop-down menu on failures tab of the TestNG view to enable running/debugging method failure only
Added: Suppport for configuring per project TestNG jar usage (project provided one or plugin provided one)
===========================================================================
5.2
Added: "-usedefaultlisteners true/false" to command line and ant
Added: EmailableReporter (from Paul Mendelson)
Added: parallel can now be "methods" or "tests". Boolean version deprecated
Added: TestNGAntTask now uses the @ syntax to invoke TestNG
Added: Command line understands @ syntax
Added: JUnitConverter uses the new syntax
Added: -groups to JUnitConverter
Fixed: Throw proper exception when a DataProvider declares parameters
Added: completely revamped JUnit support (should run all kind of JUnit tests)
Fixed: TESTNG-40 (Bug in testng-failed.xml generation)
Fixed: TESTNG-106 (Failed "@BeforeSuite" method just skipps the last test in xml-file)
Fixed: Success on 0 tests (http://forums.opensymphony.com/thread.jspa?threadID=41213)
Eclipse plug-in
Added: TESTNG-105 Automaticaly define TESTNG_HOME classpath variable
===========================================================================
5.1
Added: @Test(sequential = true)
Fixed: TESTNG-102 (Incorrect ordering of @BeforeMethod calls when a dependency is specified)
Fixed: TESTNG-101 (HTML output contains nested <P> tags and a missing <tr> tag)
Added: support for specifying test-only classpath (http://forums.opensymphony.com/thread.jspa?messageID=78048&tstart=0)
Fixed: TESTNG-93 (method selectors filtering @BeforeMethod)
Fixed: TESTNG-81 (Assert.assertFalse() displays wrong expected, actual value)
Fixed: TESTNG-59 (multiple method selectors usage results in no tests run)
Fixed: TESTNG-56 (invocation of @Before/AfterClass methods in parallel/sequential scenarios)
Fixed: TESTNG-40 (failures suite does not contain @Before/After Suite/Test methods)
Fixed: TESTNG-37 (allow passing null parameter value from testng.xml)
Fixed: TESTNG-7 (display classname when hovering method)
Eclipse plug-in
Added: run contextual test classes with parameters from suite definition files
Added: TESTNG-100 (Show HTML reports after running tests)
Added: TESTNG-97 (Double click top stack to raise comparison)
Added: TESTNG-84 (plug-in UI for suite option should support absolute path)
Added: TESTNG-20 (copy stack trace)
Fixed: TESTNG-72 (display groups with non-array values)
Fixed: TESTNG-64 (Eclipse plug-in applies added groups to all launch configurations)
Fixed: TESTNG-28 (Cannot select groups from dependent eclipse projects)
Fixed: TESTNG-25 (do not display fully qualified method name when running contextual test class)
Improved behavior:
TESTNG-98 (temporary files have guaranteed fixed names)
TESTNG-95 (Assertion failed comparison trims trailing ">")
TESTNG-70 (TestNG prevents eclipse from opening an older CVS version of a java class)
display of test hierarchy information (TESTNG-29)
===========================================================================
5.0.1
Eclipse plug-in
Added: Output directory for the tests
Added: Can now specify listener classes
===========================================================================
5.0.1
Fixed: reports generated by SuiteHTMLReporter do not work with JDK1.4
===========================================================================
5.0
Added: Ant task: support for JVM, workingDir, timeout
Added: Stack traces can be interactively shown in the HTML reports
Added: Link to testng.xml in the reports
Added: New structure for reports, suites go in their individual directory
Added: @Test(suiteName) and @Test(testName)
Added: The stack traces in reports do not include TestNG frames (system property testng.exception)
(see: http://groups.google.com/group/testng-dev/browse_thread/thread/9f4d46ade10b0fda)
Fixed: Exit with error when no methods are run
(see: http://groups.google.com/group/testng-dev/browse_thread/thread/3c26e8a5658f22ac)
Added: List of methods in alphabetical order
Fixed: Class-scoped annotations were not recognized when inherited
Added: Deprecated @Configuration and introduced @BeforeSuite/Test/Class/TestMethod
Added: Deprecated @ExpectedExceptions and moved it into @Test
Added: expectedExceptions to @Test, deprecated @ExpectedExceptions
Added: New annotations: @BeforeSuite, @BeforeTest, etc...
Fixed: Was returning an exit code of 0 if a cyclic graph was detected
Added: Interface org.testng.ITest so that tests can declare a name
Fixed: The Text reporter was reporting the square of the actual number of methods
Fixed: Bug reported by Eran about dependencies with an afterClass method
Added: IHookCallBack now receives the ITestResult in its run() method
Added: Name of suite for command line can be set with -Dtestng.suite.name=xxx
Fixed: TestNGAntTask was hardcoding m_haltOnFSP to true
Fixed: Passing a null parameter caused an NPE in the reports
Added: "listener" to the ant task (and documentation)
Added: if patch-testng-sourcedir.properties is found in the classpath
with a property "sourcedir" containing a ; separated list of
directories, this list will override -sourcedir.
===========================================================================
4.7
Added: Maven 2 plug-in
Fixed: Message formattings in TestNG assertion utility class
Fixed: @Factory methods were counted as @Test as well
http://jira.opensymphony.com/browse/TESTNG-51
Fixed: All DataProvider parameters were shown in the HTML report
Fixed: Bug in testng-failed.xml generation
Fixed: <packages> bug when using a jar file to load the test classes
Added: alwaysRun for before @Configuration methods
http://jira.opensymphony.com/browse/TESTNG-35
Fixed: groupless @Configurations were not invoked if a method depends on a group
http://jira.opensymphony.com/browse/TESTNG-45
Added: beforeGroups/afterGroups to @Configuration
Eclipse plugin:
Added: last contextual launch is available in Eclipse launcher lists
Fixed: 3.2M5 integration (removed dependency on non-existing class)
Fixed: testng-failures.xml generation
===========================================================================
4.6
Added: Documentation contains the new reports
Added: TestNG.setUseDefaultListeners(boolean)
Added: Descriptions now appear in TextReporter (verbose>=2) and the HTML reports
Added: description attribute to @Test and @Configuration
Added: combined Reporter output in the reports
Added: methods not run in the reports
Added: org.testng.IReporter
Added: threadPoolSize to @Test
Added: Reports now show relative timings (start at 0)
Added: Reports now show different colors depending on the methods' classes
Added: Reports now show all parameters used to invoke the test method
Added: org.testng.Reporter
Added: DataProviders can accept a Method as first parameter
Fixed: Extraneous implicit inclusion of a method
Eclipse plugin:
Added: Run/Debug as TestNG test from the editor contextual menu
Fixed: TESTNG-24: 'Run as testng test' does not appear of the Test annotation does not have a group
Fixed: TESTNG-18: Eclipse plugin ignores Factory annotation
Fixed: TESTNG-21: Show differences when double clicking assertion exceptions
Added: UI allows setting orientation (even more space)
http://forums.opensymphony.com/thread.jspa?threadID=17225&messageID=33805#33805
===========================================================================
4.5
Core:
Fixed: Methods were not implicitly included, only groups
Fixed: Bug with failed parent @Configuration don't skip child @Configuration/@Test invocations
Fixed: Bug with overridding @Configuration methods (both parent and child were run)
Fixed: Bug when overriding beforeClass methods in base class (cyclic graph)
Added: Support for JAAS (see org.testng.IHookable)
Fixed: Problem with nested classes inside <package name="foo.*"
Fixed: If a group is not found, mark the method as a skip instead of aborting
Fixed: testng-failed.xml was not respecting dependencies
Fixed: class/include method in testng.xml didn't work on default package
Fixed: DTD only allowed one <define>
Fixed: ArrayIndexOutOfBoundsException for jMock
Added: dependsOnMethods can contain methods from another class
Fixed: JUnitConverter required -restore, not any more (option is now a no-op)
Fixed: JUnit mode wasn't invoking setName() on test classes
Added: Regular expressions for classes in <package>
Added: Distributed TestNG
Fixed: Command line parameters and testng.xml are now cumulative
Fixed: Reports now work for multiple suites
Fixed: Was ignoring abstract classes even if they have non-abstract instances
Fixed: If setUp() failed, methods were not skipped
Fixed: Was not clearly indicating when beforeSuite fails
Added: @Configuration.inheritGroups
Fixed: inconsistency between testng.xml and objects regarding method selectors
Eclipse plug-in:
New look for the progress view.
===========================================================================
4.4
Core:
Fixed: testng-failures.xml was not excluding methods from base classes
Fixed: Bug in suites of suites for JUnit mode
===========================================================================
4.3
Core:
Fixed: testng-failures.xml was not excluding methods from base classes
Fixed: Bug in suites of suites for JUnit mode
Added: Excluded groups on command line and ant task
Fixed: When including a group, implicitly include groups depended upon
Fixed: When depending on several groups, wasn't skipped if one of them failed
Fixed: Failures weren't reported accurately in the JUnitReports report
Fixed: Wasn't throwing an exception if depending on a non-existing group
===========================================================================
4.2
Core:
Fixed: wasn't excluding methods in base classes
Added: alwaysRun for tests (soft dependencies)
Fixed: Class-level enabled=false were not honored
Fixed: Bug with multiple dataproviders on same class
Fixed: Bug with dataprovider defined in the parent class
Fixed: Bug with dataprovider defined in a subclass
Fixed: Bug with dataprovider defined in an abstract class
Fixed: testng-failures generation was excluding the methods even if a failed test depended on it
===========================================================================
4.1
Core:
Added: @DataProviders can return Iterable<Object[]>
Fixed: Superclass test methods were not called in the presence of a class @Test
Added: Reporter class to log messages in the HTML reports
===========================================================================
4.0
Core:
Fixed: suite methods now invoked only once in a hierarchy
Added: @DataProvider and @testng.data-provider
Fixed: Interleave order now respected for before/afterClass methods
Added: Can now invoke java -jar testng-2.6.jar <...>
Added: Support for BeanShell
Added: Method Selectors (IMethodSelector)
Fixed: In the absence of dependencies, @Configuration methods respect inheritance
Fixed: Bug in multithreaded dependencies on methods
Fixed: dependsOnGroups wasn't working on regular expressions
Fixed: Bug in <package> when directories contain spaces in their names
Fixed: Introduced a JDK5 dependency in the JDK1.4 build (getEnclosingClass())
Fixed: Output directory in ant task was not honored if it didn't exist
Fixed: Problem with timeout according to
http://forums.opensymphony.com/thread.jspa?threadID=6707
Eclipse plug-in:
Fixed: Wasn't handling linked directories correctly
Fixed: Bug in QuickFix implementation
Added: Quick Fix for JUnit conversion (Annotations and JavaDoc)
Fixed: Methods Run as TestNG test
Added: Package level Run as TestNG test
Fixed: Resources from the linked directories are using a wrong path when
passed to command line TestNG
IDEA plug-in:
Added: Support for JDK 1.4 (both projects and IDEA itself)
Fixed: Classes that contained only configuration were ignored
===========================================================================
2.5
Added: ITestListener.onTestStart(ITestResult)
Added: Support for <packages>
Added: Resource files for easier ant taskdefs
Fixed: @Configuration methods were not invoked with individual test methods
Fixed: Bug with ExpectedExceptions
Fixed: Didn't support nested factory classes
Fixed: NPE if -target is omitted with JDK 1.4
Fixed: @Configuration failures in a class would cause other classes to fail
Added: alwaysRun
Fixed: beforeTestClass/afterTestClass were broken for a pathological case
Added: @Configuration(alwaysRun)
Added: JUnitConverter task
Fixed: < and > characters in reports were not escaped
Eclipse plug-in:
Fixed: Class dialog wasn't showing @Factory classes
IDEA plug-in:
First release!
Documentation:
Added: Brand new look!!!
Added: Section on testng.xml
Fixed: Numbering of sections
===========================================================================
2.4
Changed: New package: testng.org
Fixed: Bug with @ExpectedException occuring the parallel mode
Fixed: Bug with parameters and beforeTest
Added: IInstanceInfo support
Fixed: methods were not excluded when included by groups
Fixed: testng-failures.xml is now including also the beforeSuite/afterSuite methods
Fixed: generating the testng-failures.xml is now working as expected
Fixed: Factories call all the tests even if some of them fail along the way
Fixed: Better JUnit support (wasn't creating individual instances)
Fixed: dependsOnGroups didn't work across different classes
Added: command line (and Ant) -groups option
Added: @Parameters (and made parameters attribute deprecated)
Added: Parameters for constructors
Fixed: Better interleaving of before/afterTestMethods
Fixed: Ant task
Fixed: TestNGException thrown when TestNG conditions are not fulfilled
Documentation:
- New assert classes
- New ways to launch
- JUnitConverter documentation
- new beforeSuite/afterSuite
===========================================================================
2.3
Fixed: Spaces are now legal in JavaDoc comments
Added: documentation for @Factory
Fixed: factories were called multiple times
Added: beforeSuite and afterSuite
Fixed: inheritance and scope now working properly for annotations
Fixed: dependsOnMethods wasn't working for 1.4
Added: Better stack traces
Added: Better syntax for included/excluded methods
Fixed: Better verbose support
Fixed: Various fixes for the Eclipse plug-in
Added: Can specify a class name on the command line
Fixed: Default package bug in JUnitConverter
Added: Regression tests for JUnitConverter
Added: -quiet option to JUnitConverter
===========================================================================
2.2
Fixed: Wasn't handling several testng.xml files correctly
Fixed: Renamed -src to -sourcedir
Fixed: Complains if no sourcedir is specified in 1.4
Added: In 1.4, don't require annotations="javadoc"
Fixed: If setUp fails, complain and mark test methods as skips
Fixed: Dependent methods weren't working for 1.4
===========================================================================
2.1
Added: Parser can accept an InputStream for testng.xml
Fixed: expected-exceptions now fails if test passes
Fixed: reports now use the suite name in HTML
Added: invocationCount and successPercentage
Added: dependsOnMethods
Added: timeOut works in non-parallel mode
===========================================================================
2.0
Added: port on JDK 1.4
===========================================================================
1.3
Added: new view: classes (still experimental)
Added: timeout on methods
Added: thread-count
Added: TestNG is now multithread, see "parallel" in <suite>
===========================================================================
1.2
Added: JUnitConverter
Fixed: Bug with afterClasses (test: AfterClassCalledAtTheEnd)
===========================================================================
1.1
Added: new links for methods and groups in the HTML report
Added: <methods>
Added: <fileset> to <testng>
===========================================================================
1.0
Fixed: Updated to the new DTD
Fixed: Suite table of contents displays failures first
Fixed: Bug in afterTestClass
Added: Validating testng.xml
Added: Scoped parameters
Added: testng.xml
Removed: Property quiet
Changed: Verbose is now an integer
Added: Dependent methods
===========================================================================
0.9
Added: Groups of groups
Added: Groups for Configuration methods
Added: Parameters
===========================================================================
0.2
Fixed: Merged TestMethod and TestClass into Test
Added: HTML report
Added: Regexps for groups
Fixed: Inheritance of methods
Fixed: ExpectedException is now called ExpectedExceptions