-
New
badargs.resultfor formatting function result checking failure messages. -
The
expectmacro now behaves like a standard Lua function, and will accept bare tables or strings (i.e. without surrounding parentheses) as a single argument:expect {1 + 1}.to_equal {2}
-
New
be.gt,be.gte,be.ltandbe.ltecomparison matchers, that will use the Lua__ltand__ltemetamethods to compare complex objects when available. See LDocs for more details. -
New
be.betweenmatcher, also__ltand__lteaware:expect (#s).to_be.between (8, 20)
-
New decorators for
be.betweento specify inclusive or exclusive ranges:expect (2).to_be.between (1, 3).exclusive
-
New
be.withinmatcher adaptor and associatedofdecorator for alternate spelling ofbe.between:expect (10).to_be.within (2.5).of (12)
-
loadandloadfilein example code will honor theenvargument now, in any supported host Lua implementation.modeis still unsupported, however. -
package.searchpathis available in examples, even on Lua 5.1 hosts. -
Much simplified example environment setup resulting in 30% speedup with typical spec-files.
-
Preliminary LDocs for Specl APIs useful for writing Matchers, expectations and spec_helper.lua.
-
badargs.diagnosenow requires arguments in the same order asstd.debug.argscheck. -
The declaration string format for
badargs.diagnoseis now a subset of the format used bystd.debug.argscheck. Particularly, note the use of square brackets for omittable arguments, and ellipsis for type of additional arguments rather than*. See LDocs for more details. -
loadstringis not supported in example code, useloadinstead. -
package.loadersis not supported in example code, even where that is the field used by the host Lua implementation; usepackage.searchersinstead. -
package.seeallis not supported in example code.
-
loadandloadfilepropagate error messages back to calling examples. -
The
getfenvandsetfenvfunctions available from examples will no longer pass non-function objects to the host Lua implementationdebug.getfenvanddebug.setfenvsymbols. -
First word of descriptions is stripped correctly in some corner cases that were missed previously.
- Specl is now released under the terms of the MIT license, the same
license that Lua uses. See
LICENSE.mdfor details.
- preliminary compatibility with Lua 5.4
- removed more than 22000 lines of autotools boilerplate from the distribution (leaving fewer than 11000 lines of actual specl files).
-
generate a working test coverage report with luacov when the --coverage option is given (assuming "luacov" is loadable).
-
runs specs in around half the time of previous releases.
- new --coverage option tries to use "luacov" for coverage reports, but actually it doesn't work.
- compatibility with recent split between lua-stdlib and optparse.
- internal
deepcopyno longer overflows the stack when copying tables that are their own metatable across nested Specl environments.
equalmatcher really does compare mutable keys correctly, which works on LuaJIT and Lua 5.3 too.
-
equalmatcher compares mutable table keys correctly, so things like this now work:expect ({[{k="mutable"}]=true}).to_equal ({[{k="mutable"}]=true})
-
badargs.diagnosealso accepts '?' in lieu of a proper function name in argument errors (as often returned by LuaJIT). -
badargs.formatwill produce abad fielddiagnostic when called as:badargs.format ("function_name", narg, nil, "field_name")
-
New matcher
raise_matchingcompares an error against a Lua pattern.
- Preliminary Lua 5.3 support.
-
badargs.formatnow formats message with the word "arguments" for all ordinal values except exactly 1, where "argument" is used. Previously, Specl erroneously used "argument" for ordinal values of less than or equal to 1. -
Ancient code to replace underscores in example description with spaces has finally removed.
-
specl.inprocessnow propagates function environments correctly, so examples inside theinprocess.callare not tallied to the main specl counters;formatter.reportdoes not lose track of what descriptions have been displayed after aninprocess.call; etc. -
When expecting automatic execution of specfiles in ./specs/*, without luaposix installed, error is diagnosed properly.
-
Specl's own modules, and modules loaded from spec-file directories are now loaded inside the function environment of the running example, and so have access to Specl's Lua extensions, such as
expectand matchers. -
inprocess.capturereturns aProcessobject, suitable for matching using thespecl.shellmatchers. -
New
to_raisematcher as an alias forto_errorto avoid flagging capitalised error messages in error matcher argument with Slingshot sanity checks. -
New programmatic
examplesfunction for building example groups and helper functions directly in Lua, whenever YAML starts to get in the way. See documentation for examples. -
New
specl.badargsmodule providesformatto generate standard argument error messages that can be passed to matchers to ensure correct error message formatting:expect (posix.write ()).to_raise (format ("write", 1, "int"))
And
diagnose, which writes a full set of expectations to check all missing, extraneous and badly typed arguments are diagnosed correctly:- context with bad arguments: badargs.diagnose (posix.write "(int, string)")
-
inprocess.capturereturns aProcess, not a pair of strings and the call results. Call results are still available in the array part of the returnedProcess. -
Specl now requires latest stdlib v40 to be installed on the module path, rather than carrying its own copies of parts of stdlib it wants to use.
-
Specl now installs as a selection of library files (in the module subdirectory
specl), and a thin wrapper script that sets up the execution environment and then callsspecl.main (arg):execute (). This simplifies debugging, and remove all the infrastructure for expandingfrommacros, and concatenating the resulting sources, as well as allowing piece-meal loading of specl modules.
-
Enhanced resolution timers now also work with luaposix > 31.
-
The report formatter would not display subsequent repeats of a context description after the first under some circumstances. It now displays context descriptions correctly in that case too.
-
Don't prevent examples from loading a different luaposix than the installed default loaded by Specl. This allows the luaposix specs to work in the build tree without priming LUA_CPATH and LUA_PATH before calling specl, for example.
-
inprocess.capturenow supports capturing results of a functable call. -
specl.shellmatchers now correctly succeed when usingnot_prefixes to specify that aProcessshould not contain some error or standard output text, and thatProcessstream containsnilto indicate the stream was not written at all. For example:expect (capture (function () print "stdout" end)). not_to_contain_error "stderr"
-
After loading system modules it depends on, Specl removes them from the Lua module cache so that other versions of those same modules (yaml, posix, posix.sys) can use Specl to check their own behaviours by loading the to-be-checked versions from
spec_helper.lua.
-
Now that this API is stable, there is full documentation for
specl.shellin doc/specl.md. -
New
specl.inprocess.capturefunction for running examples that interact with io streams.inprocess.capturewraps a function call so that standard input calls read from the supplied buffer, and any output is captured rather than leaked:inprocess.capture (function (...) print (...) io.stderr:write (io.stdin:read "*a") return "first", 2 end, {"foo", "bar"}, "input content") --> "foo\tbar\n", "input content", "first", 2
i.e. standard output, standard error, function return values
-
The report formatter no longer displays orphaned contexts of filtered-out examples.
-
Specl no longer crashes when requiring system libraries (such as math, bit32 etc) from examples or spec_helpers.
-
The
--fail-fastoption no longer stops when encountering a failing expectation in a pending example. -
Ignore any LUA_INIT or LUA_INIT_5_2 settings in the execution environment, since they can easily corrupt the behaviour of example code. If you need to write examples that deliberately interact with those envvars, use
specl.shellto isolate them correctly.
-
Built-in matchers compare nested tables and objects in about 25% of the time required previously, and is optimised for tail-call elimination for non-tree like structures.
-
Specl now loads a
spec_helper.luafrom the same directory as the spec file being executed into the example environment automatically, so there's no need to manually require it from abefore:block in each spec file any more. -
You can easily run Specl from the top of your source tree without any special LUA_PATH munging in Makefiles or wrapper scripts. Just set
package.pathandpackage.cpathto find the code being specced out in aspec_helper.luain each directory containing spec files. -
When invoked with no filename arguments,
speclwill check all your specifications, as long as they have names ending in_spec.yamland are kept in or somewhere below thespecs/directory of your project. -
The report and progress formatters now display file:line:expectation: annotations for failed and pending expectations in verbose mode (-v).
-
New command line option
--fail-fast(or-1) to stop checking expectations immediately on the first failure, rather than continuing and showing all failures. -
New command line option
--example=PATTERN(or-e PATTERN) to specify inclusive filters matched against the fully concatenated YAML descriptions of an example. Use this to select a subset of examples to (re)check. -
You can also filter out all but the examples at a specified line number either by pasting from the output of a failed or pending expectation in verbose mode, by invoking specl as:
specl specs/some_spec.yaml:44:1
or with as many
+NNas necessary to select several examples from the following file:specl +44 +48 specs/some_spec.yaml
or a combination of the above. Note that the final
:1in the former is ignored and can be omitted (it is the ordinal number of anexpectstatement in a given example), but is accepted for ease of cutting and pasting. -
From this release out, matchers using
to_are preferred to usingshould_. When negating a new syntaxto_matcher, both ofto_not_andnot_toare accepted. -
New
copymatcher, to specify that an object should be an identical copy of the expected value, but not the same object:t = {"foo"} expect (table.clone (t)).to_copy (t)
-
You can
require "specl.shell"for a swathe of additional matchers to specify results from aProcessobject:a) For exit statuses, use:
```lua expect (shell.spawn "exit 42").to_exit (42) expect (shell.spawn "exit 0").to_succeed () expect (shell.spawn "/bin/false").to_fail () ```b) For checking the contents of the output stream, use:
```lua expect (...).to_output "exact content" expect (...).to_contain_output "substring" expect (...).to_match_output "pattern" ```d) To require zero exit status with specified output stream contents:
```lua expect (...).to_succeed_with "exact content" expect (...).to_succeed_while_containing "substring" expect (...).to_succeed_while_matching "pattern" ```c) To check the standard error stream instead:
```lua expect (...).to_output_error "exact content" expect (...).to_contain_error "substring" expect (...).to_match_error "pattern" ```e) To require non-zero exit status with specified error stream contents:
```lua expect (...).to_fail_with "exact content" expect (...).to_fail_while_containing "substring" expect (...).to_fail_while_matching "pattern" ``` -
New experimental
specl.inprocessmodule provides a way to run carefully written Lua programs inside the Specl process, saving a fork/exec each time. Converting Specl itself to be reentrant, and usingspecl.inprocessfor most of its examples shaves more than 75% off the time spent inmake check: from over 20secs to under 5secs on my machine.
-
Invoking
speclwith no filename arguments no longer reads specs from standard input. You must explicitly pass a "-" argument to get that behaviour. -
RSpec has been moving from expressing
should_expectations to usingto_for technical reasons. Specl now usesto_throughout, not for technical reasons, but because:(i) expectations almost always read better with
to_instead ofshould_; (ii) it's shorter and easier to type, so is less "noisy" in among the important surrounding arguments.There are no plans to remove
should_from Specl, so you can expect to safely continue to use your existing spec files indefinitely: But all mention ofshould_has been removed from documentation, and I recommend you useto_for new spec files for the reasons above. -
When making custom matchers, all the api constructor functions are now proper methods. Previous releases forgot the initial self parameter in the unnamed matching predicate function and the following optional functions accepted by the constructor:
function (self, actual, expect) function format_actual (self, actual, ...) format_expect (self, expect, ...) format_alternatives (self, adaptor, alternatives, ...)
If you have added any custom matchers to your spec files, be sure to insert the
selfparameter to match the new calling convention as soon as you upgrade, or they will not work with this release.
-
Setting table elements from outer environment tables no longer leak into sibling examples. For example, changing entries in the _G table or adding entries to the string table affects only the current innermost environment now, just like setting other variables.
-
Unrecognised command-line options are diagnosed properly.
- Using specl.shell when Specl itself and the process being spawned require different versions of Lua no longer crashes with corrupted shared package.path settings.
-
Vastly improved error diagnostics for syntax errors in spec files, reporting filename and line-number of error locations, and without spurious Lua-runtime stack traces.
-
Support for custom per-matcher adaptors. See docs/specl.md for documentation.
-
New
a_permutation_ofadaptor for contain matcher, that allows expectations for operations that are inherently unordered:t = {} for _, v in pairs (a_big_table) do t[#t + 1] = v end expect (a_big_table).should_contain.a_permutation_of (t)
It will fail unless the same set of elements are present in the expect argument values, or the expect argument keys; like the other
containadaptors it works with strings and objects too. -
New
--unicodecommand-line option to support unicode in spec files, at the expense of inaccurate line-numbers in error messages. -
Works correctly without ansicolors installed (albeit without displaying any color!).
-
If
luaposixis installed where Specl can find it, the report and progress formatters will use it to display the time spent running specifications much more accurately.
-
Requires lyaml 4 or newer.
-
Unicode characters in spec files are no longer supported by default. LibYAML counts indices by character when reporting the offsets used to re-extract the Lua code fragments (i.e. without YAML neline folding) that Specl uses to generate correct line- numbers in error-messages. Lua string operations require byte offsets, which are incompatible.
-
When using std.objects, the contain matcher now displays the expected object type name (rather than just: "table") from the FAIL report.
- Help2man is no longer required to install specl from the distribution tarball.
-
Can now be installed directly from a release tarball by
luarocks. No need to run./configureormake, unless you want to install to a custom location, or do not use LuaRocks. -
A new
all_ofadaptor for any matcher:expect (mytable).should_contain.all_of {x, y, z}
-
containmatcher handles std.object module derived objects by coercing them to tables. -
equalmatcher performs a deep comparison of std.object module derived objects' contents. -
./bootstrapruns slowly again with 4800 lines of code reintroduced required to automatically manage Slingshot files with newbootstrap.slingshot. This doesn't have any direct effect on users, except that the Slingshot managed release process is much more robust, reducing the chances you'll end up with a still-born release.
-
Avoid
nan%divide-by-zero error output from report formatter. -
Built in report formatter displays argument strings to pending calls from examples consistently.
-
API change in matchers, generalizing the former
format_any_oftoformat_alternatives, with a newadaptorparameter for use with additional matcher adaptors. -
API change in both matchers.concat and matchers.reformat, each of which now take an additional
adaptorargument, which can be passed in fromformat_alternativesas received so that the grammatically best choice ofandororis used to display the list of alternatives when an expectation fails. -
Calling
requirein a spec file no longer artificially extracts symbols from a returned table and injects them into the local environment -- because it was inelegant and unnecessary. If you had relied on this feature, simply capture the returned table and either manually copy the symbols you need or use a fully qualified name. Previously:# Don't do this! before: require "specl.shell" describe spawn: expect (spawn "printf hello").should_output "hello"
Much better:
# Do this instead. before: spawn = (require "specl.shell").spawn describe spawn: expect (spawn "printf hello").should_output "hello"
-
Add missing documentation for
matchers.Matcher.format_any_of. -
It's no longer possible to crash Specl using example descriptions that look like other types during lyaml parsing.
-
Specl 6 had a broken
should_containmatcher, that would fail to match any expected substring with an active Lua pattern character in it (such as ?, * or % for example). -
Add missing close parenthesis in non-verbose mode report formatter summaries with failing or pending expectations.
- While not encouraged, one word descriptions are now supported, and are displayed correctly by the bundled formatters.
- This release is a significant upgrade, with many new features, and, no doubt, some new bugs.
-
Top level
beforeandafterfunctions are supported. -
A proper, documented, API for adding custom matchers.
-
A new
any_ofmethod for any matcher:expect (ctermid ()).should_match.any_of {"/.*tty.*", "/.*pts.*"}
-
You can
require "specl.shell"from a spec file (usually in the initial top-levelbefore) to get access to a shell Command constructor, aspawnexecutor function and several new matchers for querying the status of a shell command. -
package.pathis augmented, for the duration of each spec file, so thatrequirecan find and load other Lua files in the same directory. -
Additional YAML "documents" from a spec file with
---and...stream separators are no longer ignored; but treated as additional unnamed documents. -
Report formatter displays inline expectation summaries for each example when not in verbose mode.
-
Documentation is now in Jekyll format markdown for easy website regeneration.
-
Calling require in a spec file now runs in the local environment, giving access to
globalsymbols from the newly loaded file from the local namespace. Conversely, access to those same symbols is no longer available from "_G", the global environment table.
- Documentation reorganisation. README.md is much simplified, with full documentation still in markdown at docs/specl.md. The html documentation at http://gvvaughan.github.io/specl will be updated with every release from now on.
-
./bootstrapruns quickly with 4800 lines of code removed. -
./configureruns quickly with the remaining C macros removed. -
progressandreportformatters now report elapsed time rather than cpu time in their footer output. -
The
speclLUA_PATH no longer picks up its ownspecl.stdmodule by mistake when a spec file requires the lua-stdlibstdmodule.
-
The
should_errormatcher now takes arguments in the same order as the other matchers. LuaMacro injects apcallinto everyexpectinvocation, so the specifications are written intuitively:expect (error "failed").should_error "failed"
-
The Specl-1 Lua format for spec files is no longer supported, in part to clean up a lot of otherwise unused code, but also because Lua specs were accumulating too much magic to be easy to write by hand.
-
build-auxspeclc and the bundled generatedspecs/*_spec.luaspecs have been removed.
- This release is a minor update.
-
Now tested against Lua 5.1, Lua 5.2 and luajit-2.0 on every commit, thanks to travis-ci.org.
-
Pending specifications are now fully implemented and documented.
-
Unexpected passing of pending specifications is reported by progress and report formatters.
-
API for custom formatters is richer and clearer.
- Specs propagate user LUA_PATH settings to specl forks in Specls own own specifications.
- This release is a significant upgrade.
-
lyaml was spun out to a separate luarock, now required.
-
Initial support for pending examples, either using the new
pending ()function, or having an example description with an empty definition. -
pending and failed expectations are now summarized in the footer of default (progress) and report formatters.
-
Formatters display in color on supported TERM types, ansicolors is now required.
-
Color can be disabled with
--color=nocommand line option. -
Custom formatters are now supported, using the new command line option
--formatter=tap. -
The custom formatters API is documented in README.md.
-
A new TAP formatter was contributed by François Perrad.
-
Many more specifications for Specl were added, now that specl is featureful enough to support BDD development of itself.
-
Error message from invalid Lua in example definitions are now reported correctly.
-
Runner environments are more robust, see README.md for details.
-
Specl no longer uses lua-stdlib (to break a cyclic dependency when using specl to run lua-stdlib spec-files).
-vnow behaves differently, and simply requests more verbose output from the selected formatter, use-freportto select the report formatter like-vdid in release 2 and earlier.
-
Now compatible with Lua 5.2 and Lua 5.1.
-
Primary format for spec files is now YAML (specl-1 format spec files are still supported).
-
Requires libyaml-0.1.4 to be installed before building.
-
Includes some YAML specifications for Specl.
-
Initial proof-of concept for an RSpec inspired framework for and in Lua.
-
The spec file syntax is a bit horrid in pure Lua, but the next release uses YAML and is much nicer!