-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
generation of functions #8
Comments
cc @jmid might be of interest to you, too |
Sorry for being a bit late with input. I looked a bit into this topic a year ago. Koen's talk is available here: https://www.youtube.com/watch?v=CH8UQJiv9Q4 A second one is a talk by Ulf Norell on 'Generating Random Functions'. |
There is also the approach used by JST: https://blogs.janestreet.com/quickcheck-for-core/ (based, apparently, on building finite decision trees on the parameters). It looks quite nice to me. |
need some tests now. |
Hey @jmid I know your course has probably ended a while ago, but would you per chance have a bit of time to review this? I'd be glad to have your opinion on the API for functions… |
Very cool contribution - thanks for this! I'm hard pressed to make the coming SAS paper deadline (8 days from now). I'll be happy to take a closer look after that. If you are still looking for example tests Ulf Norell has a cool and cute example of inferring game strategies (as functions) for the game of 'Nim' in the above linked video. |
Ok, I've given the function generators a spin. I've uploaded the outcome here: If you have a look at the outcome you will probably agree with these observations:
This last one is a broader question: can QCheck safely invoke itself recursively? |
Ah, interesting, thank you! However, look at 2085ae5 (not released yet) which addresses the current issue. Hopefully the benchmarks you list can be improved by using the new API. Do you think it's possible? The combinators such as Note: what's |
Also, I'm not sure what qcheck calling itself would involve, sounds dangerous… Maybe a special combinator like |
Actually, could you open another issue for recursive invocation? It seems what you want is actually qcheck-based synthesis (which can fail in its own ways), so it's clearly non trivial and deserves its own issue. |
I need to take a look at the Claessen's paper you linked. The API I added (inspired from JST's quickcheck) is quite sophisticated to use, so maybe something still based on a finite mapping |
Ah, sorry for messing up: I mistakingly thought I was already using your latest additions.
|
Ok, ping me when you have an update! :-) |
Hm. I checked out and built the latest branch. I can easily port the first two examples, by prefixing with
If I instead define |
Does it shring properly now? I'm not totally satisfied with the API, it's more cumbersome to use; of course it's hard to provide |
Before opening a new issue on recursive invocation, I would like to understand the API that Norell is using for his second Nim example. As I understand it, he defines a property-based test All of these are defined approximately 15min into https://vimeo.com/143848099 I cannot find any API info on @jlouis can you enlighten us a bit on what's going on? |
@c-cube The test of this false property succeeds with |
Oh, annoying :/ |
If I lower the test count to 2(!) with
24008238 shrink steps! No wonder it took over a minute :-o Are there any low hanging fruits for improving the shrinking approach? This situation echoes another experience of mine. For a program generator I wrote, I ended up instrumenting it to print out Finally, I still don't understand why the version with |
Claessen's work was the original stuff for this indeed. Ulf's work works in a dynamically typed language, but perhaps the Haskell approach is closer to OCaml's solution. It depends a bit on how you can get at the types I think. As for the example: we are playing Nim. The
finally, Indeed, Ulf is just using the QuickCheck system inside the QuickCheck system. It isn't too different from having an inner induction proof running while processing an outer induction proof. I'm not sure you need to "tie the knot" on the two things at the type level since the inner use is separable from the outer (I may be wrong however, have not given it much thought and the addition of types might prove problematic as the type becomes recursive, and thus requires some iso-recursive handling). |
Ok, so I think I get the gist of Claessen's paper. I'm not very happy with all the ideas in it, but nevertheless, here is a new design draft for (simpler) generation of functions: The main idea is to define a function Practically, for the user, it should look like this: module Observable : sig
type 'a t (* basically, a printer + eq + hash *)
val int : int t
val bool : bool t
val make : ?eq:'a eq -> ?hash:'a hash -> 'a Print.t -> 'a t
end
(* for direct extraction of the function by pattern matching *)
type ('a, 'b) fun_ = Fun : ('a, 'b) secret_fun_state * ('a -> 'b) -> ('a, 'b) fun_
val apply : ('a, 'b) fun_ -> 'a -> 'b
val fun1 : 'a Observable.t -> 'b arbitrary -> ('a, 'b) fun_ arbitrary |
@jmid that simple property taking 24M steps of shrinking hints that the current approach is bad. We definitely need something that looks at the input it is given instead of guessing. |
What I think is happening, statistically, is that we are essentially
invoking the law of large numbers (LNN), or in french: Loi des grands
nombres. As the number of test cases grow large, we tend to hit every
possible case. And computers are getting really good at generating insane
amounts of numbers quickly. This somewhat challenges our minds as we would
tend to expect you need exhaustive testing in order to be sure you have hit
everything. But in practice, since you are often rerunning yuur test cases
over and over, exhaustion is of less importance as every possible path is
likely to be run at some point.
The obvious counterexample is something like
match sum [x; y; z; a; b; c] with
| 37 -> true
| _ -> false
where x,y,c,a,b,c are integers. It is not very likely to be hit in
randomized methods. You need either smallcheck or concolic testing (where
you say "dear SMT solver, you can't find a situation where x + y + z + a +
b + c = 37!" and it then comes up with a counterexample to that claim).
…On Tue, May 9, 2017 at 4:50 PM Simon Cruanes ***@***.***> wrote:
@jmid <https://github.com/jmid> that simple property taking 24M steps of
shrinking hints that the current approach is bad. We definitely need
something that looks at the input it is given instead of guessing.
@jlouis <https://github.com/jlouis> thanks for the explanations. This way
of using quickcheck is quite impressive, and I would rather have expected
it in a (lazy) smallcheck setting. I'll think about a val find_example :
?count:int -> ('a -> bool) -> 'a arbitrary -> 'a option to be used inside
the properties.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#8 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAAWH-9nqLiYsoWztDK0HrwQQfZR_1loks5r4H0zgaJpZM4LVcrP>
.
|
Indeed, qcheck has been very helpful for me so far. For counterexamples that are hard to find, I wrote https://github.com/c-cube/smbc/ last summer (for pure logic in proof assistants, though, it's not ready for OCaml at all). |
@c-cube Sounds good, this also agrees with my understanding of Claessen's work. |
I still don't really get the point of the translations… |
@c-cube Ok, this works nicely, thanks! I've just pushed the changes needed to the examples here: https://github.com/jmid/qcheck-fun A few observations:
Looking at the code, it seems to me that |
Thanks for the feedback.
If you have small test cases that exercise the library (such as the fold_left/fold_right one?), please consider contributing them in a PR! It's always nice to test the testing libraries ;-) |
Ok, thanks. For large tables (and for "nested"/2-argument functions) like You are more than welcome to take the tests and incorporate them in QCheck's own testsuite. That will be more effective than me figuring out where negative tests fit in and fighting git to make it happen... ;-) |
Maybe I can add https://github.com/jmid/qcheck-fun as a submodule… The list shrinker would do the same, I believe. But give me a second, I'll try another strategy! |
Perhaps I should try to build a uncurried table, actually — that might work with GADTs. This way, the shrinking is done by removing one tuple, or by shrinking an entry. |
Oh, I actually looked at QCheck's list shrinker before, and thought it did something more advanced, but it doesn't seem to:
In comparison, e.g., the Haskell shrinker (defined as |
Please open an issue for the better |
I've often used a free construction: A list is a binary tree which is
appended together. It shrinks to a subtree, either the left or right one.
By tracking where we glued together the list in the tree, we essentially
have a free construction. But for a generic variant, I'd definitely go with
something simpler. You can also go with a dropping variant which
successively tries to drop more and more aggressively: if we can drop one
element, lets try dropping 2,3,5,8,13,... or something such. The goal is to
quickly remove noise, but a few initial probe attempts are not going to be
a problem I think: Erlang is bytecode-interpreted and slow. But shrinking
is still plenty fast. You have at least one order of magnitude of
efficiency in a naive OCaml implementation.
…On Wed, May 10, 2017 at 1:17 PM Simon Cruanes ***@***.***> wrote:
Please open an issue for the better Shrink.list :-)
I wrote the simplest version I could think of, but I suppose something
based on divide-and-conquer (removing half the elements, or a quarter,
etc.) would be faster in many cases…
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#8 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAAWH3msi5ASrmSAXMi_4Q7j4FjP4k38ks5r4Zy2gaJpZM4LVcrP>
.
|
Thanks @jlouis . I will probably imitate the code from Haskell's quickcheck but using arrays (mapping from/to lists). |
@jmid other remark: perhaps using uncurried functions (with the |
It seems that the uncurried version of |
Yes, I believe I wrote both versions (curried and uncurried) in the example tests (following Claessen) |
My bad, I didn't see it. Anyway it shrinks faster… |
@jmid I have updated functions again, it seems to shrink faster in some cases (but |
I tried rerunning the function tests with the newest version. My impression is the same as yours: sometimes the Regarding the API, I was trying to wrap my head around the |
I wonder if the example is slow because it uses the About tuples, yeah, the base case should be a pair, not an empty tuple. I'll fix it and come back to you. edit: actually I remember, the current construction with unit is nice because GADTs are not ambiguous. It's a classic construction for heterogeneous lists (see the code of |
Also note that the user should generally use |
@jmid if you don't have objections, I am going to consider this issue as solved soon :-) |
You are welcome to close it. I just reran my tests with functions put last in the tuple. There were still occasional long runs but it generally seems faster than I've experienced previously. In these situations (with functions or long lists of large integers) it would be welcome with some feedback to an interactive user, e.g., just printing The advise of putting functions last in a tuple (shrunk in lexicographic order, left-to-right) may be valuable to other users, so it would be great to put it in the documentation. |
CHANGES: ## 0.9 - add `qcheck-ounit` sublibrary - use environment variables to configure `qcheck-alcotest` tests - alcotest backend for qcheck - make `qcheck.ounit` tests verbose by default - make `qcheck` is a compatibility library, depends on `qcheck-core` - split lib into `qcheck` and `qcheck.ounit` - add `TestResult.is_success` helper - give access to list of instances in test results - allow setting `time_between_msg` in runner - chore: remove submodule - chore: add travis support - doc: explanations about qcheck.ounit runners - doc: update readme ## 0.8 - migrate to jbuilder - fix warnings - add some useful functions - update oasis files (close c-cube/qcheck#48) - update copyright header (closes c-cube/qcheck#47) ## 0.7 - switch to BSD license, make it more explicit (close c-cube/qcheck#43) - improve multi-line message printing in ounit (closes c-cube/qcheck#46) - fix complexity of `add_stat` - allow negative entries in statistics (see c-cube/qcheck#40) - add a way for tests to report messages to the user (see c-cube/qcheck#39) - add `QCheck.Shrink.int_aggressive` and make default int shrinker faster - shrinker for `map_keep_input` - add `QCheck.set_gen`, missing for some reason - more compact verbose output (see c-cube/qcheck#33) - better handling of dynamic progress line - Add colors to checkmarks in verbose mode - improve statistics display for runner - recover exception of shrunk input - print status line before the solving starts ## 0.6 - add `find_example` and `find_example_gen` to synthesize values from properties (see c-cube/qcheck#31) - add `QCheck.gen` for accessing the random generator easily - colorful runners, with `--no-colors` to disable them - add more generator (for corner cases) - better generation of random functions (see c-cube/qcheck#8), using `Observable` and an efficient internal representation using heterogeneous tuples, printing, and shrinking. deprecate old hacks. - add statistics gathering and display (see c-cube/qcheck#30) - better printing of Tuple - improve `Shrink.{array,list}` (see c-cube/qcheck#32) - Change asserts to raise `Invalid_arg` (following the doc), and update doc - Change `Gen.{int_bount,int_range}` to support up to 2^62 ## 0.5.3.1 - fix regression in runner output (print results of `collect`) - update the `@since` tags ## 0.5.3 - missing char in `Gen.char` (close c-cube/qcheck#23) - add `test` and `doc` to opam - add `small_list` generator - add `~long_factor` to tests and runner, for long tests - add more examples in readme, better doc for runners - improved reporting when running qcheck tests - add `Test.get_count` on test cells ## 0.5.2 - Add cli option for backtraces in `QCheck_runner` - Add test case for raising exception - Better handling of backtraces - All tests now have a name - Add step function called on each instance in a test - make `small_int` a deprecated alias to `small_nat` - add `small_signed_int` - remove some warnings - use safe-string, and fix related bug - Add long tests options to `QCheck_runner` - Add `length` specification for `to_ounit2_test` - Added paragraph in README about long tests ## 0.5.1 - document exceptions - add `small_nat`, change `small_int` semantics (close c-cube/qcheck#10) - add `QCheck.assume_fail` - add `QCheck.assume`; explain preconditions a bit (close c-cube/qcheck#9) - Polish documentation - Added quad support uniformly ## 0.5 - merge back from `qtest`: big changes in API, shrinking, use `'a arbitrary` type that combines printer, generator, shrinker, etc. (see git log) - merlin file - reorganize sources, `_oasis`, `.merlin`, etc. ## 0.4 - bugfix in `fix_fuel` - if verbose enabled, print each test case - add `QCheck.run_main` - `QCheck_ounit.~::` - add `(>:::)` - add `qcheck_ounit ml{lib,dylib}` - trivial ounit integration - make `test_cell.name` optional - `Arbitrary.fix_fuel(_gen)`: add a recursive case - `Arbitrary.fix_fuel_gen`, similar to `fix_fuel` but threading a state bottom-up to make choices depend on the path - `Arbitrary.fail_fix` to fail in a fixpoint - helper cases for `Arbitrary.fix_fuel` ## 0.3 - get rid of submodule `generator` - `Arbitrary.fix_fuel`, to generate complex recursive structures - new combinators (infix map, applicative funs, shuffle) - remove generator/Generator, and a deprecation warning - output of printers of lists/arrays now parsable by ocaml toplevel ## 0.2 - integrate Gabriel Scherer's `Generator` into `QCheck` - add `|||` - add `Prop.raises` - print the faulty instance in case of error (if a printer is available) - some combinators for `QCheck.Arbitrary` - `QCheck.mk_test` takes more arguments ## 0.1 - oasis based build system - source files
CHANGES: ## 0.9 - add `qcheck-ounit` sublibrary - use environment variables to configure `qcheck-alcotest` tests - alcotest backend for qcheck - make `qcheck.ounit` tests verbose by default - make `qcheck` is a compatibility library, depends on `qcheck-core` - split lib into `qcheck` and `qcheck.ounit` - add `TestResult.is_success` helper - give access to list of instances in test results - allow setting `time_between_msg` in runner - chore: remove submodule - chore: add travis support - doc: explanations about qcheck.ounit runners - doc: update readme ## 0.8 - migrate to jbuilder - fix warnings - add some useful functions - update oasis files (close c-cube/qcheck#48) - update copyright header (closes c-cube/qcheck#47) ## 0.7 - switch to BSD license, make it more explicit (close c-cube/qcheck#43) - improve multi-line message printing in ounit (closes c-cube/qcheck#46) - fix complexity of `add_stat` - allow negative entries in statistics (see c-cube/qcheck#40) - add a way for tests to report messages to the user (see c-cube/qcheck#39) - add `QCheck.Shrink.int_aggressive` and make default int shrinker faster - shrinker for `map_keep_input` - add `QCheck.set_gen`, missing for some reason - more compact verbose output (see c-cube/qcheck#33) - better handling of dynamic progress line - Add colors to checkmarks in verbose mode - improve statistics display for runner - recover exception of shrunk input - print status line before the solving starts ## 0.6 - add `find_example` and `find_example_gen` to synthesize values from properties (see c-cube/qcheck#31) - add `QCheck.gen` for accessing the random generator easily - colorful runners, with `--no-colors` to disable them - add more generator (for corner cases) - better generation of random functions (see c-cube/qcheck#8), using `Observable` and an efficient internal representation using heterogeneous tuples, printing, and shrinking. deprecate old hacks. - add statistics gathering and display (see c-cube/qcheck#30) - better printing of Tuple - improve `Shrink.{array,list}` (see c-cube/qcheck#32) - Change asserts to raise `Invalid_arg` (following the doc), and update doc - Change `Gen.{int_bount,int_range}` to support up to 2^62 ## 0.5.3.1 - fix regression in runner output (print results of `collect`) - update the `@since` tags ## 0.5.3 - missing char in `Gen.char` (close c-cube/qcheck#23) - add `test` and `doc` to opam - add `small_list` generator - add `~long_factor` to tests and runner, for long tests - add more examples in readme, better doc for runners - improved reporting when running qcheck tests - add `Test.get_count` on test cells ## 0.5.2 - Add cli option for backtraces in `QCheck_runner` - Add test case for raising exception - Better handling of backtraces - All tests now have a name - Add step function called on each instance in a test - make `small_int` a deprecated alias to `small_nat` - add `small_signed_int` - remove some warnings - use safe-string, and fix related bug - Add long tests options to `QCheck_runner` - Add `length` specification for `to_ounit2_test` - Added paragraph in README about long tests ## 0.5.1 - document exceptions - add `small_nat`, change `small_int` semantics (close c-cube/qcheck#10) - add `QCheck.assume_fail` - add `QCheck.assume`; explain preconditions a bit (close c-cube/qcheck#9) - Polish documentation - Added quad support uniformly ## 0.5 - merge back from `qtest`: big changes in API, shrinking, use `'a arbitrary` type that combines printer, generator, shrinker, etc. (see git log) - merlin file - reorganize sources, `_oasis`, `.merlin`, etc. ## 0.4 - bugfix in `fix_fuel` - if verbose enabled, print each test case - add `QCheck.run_main` - `QCheck_ounit.~::` - add `(>:::)` - add `qcheck_ounit ml{lib,dylib}` - trivial ounit integration - make `test_cell.name` optional - `Arbitrary.fix_fuel(_gen)`: add a recursive case - `Arbitrary.fix_fuel_gen`, similar to `fix_fuel` but threading a state bottom-up to make choices depend on the path - `Arbitrary.fail_fix` to fail in a fixpoint - helper cases for `Arbitrary.fix_fuel` ## 0.3 - get rid of submodule `generator` - `Arbitrary.fix_fuel`, to generate complex recursive structures - new combinators (infix map, applicative funs, shuffle) - remove generator/Generator, and a deprecation warning - output of printers of lists/arrays now parsable by ocaml toplevel ## 0.2 - integrate Gabriel Scherer's `Generator` into `QCheck` - add `|||` - add `Prop.raises` - print the faulty instance in case of error (if a printer is available) - some combinators for `QCheck.Arbitrary` - `QCheck.mk_test` takes more arguments ## 0.1 - oasis based build system - source files
CHANGES: ## 0.9 - add `qcheck-ounit` sublibrary - use environment variables to configure `qcheck-alcotest` tests - alcotest backend for qcheck - make `qcheck.ounit` tests verbose by default - make `qcheck` is a compatibility library, depends on `qcheck-core` - split lib into `qcheck` and `qcheck.ounit` - add `TestResult.is_success` helper - give access to list of instances in test results - allow setting `time_between_msg` in runner - chore: remove submodule - chore: add travis support - doc: explanations about qcheck.ounit runners - doc: update readme ## 0.8 - migrate to jbuilder - fix warnings - add some useful functions - update oasis files (close c-cube/qcheck#48) - update copyright header (closes c-cube/qcheck#47) ## 0.7 - switch to BSD license, make it more explicit (close c-cube/qcheck#43) - improve multi-line message printing in ounit (closes c-cube/qcheck#46) - fix complexity of `add_stat` - allow negative entries in statistics (see c-cube/qcheck#40) - add a way for tests to report messages to the user (see c-cube/qcheck#39) - add `QCheck.Shrink.int_aggressive` and make default int shrinker faster - shrinker for `map_keep_input` - add `QCheck.set_gen`, missing for some reason - more compact verbose output (see c-cube/qcheck#33) - better handling of dynamic progress line - Add colors to checkmarks in verbose mode - improve statistics display for runner - recover exception of shrunk input - print status line before the solving starts ## 0.6 - add `find_example` and `find_example_gen` to synthesize values from properties (see c-cube/qcheck#31) - add `QCheck.gen` for accessing the random generator easily - colorful runners, with `--no-colors` to disable them - add more generator (for corner cases) - better generation of random functions (see c-cube/qcheck#8), using `Observable` and an efficient internal representation using heterogeneous tuples, printing, and shrinking. deprecate old hacks. - add statistics gathering and display (see c-cube/qcheck#30) - better printing of Tuple - improve `Shrink.{array,list}` (see c-cube/qcheck#32) - Change asserts to raise `Invalid_arg` (following the doc), and update doc - Change `Gen.{int_bount,int_range}` to support up to 2^62 ## 0.5.3.1 - fix regression in runner output (print results of `collect`) - update the `@since` tags ## 0.5.3 - missing char in `Gen.char` (close c-cube/qcheck#23) - add `test` and `doc` to opam - add `small_list` generator - add `~long_factor` to tests and runner, for long tests - add more examples in readme, better doc for runners - improved reporting when running qcheck tests - add `Test.get_count` on test cells ## 0.5.2 - Add cli option for backtraces in `QCheck_runner` - Add test case for raising exception - Better handling of backtraces - All tests now have a name - Add step function called on each instance in a test - make `small_int` a deprecated alias to `small_nat` - add `small_signed_int` - remove some warnings - use safe-string, and fix related bug - Add long tests options to `QCheck_runner` - Add `length` specification for `to_ounit2_test` - Added paragraph in README about long tests ## 0.5.1 - document exceptions - add `small_nat`, change `small_int` semantics (close c-cube/qcheck#10) - add `QCheck.assume_fail` - add `QCheck.assume`; explain preconditions a bit (close c-cube/qcheck#9) - Polish documentation - Added quad support uniformly ## 0.5 - merge back from `qtest`: big changes in API, shrinking, use `'a arbitrary` type that combines printer, generator, shrinker, etc. (see git log) - merlin file - reorganize sources, `_oasis`, `.merlin`, etc. ## 0.4 - bugfix in `fix_fuel` - if verbose enabled, print each test case - add `QCheck.run_main` - `QCheck_ounit.~::` - add `(>:::)` - add `qcheck_ounit ml{lib,dylib}` - trivial ounit integration - make `test_cell.name` optional - `Arbitrary.fix_fuel(_gen)`: add a recursive case - `Arbitrary.fix_fuel_gen`, similar to `fix_fuel` but threading a state bottom-up to make choices depend on the path - `Arbitrary.fail_fix` to fail in a fixpoint - helper cases for `Arbitrary.fix_fuel` ## 0.3 - get rid of submodule `generator` - `Arbitrary.fix_fuel`, to generate complex recursive structures - new combinators (infix map, applicative funs, shuffle) - remove generator/Generator, and a deprecation warning - output of printers of lists/arrays now parsable by ocaml toplevel ## 0.2 - integrate Gabriel Scherer's `Generator` into `QCheck` - add `|||` - add `Prop.raises` - print the faulty instance in case of error (if a printer is available) - some combinators for `QCheck.Arbitrary` - `QCheck.mk_test` takes more arguments ## 0.1 - oasis based build system - source files
…0.14) CHANGES: ## 0.14 - modify `int_range` to make it accept ranges bigger than `max_int`. - less newline-verbose stats - add `int{32,64}` shrinkers to arbitrary gens - add `int{32,int64}` shrinkers - move to ounit2 for `QCheck_ounit` ## 0.13 - make counter private - Add debug shrinking log - fix: small fix related to stdlib/pervasives - feat: add flatten combinators in `gen` ## 0.12 - fix singleton list shrinking - feat: add `Gen.char_range` and `Gen.(<$>)` (credit @spewspews) ## 0.11 - Add `QCheck.Gen.{string_of,string_readable}` - fix `int_bound` bound inclusiveness problem - change implementation of `int_bound` to generate values using `Random.State.int` for `bound < 2^30` - add weighted shuffled lists generator - add `float_range` to generate a floating-point number in the given range (inclusive) - add `float_bound_inclusive` and `float_bound_exclusive` to generate floating-point numbers between 0 and a given bound ## 0.10 - `Shrink`: decompose Shrink.list into Shrink.list_spine and Shrink.list_elems - `Gen.fix` has a more general and useful type - update README to include `Rely` section (qcheck now available for reason-native!) - Fix stat printing - speed-up list shrinker - Better int shrinking - core: modify proba distributions again, add `big_nat` - feat: add `small_array`, modify distributions - print number of warnings in runner's summary - refactor: modify type of results to make them more accurate - feat: warn/fail if too many tests passed only b/c precondition failed ## 0.9 - add `qcheck-ounit` sublibrary - use environment variables to configure `qcheck-alcotest` tests - alcotest backend for qcheck - make `qcheck.ounit` tests verbose by default - make `qcheck` is a compatibility library, depends on `qcheck-core` - split lib into `qcheck` and `qcheck.ounit` - add `TestResult.is_success` helper - give access to list of instances in test results - allow setting `time_between_msg` in runner - chore: remove submodule - chore: add travis support - doc: explanations about qcheck.ounit runners - doc: update readme ## 0.8 - migrate to jbuilder - fix warnings - add some useful functions - update oasis files (close c-cube/qcheck#48) - update copyright header (closes c-cube/qcheck#47) ## 0.7 - switch to BSD license, make it more explicit (close c-cube/qcheck#43) - improve multi-line message printing in ounit (closes c-cube/qcheck#46) - fix complexity of `add_stat` - allow negative entries in statistics (see c-cube/qcheck#40) - add a way for tests to report messages to the user (see c-cube/qcheck#39) - add `QCheck.Shrink.int_aggressive` and make default int shrinker faster - shrinker for `map_keep_input` - add `QCheck.set_gen`, missing for some reason - more compact verbose output (see c-cube/qcheck#33) - better handling of dynamic progress line - Add colors to checkmarks in verbose mode - improve statistics display for runner - recover exception of shrunk input - print status line before the solving starts ## 0.6 - add `find_example` and `find_example_gen` to synthesize values from properties (see c-cube/qcheck#31) - add `QCheck.gen` for accessing the random generator easily - colorful runners, with `--no-colors` to disable them - add more generator (for corner cases) - better generation of random functions (see c-cube/qcheck#8), using `Observable` and an efficient internal representation using heterogeneous tuples, printing, and shrinking. deprecate old hacks. - add statistics gathering and display (see c-cube/qcheck#30) - better printing of Tuple - improve `Shrink.{array,list}` (see c-cube/qcheck#32) - Change asserts to raise `Invalid_arg` (following the doc), and update doc - Change `Gen.{int_bount,int_range}` to support up to 2^62 ## 0.5.3.1 - fix regression in runner output (print results of `collect`) - update the `@since` tags ## 0.5.3 - missing char in `Gen.char` (close c-cube/qcheck#23) - add `test` and `doc` to opam - add `small_list` generator - add `~long_factor` to tests and runner, for long tests - add more examples in readme, better doc for runners - improved reporting when running qcheck tests - add `Test.get_count` on test cells ## 0.5.2 - Add cli option for backtraces in `QCheck_runner` - Add test case for raising exception - Better handling of backtraces - All tests now have a name - Add step function called on each instance in a test - make `small_int` a deprecated alias to `small_nat` - add `small_signed_int` - remove some warnings - use safe-string, and fix related bug - Add long tests options to `QCheck_runner` - Add `length` specification for `to_ounit2_test` - Added paragraph in README about long tests ## 0.5.1 - document exceptions - add `small_nat`, change `small_int` semantics (close c-cube/qcheck#10) - add `QCheck.assume_fail` - add `QCheck.assume`; explain preconditions a bit (close c-cube/qcheck#9) - Polish documentation - Added quad support uniformly ## 0.5 - merge back from `qtest`: big changes in API, shrinking, use `'a arbitrary` type that combines printer, generator, shrinker, etc. (see git log) - merlin file - reorganize sources, `_oasis`, `.merlin`, etc. ## 0.4 - bugfix in `fix_fuel` - if verbose enabled, print each test case - add `QCheck.run_main` - `QCheck_ounit.~::` - add `(>:::)` - add `qcheck_ounit ml{lib,dylib}` - trivial ounit integration - make `test_cell.name` optional - `Arbitrary.fix_fuel(_gen)`: add a recursive case - `Arbitrary.fix_fuel_gen`, similar to `fix_fuel` but threading a state bottom-up to make choices depend on the path - `Arbitrary.fail_fix` to fail in a fixpoint - helper cases for `Arbitrary.fix_fuel` ## 0.3 - get rid of submodule `generator` - `Arbitrary.fix_fuel`, to generate complex recursive structures - new combinators (infix map, applicative funs, shuffle) - remove generator/Generator, and a deprecation warning - output of printers of lists/arrays now parsable by ocaml toplevel ## 0.2 - integrate Gabriel Scherer's `Generator` into `QCheck` - add `|||` - add `Prop.raises` - print the faulty instance in case of error (if a printer is available) - some combinators for `QCheck.Arbitrary` - `QCheck.mk_test` takes more arguments ## 0.1 - oasis based build system - source files
…0.14) CHANGES: ## 0.14 - modify `int_range` to make it accept ranges bigger than `max_int`. - less newline-verbose stats - add `int{32,64}` shrinkers to arbitrary gens - add `int{32,int64}` shrinkers - move to ounit2 for `QCheck_ounit` ## 0.13 - make counter private - Add debug shrinking log - fix: small fix related to stdlib/pervasives - feat: add flatten combinators in `gen` ## 0.12 - fix singleton list shrinking - feat: add `Gen.char_range` and `Gen.(<$>)` (credit @spewspews) ## 0.11 - Add `QCheck.Gen.{string_of,string_readable}` - fix `int_bound` bound inclusiveness problem - change implementation of `int_bound` to generate values using `Random.State.int` for `bound < 2^30` - add weighted shuffled lists generator - add `float_range` to generate a floating-point number in the given range (inclusive) - add `float_bound_inclusive` and `float_bound_exclusive` to generate floating-point numbers between 0 and a given bound ## 0.10 - `Shrink`: decompose Shrink.list into Shrink.list_spine and Shrink.list_elems - `Gen.fix` has a more general and useful type - update README to include `Rely` section (qcheck now available for reason-native!) - Fix stat printing - speed-up list shrinker - Better int shrinking - core: modify proba distributions again, add `big_nat` - feat: add `small_array`, modify distributions - print number of warnings in runner's summary - refactor: modify type of results to make them more accurate - feat: warn/fail if too many tests passed only b/c precondition failed ## 0.9 - add `qcheck-ounit` sublibrary - use environment variables to configure `qcheck-alcotest` tests - alcotest backend for qcheck - make `qcheck.ounit` tests verbose by default - make `qcheck` is a compatibility library, depends on `qcheck-core` - split lib into `qcheck` and `qcheck.ounit` - add `TestResult.is_success` helper - give access to list of instances in test results - allow setting `time_between_msg` in runner - chore: remove submodule - chore: add travis support - doc: explanations about qcheck.ounit runners - doc: update readme ## 0.8 - migrate to jbuilder - fix warnings - add some useful functions - update oasis files (close c-cube/qcheck#48) - update copyright header (closes c-cube/qcheck#47) ## 0.7 - switch to BSD license, make it more explicit (close c-cube/qcheck#43) - improve multi-line message printing in ounit (closes c-cube/qcheck#46) - fix complexity of `add_stat` - allow negative entries in statistics (see c-cube/qcheck#40) - add a way for tests to report messages to the user (see c-cube/qcheck#39) - add `QCheck.Shrink.int_aggressive` and make default int shrinker faster - shrinker for `map_keep_input` - add `QCheck.set_gen`, missing for some reason - more compact verbose output (see c-cube/qcheck#33) - better handling of dynamic progress line - Add colors to checkmarks in verbose mode - improve statistics display for runner - recover exception of shrunk input - print status line before the solving starts ## 0.6 - add `find_example` and `find_example_gen` to synthesize values from properties (see c-cube/qcheck#31) - add `QCheck.gen` for accessing the random generator easily - colorful runners, with `--no-colors` to disable them - add more generator (for corner cases) - better generation of random functions (see c-cube/qcheck#8), using `Observable` and an efficient internal representation using heterogeneous tuples, printing, and shrinking. deprecate old hacks. - add statistics gathering and display (see c-cube/qcheck#30) - better printing of Tuple - improve `Shrink.{array,list}` (see c-cube/qcheck#32) - Change asserts to raise `Invalid_arg` (following the doc), and update doc - Change `Gen.{int_bount,int_range}` to support up to 2^62 ## 0.5.3.1 - fix regression in runner output (print results of `collect`) - update the `@since` tags ## 0.5.3 - missing char in `Gen.char` (close c-cube/qcheck#23) - add `test` and `doc` to opam - add `small_list` generator - add `~long_factor` to tests and runner, for long tests - add more examples in readme, better doc for runners - improved reporting when running qcheck tests - add `Test.get_count` on test cells ## 0.5.2 - Add cli option for backtraces in `QCheck_runner` - Add test case for raising exception - Better handling of backtraces - All tests now have a name - Add step function called on each instance in a test - make `small_int` a deprecated alias to `small_nat` - add `small_signed_int` - remove some warnings - use safe-string, and fix related bug - Add long tests options to `QCheck_runner` - Add `length` specification for `to_ounit2_test` - Added paragraph in README about long tests ## 0.5.1 - document exceptions - add `small_nat`, change `small_int` semantics (close c-cube/qcheck#10) - add `QCheck.assume_fail` - add `QCheck.assume`; explain preconditions a bit (close c-cube/qcheck#9) - Polish documentation - Added quad support uniformly ## 0.5 - merge back from `qtest`: big changes in API, shrinking, use `'a arbitrary` type that combines printer, generator, shrinker, etc. (see git log) - merlin file - reorganize sources, `_oasis`, `.merlin`, etc. ## 0.4 - bugfix in `fix_fuel` - if verbose enabled, print each test case - add `QCheck.run_main` - `QCheck_ounit.~::` - add `(>:::)` - add `qcheck_ounit ml{lib,dylib}` - trivial ounit integration - make `test_cell.name` optional - `Arbitrary.fix_fuel(_gen)`: add a recursive case - `Arbitrary.fix_fuel_gen`, similar to `fix_fuel` but threading a state bottom-up to make choices depend on the path - `Arbitrary.fail_fix` to fail in a fixpoint - helper cases for `Arbitrary.fix_fuel` ## 0.3 - get rid of submodule `generator` - `Arbitrary.fix_fuel`, to generate complex recursive structures - new combinators (infix map, applicative funs, shuffle) - remove generator/Generator, and a deprecation warning - output of printers of lists/arrays now parsable by ocaml toplevel ## 0.2 - integrate Gabriel Scherer's `Generator` into `QCheck` - add `|||` - add `Prop.raises` - print the faulty instance in case of error (if a printer is available) - some combinators for `QCheck.Arbitrary` - `QCheck.mk_test` takes more arguments ## 0.1 - oasis based build system - source files
…0.14) CHANGES: ## 0.14 - modify `int_range` to make it accept ranges bigger than `max_int`. - less newline-verbose stats - add `int{32,64}` shrinkers to arbitrary gens - add `int{32,int64}` shrinkers - move to ounit2 for `QCheck_ounit` ## 0.13 - make counter private - Add debug shrinking log - fix: small fix related to stdlib/pervasives - feat: add flatten combinators in `gen` ## 0.12 - fix singleton list shrinking - feat: add `Gen.char_range` and `Gen.(<$>)` (credit @spewspews) ## 0.11 - Add `QCheck.Gen.{string_of,string_readable}` - fix `int_bound` bound inclusiveness problem - change implementation of `int_bound` to generate values using `Random.State.int` for `bound < 2^30` - add weighted shuffled lists generator - add `float_range` to generate a floating-point number in the given range (inclusive) - add `float_bound_inclusive` and `float_bound_exclusive` to generate floating-point numbers between 0 and a given bound ## 0.10 - `Shrink`: decompose Shrink.list into Shrink.list_spine and Shrink.list_elems - `Gen.fix` has a more general and useful type - update README to include `Rely` section (qcheck now available for reason-native!) - Fix stat printing - speed-up list shrinker - Better int shrinking - core: modify proba distributions again, add `big_nat` - feat: add `small_array`, modify distributions - print number of warnings in runner's summary - refactor: modify type of results to make them more accurate - feat: warn/fail if too many tests passed only b/c precondition failed ## 0.9 - add `qcheck-ounit` sublibrary - use environment variables to configure `qcheck-alcotest` tests - alcotest backend for qcheck - make `qcheck.ounit` tests verbose by default - make `qcheck` is a compatibility library, depends on `qcheck-core` - split lib into `qcheck` and `qcheck.ounit` - add `TestResult.is_success` helper - give access to list of instances in test results - allow setting `time_between_msg` in runner - chore: remove submodule - chore: add travis support - doc: explanations about qcheck.ounit runners - doc: update readme ## 0.8 - migrate to jbuilder - fix warnings - add some useful functions - update oasis files (close c-cube/qcheck#48) - update copyright header (closes c-cube/qcheck#47) ## 0.7 - switch to BSD license, make it more explicit (close c-cube/qcheck#43) - improve multi-line message printing in ounit (closes c-cube/qcheck#46) - fix complexity of `add_stat` - allow negative entries in statistics (see c-cube/qcheck#40) - add a way for tests to report messages to the user (see c-cube/qcheck#39) - add `QCheck.Shrink.int_aggressive` and make default int shrinker faster - shrinker for `map_keep_input` - add `QCheck.set_gen`, missing for some reason - more compact verbose output (see c-cube/qcheck#33) - better handling of dynamic progress line - Add colors to checkmarks in verbose mode - improve statistics display for runner - recover exception of shrunk input - print status line before the solving starts ## 0.6 - add `find_example` and `find_example_gen` to synthesize values from properties (see c-cube/qcheck#31) - add `QCheck.gen` for accessing the random generator easily - colorful runners, with `--no-colors` to disable them - add more generator (for corner cases) - better generation of random functions (see c-cube/qcheck#8), using `Observable` and an efficient internal representation using heterogeneous tuples, printing, and shrinking. deprecate old hacks. - add statistics gathering and display (see c-cube/qcheck#30) - better printing of Tuple - improve `Shrink.{array,list}` (see c-cube/qcheck#32) - Change asserts to raise `Invalid_arg` (following the doc), and update doc - Change `Gen.{int_bount,int_range}` to support up to 2^62 ## 0.5.3.1 - fix regression in runner output (print results of `collect`) - update the `@since` tags ## 0.5.3 - missing char in `Gen.char` (close c-cube/qcheck#23) - add `test` and `doc` to opam - add `small_list` generator - add `~long_factor` to tests and runner, for long tests - add more examples in readme, better doc for runners - improved reporting when running qcheck tests - add `Test.get_count` on test cells ## 0.5.2 - Add cli option for backtraces in `QCheck_runner` - Add test case for raising exception - Better handling of backtraces - All tests now have a name - Add step function called on each instance in a test - make `small_int` a deprecated alias to `small_nat` - add `small_signed_int` - remove some warnings - use safe-string, and fix related bug - Add long tests options to `QCheck_runner` - Add `length` specification for `to_ounit2_test` - Added paragraph in README about long tests ## 0.5.1 - document exceptions - add `small_nat`, change `small_int` semantics (close c-cube/qcheck#10) - add `QCheck.assume_fail` - add `QCheck.assume`; explain preconditions a bit (close c-cube/qcheck#9) - Polish documentation - Added quad support uniformly ## 0.5 - merge back from `qtest`: big changes in API, shrinking, use `'a arbitrary` type that combines printer, generator, shrinker, etc. (see git log) - merlin file - reorganize sources, `_oasis`, `.merlin`, etc. ## 0.4 - bugfix in `fix_fuel` - if verbose enabled, print each test case - add `QCheck.run_main` - `QCheck_ounit.~::` - add `(>:::)` - add `qcheck_ounit ml{lib,dylib}` - trivial ounit integration - make `test_cell.name` optional - `Arbitrary.fix_fuel(_gen)`: add a recursive case - `Arbitrary.fix_fuel_gen`, similar to `fix_fuel` but threading a state bottom-up to make choices depend on the path - `Arbitrary.fail_fix` to fail in a fixpoint - helper cases for `Arbitrary.fix_fuel` ## 0.3 - get rid of submodule `generator` - `Arbitrary.fix_fuel`, to generate complex recursive structures - new combinators (infix map, applicative funs, shuffle) - remove generator/Generator, and a deprecation warning - output of printers of lists/arrays now parsable by ocaml toplevel ## 0.2 - integrate Gabriel Scherer's `Generator` into `QCheck` - add `|||` - add `Prop.raises` - print the faulty instance in case of error (if a printer is available) - some combinators for `QCheck.Arbitrary` - `QCheck.mk_test` takes more arguments ## 0.1 - oasis based build system - source files
int -> foo
, generate a random decision tree on integers (based on intervals and, say, special cases like 0), then generate a randomfoo
in each leafThe text was updated successfully, but these errors were encountered: