Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make tests pass and improve shrinking for prop_flat_map #269

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 3 additions & 3 deletions proptest-derive/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "proptest-derive"
version = "0.3.0"
version = "0.3.1"
authors = ["Mazdak Farrokhzad <twingoow@gmail.com>"]
license = "MIT/Apache-2.0"
readme = "README.md"
Expand All @@ -22,12 +22,12 @@ edition = "2018"
proc-macro = true

[dev-dependencies]
proptest = { version = "1.0.0", path = "../proptest" }
proptest = { version = "1.1.0", path = "../proptest" }
# We don't actually run the tests on stable since some of them use nightly
# features. However, due to
# https://github.com/laumann/compiletest-rs/issues/166, the default features of
# compiletest-rs fail to compile, but the stable fallback works fine.
compiletest_rs = { version = "0.3.19", features = ["tmp", "stable"] }
compiletest_rs = { version = "0.8.0", features = ["tmp", "stable"] }
# criterion is used for benchmarks.
criterion = "0.2"

Expand Down
4 changes: 2 additions & 2 deletions proptest-derive/tests/compile-fail/no-arbitrary.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,5 @@ fn main() {}

struct T0;

#[derive(Debug, Arbitrary)] //~ Arbitrary` is not satisfied [E0277]
struct T1 { f0: T0, }
#[derive(Debug, Arbitrary)] //~ ERROR Arbitrary` is not satisfied [E0277]
struct T1 { f0: T0, } //~ ERROR Arbitrary` is not satisfied [E0277]
2 changes: 1 addition & 1 deletion proptest-derive/tests/compiletest.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ fn run_mode(src: &'static str, mode: &'static str) {
config.target_rustcflags =
Some("-L ../target/debug/deps --edition=2018".to_owned());
if let Ok(name) = env::var("TESTNAME") {
config.filter = Some(name);
config.filters = vec![name];
}
config.src_base = format!("tests/{}", src).into();

Expand Down
17 changes: 17 additions & 0 deletions proptest/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,20 @@
## Unreleased

### Breaking Changes

- `prop_flat_map` can only be used with closures that produce a strategy that
implements `Copy`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned about this, I often use Strategies over String, sometimes even Just(String), which is Clone but not Copy.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that perhaps a typo? The initial issue mentions Clone.

Regardless, it seems a little risky to require Clone as it might permanently break lots of proptests out there. Would it make sense to instead provide it under a new name, preserving the current prop_flat_map? Not ideal either, I know...

I otherwise am happy to see some effort in trying to do something about the shrinking behavior for prop_flat_map. Like @nzeh I've run into the issue of poor shrinking many times.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ugh. Yes, that's a typo. The updated implementation in the pull request requires Cloneable strategies, not ones that implement Copy. I aggree with @Andlon that adding even the requirement that the strategy implement Clone is suboptimal. After spending quite some time thinking about how to avoid it at the time, I concluded that doing what my shrink implementation does (shrink the outer strategy first and then the inner one) is not possible with the current architecture of the ValueTree trait. I think it is possible if we completely overhaul the architecture of at least parts of proptest, but that was not something I intended to undertake at the time. Moreover, this overhaul in itself could lead to breakage of code that uses proptest because it would involve changing at least the ValueTree trait.


- Half-bounded ranges (`x..`, `..x`, and `..=x`) are no longer supported for
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This problem was fixed in #261 . These changes aren't needed anymore.

`f32` and `f64`.

### New Features

- The shrink strategy for `prop_flat_map` now shrinks the "outer" strategy
first (the one that produces the values given to `prop_flat_map`) and
shrinks the "inner" strategy" (the one produced by `prop_flat_map`) only
when the outer strategy cannot be shrunk further.

## 1.0.0

### Breaking Changes
Expand Down
2 changes: 1 addition & 1 deletion proptest/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "proptest"
version = "1.0.0"
version = "1.1.0"
authors = ["Jason Lingle"]
license = "MIT/Apache-2.0"
readme = "README.md"
Expand Down
90 changes: 73 additions & 17 deletions proptest/src/num.rs
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ macro_rules! int_any {
};
}

macro_rules! numeric_api {
macro_rules! int_api {
($typ:ident, $epsilon:expr) => {
impl Strategy for ::core::ops::Range<$typ> {
type Tree = BinarySearch;
Expand Down Expand Up @@ -137,6 +137,40 @@ macro_rules! numeric_api {
};
}

macro_rules! float_api {
($typ:ident, $epsilon:expr) => {
impl Strategy for ::core::ops::Range<$typ> {
type Tree = BinarySearch;
type Value = $typ;

fn new_tree(&self, runner: &mut TestRunner) -> NewTree<Self> {
Ok(BinarySearch::new_clamped(
self.start,
$crate::num::sample_uniform(runner, self.clone()),
self.end - $epsilon,
))
}
}

impl Strategy for ::core::ops::RangeInclusive<$typ> {
type Tree = BinarySearch;
type Value = $typ;

fn new_tree(&self, runner: &mut TestRunner) -> NewTree<Self> {
Ok(BinarySearch::new_clamped(
*self.start(),
$crate::num::sample_uniform_incl(
runner,
*self.start(),
*self.end(),
),
*self.end(),
))
}
}
};
}

macro_rules! signed_integer_bin_search {
($typ:ident) => {
#[allow(missing_docs)]
Expand Down Expand Up @@ -234,7 +268,7 @@ macro_rules! signed_integer_bin_search {
}
}

numeric_api!($typ, 1);
int_api!($typ, 1);
}
};
}
Expand Down Expand Up @@ -322,7 +356,7 @@ macro_rules! unsigned_integer_bin_search {
}
}

numeric_api!($typ, 1);
int_api!($typ, 1);
}
};
}
Expand Down Expand Up @@ -842,7 +876,7 @@ macro_rules! float_bin_search {
}
}

numeric_api!($typ, 0.0);
float_api!($typ, 0.0);
}
};
}
Expand Down Expand Up @@ -1032,7 +1066,7 @@ mod test {
}

mod contract_sanity {
macro_rules! contract_sanity {
macro_rules! contract_sanity_int {
($t:tt) => {
mod $t {
use crate::strategy::check_strategy_sanity;
Expand Down Expand Up @@ -1067,18 +1101,40 @@ mod test {
}
};
}
contract_sanity!(u8);
contract_sanity!(i8);
contract_sanity!(u16);
contract_sanity!(i16);
contract_sanity!(u32);
contract_sanity!(i32);
contract_sanity!(u64);
contract_sanity!(i64);
contract_sanity!(usize);
contract_sanity!(isize);
contract_sanity!(f32);
contract_sanity!(f64);

macro_rules! contract_sanity_float {
($t:tt) => {
mod $t {
use crate::strategy::check_strategy_sanity;

const FOURTY_TWO: $t = 42 as $t;
const FIFTY_SIX: $t = 56 as $t;

#[test]
fn range() {
check_strategy_sanity(FOURTY_TWO..FIFTY_SIX, None);
}

#[test]
fn range_inclusive() {
check_strategy_sanity(FOURTY_TWO..=FIFTY_SIX, None);
}
}
};
}

contract_sanity_int!(u8);
contract_sanity_int!(i8);
contract_sanity_int!(u16);
contract_sanity_int!(i16);
contract_sanity_int!(u32);
contract_sanity_int!(i32);
contract_sanity_int!(u64);
contract_sanity_int!(i64);
contract_sanity_int!(usize);
contract_sanity_int!(isize);
contract_sanity_float!(f32);
contract_sanity_float!(f64);
}

#[test]
Expand Down
80 changes: 34 additions & 46 deletions proptest/src/strategy/flatten.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@
// except according to those terms.

use crate::std_facade::{fmt, Arc};
use core::mem;

use crate::strategy::fuse::Fuse;
use crate::strategy::traits::*;
use crate::test_runner::*;
use std::mem;

/// Adaptor that flattens a `Strategy` which produces other `Strategy`s into a
/// `Strategy` that picks one of those strategies and then picks values from
Expand All @@ -33,6 +33,7 @@ impl<S: Strategy> Flatten<S> {
impl<S: Strategy> Strategy for Flatten<S>
where
S::Value: Strategy,
<S::Value as Strategy>::Tree: Clone,
{
type Tree = FlattenValueTree<S::Tree>;
type Value = <S::Value as Strategy>::Value;
Expand All @@ -50,19 +51,7 @@ where
{
meta: Fuse<S>,
current: Fuse<<S::Value as Strategy>::Tree>,
// The final value to produce after successive calls to complicate() on the
// underlying objects return false.
final_complication: Option<Fuse<<S::Value as Strategy>::Tree>>,
// When `simplify()` or `complicate()` causes a new `Strategy` to be
// chosen, we need to find a new failing input for that case. To do this,
// we implement `complicate()` by regenerating values up to a number of
// times corresponding to the maximum number of test cases. A `simplify()`
// which does not cause a new strategy to be chosen always resets
// `complicate_regen_remaining` to 0.
//
// This does unfortunately depart from the direct interpretation of
// simplify/complicate as binary search, but is still easier to think about
// than other implementations of higher-order strategies.
last_complication: Option<Fuse<<S::Value as Strategy>::Tree>>,
runner: TestRunner,
complicate_regen_remaining: u32,
}
Expand All @@ -77,7 +66,7 @@ where
FlattenValueTree {
meta: self.meta.clone(),
current: self.current.clone(),
final_complication: self.final_complication.clone(),
last_complication: self.last_complication.clone(),
runner: self.runner.clone(),
complicate_regen_remaining: self.complicate_regen_remaining,
}
Expand All @@ -94,7 +83,7 @@ where
f.debug_struct("FlattenValueTree")
.field("meta", &self.meta)
.field("current", &self.current)
.field("final_complication", &self.final_complication)
.field("last_complication", &self.last_complication)
.field(
"complicate_regen_remaining",
&self.complicate_regen_remaining,
Expand All @@ -112,7 +101,7 @@ where
Ok(FlattenValueTree {
meta: Fuse::new(meta),
current: Fuse::new(current),
final_complication: None,
last_complication: None,
runner: runner.partial_clone(),
complicate_regen_remaining: 0,
})
Expand All @@ -122,6 +111,7 @@ where
impl<S: ValueTree> ValueTree for FlattenValueTree<S>
where
S::Value: Strategy,
<S::Value as Strategy>::Tree: Clone,
{
type Value = <S::Value as Strategy>::Value;

Expand All @@ -130,33 +120,28 @@ where
}

fn simplify(&mut self) -> bool {
self.current.disallow_complicate();

if self.meta.simplify() {
if let Ok(v) = self.meta.current().new_tree(&mut self.runner) {
self.last_complication = Some(Fuse::new(v));
mem::swap(
self.last_complication.as_mut().unwrap(),
&mut self.current,
);
self.complicate_regen_remaining = self.runner.config().cases;
return true;
} else {
self.meta.disallow_simplify();
}
}

self.complicate_regen_remaining = 0;
let mut old_current = self.current.clone();
old_current.disallow_simplify();

if self.current.simplify() {
// Now that we've simplified the derivative value, we can't
// re-complicate the meta value unless it gets simplified again.
// We also mustn't complicate back to whatever's in
// `final_complication` since the new state of `self.current` is
// the most complicated state.
self.meta.disallow_complicate();
self.final_complication = None;
true
} else if !self.meta.simplify() {
false
} else if let Ok(v) = self.meta.current().new_tree(&mut self.runner) {
// Shift current into final_complication and `v` into
// `current`. We also need to prevent that value from
// complicating beyond the current point in the future
// since we're going to return `true` from `simplify()`
// ourselves.
self.current.disallow_complicate();
self.final_complication = Some(Fuse::new(v));
mem::swap(
self.final_complication.as_mut().unwrap(),
&mut self.current,
);
// Initially complicate by regenerating the chosen value.
self.complicate_regen_remaining = self.runner.config().cases;
self.last_complication = Some(old_current);
true
} else {
false
Expand All @@ -177,17 +162,20 @@ where
}
}

if self.current.complicate() {
return true;
} else if self.meta.complicate() {
if self.meta.complicate() {
if let Ok(v) = self.meta.current().new_tree(&mut self.runner) {
self.complicate_regen_remaining = self.runner.config().cases;
self.current = Fuse::new(v);
self.complicate_regen_remaining = self.runner.config().cases;
return true;
} else {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dead code?

}
}

if let Some(v) = self.final_complication.take() {
if self.current.complicate() {
return true;
}

if let Some(v) = self.last_complication.take() {
self.current = v;
true
} else {
Expand Down