Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exhaustive search via transition enumeration #542

Closed
andrey-kuprianov opened this issue Feb 4, 2021 · 3 comments · Fixed by #833
Closed

Exhaustive search via transition enumeration #542

andrey-kuprianov opened this issue Feb 4, 2021 · 3 comments · Fixed by #833
Assignees
Labels
FMBT Feature: support for model-based testing new New issue to be triaged.
Milestone

Comments

@andrey-kuprianov
Copy link
Contributor

The ability to enumere counterexamples is crucial in such usage scenarios as model-based testing, where counterexamples are used as test cases. As the first step towards #79, we need to implement the simplest exhaustive search for counterexamples: via enumerating sequences of transitions.

  • the tool should support --continue option;
  • every new counterexample returned with -continue should use a different sequence of transitions leading to an invariant violation.

In case of a lot of non-determinism this can of course blow up, because it means enumerating the tree branches. But exhaustiveness on the base of transition sequences is the simplest to implement, and offers an intuitive mental picture for the end user. This strategy can later be complemented in the scope of #79 with different ones (based on the values of state variables for example).

@andrey-kuprianov andrey-kuprianov added new New issue to be triaged. FMBT Feature: support for model-based testing labels Feb 4, 2021
@konnov konnov added this to the February iteration milestone Feb 4, 2021
@konnov
Copy link
Contributor

konnov commented Feb 4, 2021

@Kukovec do you like to give it a try? The new transition executor is documented in ADR003, which is used by SeqModelChecker.

@konnov
Copy link
Contributor

konnov commented Apr 12, 2021

I would de-prioritize this one in favor of #730

@konnov
Copy link
Contributor

konnov commented May 16, 2021

Last time we have decided to prioritize this approach, at least as a provisional solution. I think there is a nice way to implement enumeration of counterexamples and make it useful in various scenarios. See a detailed description in the discussion #827.

@konnov konnov assigned konnov and unassigned Kukovec May 16, 2021
@konnov konnov modified the milestones: backlog2021, May iteration May 16, 2021
konnov added a commit that referenced this issue Jun 11, 2021
* closes #542: enumeration of counterexamples by view abstraction

* simplify the code

* update the manual with enumeration of counterexamples

* changelog entry

* add the missing page

* fix formatting

* reporting normal errors instead of failures

* fix types in the decoder

* enumerating the same invariant over predicates

* path constraints in the right place

* Update tla-bmcmt/src/main/scala/at/forsyte/apalache/tla/bmcmt/SeqModelChecker.scala

Co-authored-by: Kukovec <jkukovec@forsyte.at>

* added a comment as suggested by Jure

* fix formatting

Co-authored-by: Kukovec <jkukovec@forsyte.at>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
FMBT Feature: support for model-based testing new New issue to be triaged.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants