Implementation for learning MMLTs, new model for collecting statistics

CHANGELOG.md

@@ -8,11 +8,14 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 ### Added
 
+* Added a new (L*-based) learning algorithm for *Mealy machines with local timers* (MMLTs), including support for parallel queries, caching, and conformance testing (thanks to [Paul Kogel](https://github.com/pdev55)).
 * Added an `EarlyExitEQOracle` which for a given `AdaptiveMembershipOracle` and `TestWordGenerator` stops the evaluation of (potentially long) Mealy-based equivalence tests as soon as a mismatch with the hypothesis is detected, potentially improving the symbol performance of the given equivalence oracle.
 
 ### Changed
 
 * LearnLib now requires Java 17 at runtime.
+* Statistics collection has received a major rework. Previously, classes would implement the `StatisticCollector` interface and return a `StatisticData` object which 1) only allows for describing a very limited amount of data, and 2) requires you to keep track of all the objects that collect data. This approach has been *replaced* by a new `StatisticsService`. Instances of this service can be obtained similar to a logger via `Statistics.getService()` and require you to provide an implementation of this service on the classpath (a default one is provided by the `learnlib-statistics` module). The new service allows arbitrary components to collect various data which can be conveniently extracted based on the new `StatisticsKey`s used by the components. For more details on advanced scenarios (such as multi-threaded benchmarking), see the documentation of the respective classes. While this may require you to adjust the way you are collecting statistics, all functionality from beforehand should still be available.
+  * `SimpleProfiler` has been replaced by the new clock-based statistics.
 * The `generateTestWords` method of `AbstractTestWordEQOracle` now needs to be public.
 
 ### Fixed

README.md

@@ -17,19 +17,19 @@ While certain features have been stripped for improved modularity, development h
 Currently, the following learning algorithms with respective target models are supported:
 
 
-| Algorithm (active)  | Target models               |     | Algorithm (passive)   | Models                |
-|---------------------|-----------------------------|-----|-----------------------|-----------------------|
-| AAAR                | `DFA` `Mealy` `Moore`       |     | OSTIA                 | `SST`                 |
-| ADT                 | `Mealy`                     |     | RPNI (incl. variants) | `DFA` `Mealy` `Moore` |
-| DHC                 | `Mealy`                     |     |                       |                       |
-| Kearns & Vazirani   | `DFA` `Mealy`               |     |                       |                       |
-| Lambda              | `DFA` `Mealy`               |     |                       |                       |
-| L#                  | `Mealy`                     |     |                       |                       |
-| L* (incl. variants) | `DFA` `Mealy` `Moore`       |     |                       |                       |
-| NL*                 | `NFA`                       |     |                       |                       |
-| Observation Pack    | `DFA` `Mealy` `Moore` `VPA` |     |                       |                       |
-| Procedural          | `SPA` `SBA` `SPMM`          |     |                       |                       |
-| TTT                 | `DFA` `Mealy` `Moore` `VPA` |     |                       |                       |
+| Algorithm (active)  | Target models                |     | Algorithm (passive)   | Models                |
+|---------------------|------------------------------|-----|-----------------------|-----------------------|
+| AAAR                | `DFA` `Mealy` `Moore`        |     | OSTIA                 | `SST`                 |
+| ADT                 | `Mealy`                      |     | RPNI (incl. variants) | `DFA` `Mealy` `Moore` |
+| DHC                 | `Mealy`                      |     |                       |                       |
+| Kearns & Vazirani   | `DFA` `Mealy`                |     |                       |                       |
+| Lambda              | `DFA` `Mealy`                |     |                       |                       |
+| L#                  | `Mealy`                      |     |                       |                       |
+| L* (incl. variants) | `DFA` `Mealy` `Moore` `MMLT` |     |                       |                       |
+| NL*                 | `NFA`                        |     |                       |                       |
+| Observation Pack    | `DFA` `Mealy` `Moore` `VPA`  |     |                       |                       |
+| Procedural          | `SPA` `SBA` `SPMM`           |     |                       |                       |
+| TTT                 | `DFA` `Mealy` `Moore` `VPA`  |     |                       |                       |
 
 Additionally, LearnLib offers a variety of tools to ease the practical application of automata learning on real-world systems.
 This includes drivers and mappers for interfacing software systems with the LearnLib API as well as caches and parallelization for improving the overall performance of the learning setup.

algorithms/active/adt/src/test/java/de/learnlib/algorithm/adt/it/ADTIT.java

@@ -42,6 +42,7 @@
 import de.learnlib.oracle.equivalence.MealySimulatorEQOracle;
 import de.learnlib.oracle.membership.MealySimulatorOracle;
 import de.learnlib.oracle.membership.SULAdaptiveOracle;
+import de.learnlib.statistic.Statistics;
 import de.learnlib.sul.SUL;
 import de.learnlib.testsupport.MQ2AQWrapper;
 import de.learnlib.testsupport.it.learner.AbstractMealyLearnerIT;
@@ -130,6 +131,7 @@ public void testIssue137() throws IOException, FormatException {
                     for (int seed = 0; seed < 50; seed++) {
                         long last = 0;
                         for (int iter = 0; iter < 5; iter++) {
+                            Statistics.getService().clear();
                             final CounterAdaptiveQueryOracle<String, String> counter =
                                     new CounterAdaptiveQueryOracle<>(aqo);
                             final ADTLearner<String, String> learner = new ADTLearner<>(alphabet,
@@ -151,7 +153,8 @@ public void testIssue137() throws IOException, FormatException {
 
                             exp.run();
 
-                            final long count = counter.getResetCounter().getCount();
+                            final long count =
+                                    Statistics.getService().getCount(CounterAdaptiveQueryOracle.KEY_RESET).orElse(0L);
 
                             if (iter == 0) {
                                 last = count;

algorithms/active/lstar/pom.xml

@@ -50,6 +50,10 @@ limitations under the License.
             <groupId>de.learnlib</groupId>
             <artifactId>learnlib-counterexamples</artifactId>
         </dependency>
+        <dependency>
+            <groupId>de.learnlib</groupId>
+            <artifactId>learnlib-symbol-filters</artifactId>
+        </dependency>
         <dependency>
             <groupId>de.learnlib</groupId>
             <artifactId>learnlib-util</artifactId>
@@ -124,6 +128,21 @@ limitations under the License.
             <groupId>org.testng</groupId>
             <artifactId>testng</artifactId>
         </dependency>
+        <dependency>
+            <groupId>net.automatalib</groupId>
+            <artifactId>automata-serialization-dot</artifactId>
+            <scope>test</scope>
+        </dependency>
+        <dependency>
+            <groupId>de.learnlib</groupId>
+            <artifactId>learnlib-cache</artifactId>
+            <scope>test</scope>
+        </dependency>
+        <dependency>
+            <groupId>de.learnlib</groupId>
+            <artifactId>learnlib-statistics</artifactId>
+            <scope>test</scope>
+        </dependency>
     </dependencies>
 
     <build>
@@ -134,6 +153,16 @@ limitations under the License.
                     <artifactId>maven-surefire-plugin</artifactId>
                     <configuration>
                         <!-- append to existing argLine to nicely work together with jacoco plugin -->
+                        <!-- allow tests access to util and statistics -->
+                        <argLine>@{argLine} --add-reads=de.learnlib.algorithm.lstar=net.automatalib.util</argLine>
+                    </configuration>
+                </plugin>
+                <plugin>
+                    <groupId>org.apache.maven.plugins</groupId>
+                    <artifactId>maven-failsafe-plugin</artifactId>
+                    <configuration>
+                        <!-- append to existing argLine to nicely work together with jacoco plugin -->
+                        <!-- allow tests access to util and statistics -->
                         <argLine>@{argLine} --add-reads=de.learnlib.algorithm.lstar=net.automatalib.util</argLine>
                     </configuration>
                 </plugin>