Merge support for fact based unit tests

https://docs.microsoft.com/en-us/power-query/handlingunittesting

Close #14

Docs/README.md

@@ -2,10 +2,7 @@
 
 ## Table of contents
 
-- [List of LibPQ modules][modules]
-- [Unit testing in Power Query with LibPQ][unittesting]
-- [Docstrings in LibPQ][docstrings]
-
-[docstrings]: Docstrings.md
-[modules]: Modules.md
-[unittesting]: UnitTesting.md
+- [List of LibPQ modules](Modules.md)
+- [Unit testing in Power Query with LibPQ](UnitTesting.md)
+    - [Support for fact based tests](UnitTesting_with_Facts.md)
+- [Docstrings in LibPQ](Docstrings.md)

Docs/UnitTesting.md

@@ -10,9 +10,9 @@ modules:
 - **[UnitTest.Run][Run]** - runner for individual test suites
 - **[UnitTest.Subtests][Subtests]** - helper function for writing subtests
 
-# Writing tests
+## Writing tests
 
-## Test function (test case)
+### Test function (test case)
 
 Individual test cases are implemented using test functions. A test function is
 a function that takes zero arguments and produces one of three outcomes when
@@ -26,7 +26,7 @@ called:
 3. If the test code itself raises an error the test is considered neither
    succesfull nor failed, and the error must not be silenced (*ERROR*)
 
-## Test suite
+### Test suite
 
 A test suite is a collection of test functions, usually related to each other
 in some way. Test suite has to store test functions as a record with field
@@ -45,7 +45,7 @@ introduced.
 Suites without the metadata will still be executable by test runners but will
 not be visible to discovery tools.
 
-## Assertion functions
+### Assertion functions
 
 In order to simplify writing the tests LibPQ offers some useful assertion
 functions in [UnitTest.Assert][Assert] module.
@@ -54,7 +54,7 @@ Each of the functions in that module is a valid test function that tests some
 simple assertion. You can use them as building blocks for writing your own
 tests.
 
-## Subtests
+### Subtests
 
 When you need to run the same tests against slightly different parameters
 writing all the variations one by one can become tedious.
@@ -65,7 +65,7 @@ and that error will be treated as the test result.
 
 There is also a [helper function][Subtests] that simplifies creating subtests.
 
-## Test fixtures
+### Test fixtures
 
 At this moment there are no special tools for defining and invoking test
 fixtures, but `setUp` and `tearDown` field names in test suite record should be
@@ -75,23 +75,33 @@ To create test fixture now you can add a helper function to the test suite and
 store it in any field which name does not start with "test" prefix. Such
 function(s) will have to be invoked explicitly from each individual test.
 
-## Sample code
+### Sample code
 
 Check the code of [sample][Sample] test suite and the test suite
 [snippet][Snippet] to see LibPQ unit tests in action.
 
-# Running tests and test discovery
+## Running tests and test discovery
 
 Use [UnitTest.Run][Run] function to run individual test suites (passed by
 module name or as the test suite record).
 
 Use [UnitTest.Discover][Discover] function to automatically discover and run
 all test suites from local sources.
 
+## Fact based unit tests
+
+If you are familiar with [fact based unit tests][Microsoft Unit Testing] (as
+shown in Microsoft documentation) you can continue to use that approach with
+LibPQ UnitTest framework. Integrating existing fact based test suites will
+require some extra code, but the tests themselves need no modification. See
+[this article][Fact based tests] for more information.
+
 [Assert]: ../Modules/UnitTest.Assert.pq
 [Constants]: ../Modules/UnitTest.Constants.pq
 [Discover]: ../Modules/UnitTest.Discover.pq
 [Run]: ../Modules/UnitTest.Run.pq
 [Sample]: ../Samples/Tests.Sample.pq
 [Subtests]: ../Modules/UnitTest.Subtests.pq
 [Snippet]: ../Samples/Tests.Snippet.pq
+[Microsoft Unit Testing]: https://docs.microsoft.com/en-us/power-query/handlingunittesting
+[Fact based tests]: UnitTesting_with_Facts.md

Docs/UnitTesting_with_Facts.md

@@ -0,0 +1,78 @@
+# Fact based unit tests
+
+If you are familiar with [fact based unit tests][Microsoft Unit Testing] (as
+shown in Microsoft documentation) you can continue to use that approach with
+LibPQ UnitTest framework. Integrating existing fact based test suites will
+require some extra code, but the tests themselves need no modification.
+
+> **NOTE:** Authors believe that LibPQ [UnitTest] offers more features and has
+> more stable API than fact based tests offered by Microsoft. We recommend
+> writing new tests with native [UnitTest] approach and using fact based tests
+> only to integrate pre-existing test code
+
+## Writing fact based unit tests
+
+With fact based tests a **test case** is a fact generated by `Fact()` function
+that takes three arguments: test description, expected value, actual value.
+Such facts are grouped into **test suites** by packing them in a list. Test
+suites are executed with `Facts.Summarize()` function.
+
+LibPQ provides its own implementation of [Fact] and [Facts.Summarize]
+functions. Those functions have the same API as the ones described in
+Microsoft's article but they produce different output to enable compatibility
+with LibPQ UnitTest framework.
+
+To run fact based tests with LibPQ you need to import those functions into
+your test code. No other modifications are required.
+
+```javascript
+// ...beginning of the test code (skipped)...
+
+    // Import LibPQ functions that enable fact based tests
+    Fact = LibPQ("UnitTest.Fact"),
+    Facts.Summarize = LibPQ("UnitTest.Facts.Summarize"),
+
+    // Sample fact based test
+    Fact("Values should be equal (using a let statement)",
+        "Hello World",
+        let
+            a = "Hello World"
+        in
+            a
+    ),
+
+// ...the rest of the test code (skipped)...
+```
+
+## Test discovery for fact based tests
+
+LibPQ UnitTest framework supports test discovery for fact based unit tests. To
+make your tests discoverable you need to:
+
+- Store the test suite (list of facts) in a separate LibPQ module
+- Add the following metadata to the test suite: `LibPQ.TestSuite = "Facts"`
+
+Such test will be discovered and executed automatically by
+[UnitTest.Discover]. See [sample test] for more information.
+
+## Converting existing test code for use with LibPQ
+
+The only thing you need to do to execute your existing fact based tests with
+LibPQ is to inject the LibPQ versions of [Fact] and [Facts.Summarize]
+functions into your test code. You can do that by adding the following import
+statements to your modules:
+
+```javascript
+Fact = LibPQ("UnitTest.Fact"),
+Facts.Summarize = LibPQ("UnitTest.Facts.Summarize"),
+```
+
+After that you can consider making your test suite discoverable by LibPQ (as
+described previously in this article).
+
+[Microsoft Unit Testing]: https://docs.microsoft.com/en-us/power-query/handlingunittesting
+[UnitTest]: UnitTesting.md
+[Fact]: ../Modules/UnitTest.Fact.pq
+[Facts.Summarize]: ../Modules/UnitTest.Facts.Summarize.pq
+[UnitTest.Discover]: ../Modules/UnitTest.Discover.pq
+[sample test]: ../Modules/Tests.MicrosoftUnitTestDemo.pq

Modules/Tests.MicrosoftUnitTestDemo.pq

@@ -0,0 +1,38 @@
+/**
+Demo of LibPQ's compatibility with Microsoft's unit testing framework
+
+https://docs.microsoft.com/en-us/power-query/handlingunittesting
+**/
+
+let
+    Fact = LibPQ("UnitTest.Fact"),
+    Facts.Summarize = LibPQ("UnitTest.Facts.Summarize"),
+
+    UnitTesting.ReturnsABC = () => "ABC",
+    UnitTesting.Returns123 = () => "123",
+    UnitTesting.ReturnTableWithFiveRows = () => Table.Repeat(#table({"a"},{{1}}),5),
+
+    facts =
+    {
+        Fact("Check that this function returns 'ABC'",
+            "ABC",
+            UnitTesting.ReturnsABC()
+        ),
+        Fact("Check that this function returns '123'",
+            "123",
+            UnitTesting.Returns123()
+        ),
+        Fact("Result should contain 5 rows",
+            5,
+            Table.RowCount(UnitTesting.ReturnTableWithFiveRows())
+        ),
+        Fact("Values should be equal (using a let statement)",
+            "Hello World",
+            let
+                a = "Hello World"
+            in
+                a
+        )
+    }
+in
+    facts meta [LibPQ.TestSuite = "Facts"]

Modules/UnitTest.Constants.pq

@@ -10,5 +10,8 @@ Constants for LibPQ UnitTest framework
 
     /* Metadata field that indicates the record is LibPQ test suite */
     Suite.MetaField = "LibPQ.TestSuite",
-    Suite.MetaValue = 1
+    Suite.Runners = [
+        1 = "UnitTest.Run",
+        Facts = "UnitTest.Facts.Summarize"
+    ]
 ]

Modules/UnitTest.Discover.pq

@@ -14,27 +14,41 @@ let
     UnitTest.Run = LibPQ("UnitTest.Run"),
 
     SuitesFilter = Table.SelectRows(
-        Record.ToTable(Library),
-        each (
-            try
-                Record.Field(
-                    Value.Metadata([Value]),
-                    Config[Suite.MetaField]
-                )
-            otherwise
-                null
-        ) = Config[Suite.MetaValue]
+        Table.AddColumn(
+            Record.ToTable(Library),
+            "SuiteVersion",
+            each
+                try Text.From(
+                    Record.Field(
+                        Value.Metadata([Value]),
+                        Config[Suite.MetaField]
+                    )
+                ) otherwise
+                    null
+        ),
+        each [SuiteVersion] <> null
     ),
     Suites = Table.RenameColumns(
         SuitesFilter,
         List.Zip({
             Table.ColumnNames(SuitesFilter),
-            {"Suite", "Object"}
+            {"Suite", "Object", "Version"}
         })
     ),
-    Run = Table.AddColumn(Suites, "Results", each UnitTest.Run(null, [Object])),
+    SuiteRunners = Record.FromList(
+        List.Transform(
+            Record.FieldValues(Config[Suite.Runners]),
+            each LibPQ(_)
+        ),
+        Record.FieldNames(Config[Suite.Runners])
+    ),
+    Run = Table.AddColumn(
+        Suites,
+        "Results",
+        each Record.Field(SuiteRunners, [Version])([Object])
+    ),
     Expanded = Table.ExpandTableColumn(
-        Run,
+        Table.RemoveColumns(Run, "Version"),
         "Results",
         Table.ColumnNames(Run[Results]{0})
     ),

Modules/UnitTest.Fact.pq

@@ -0,0 +1,15 @@
+/**
+Helper function that packs arguments into a record.
+
+This is a part of compatibility layer between LibPQ and Microsoft's framework
+for unit testing in Power Query
+
+More information: https://docs.microsoft.com/en-us/power-query/handlingunittesting
+**/
+
+(test_name as text, expected_value, actual_value) =>
+[
+    name = test_name,
+    expected = expected_value,
+    actual = actual_value
+]

Modules/UnitTest.Facts.Summarize.pq

@@ -0,0 +1,28 @@
+/**
+Run tests represented as a list of facts.
+
+This is a part of compatibility layer between LibPQ and Microsoft's framework
+for unit testing in Power Query
+
+More information: https://docs.microsoft.com/en-us/power-query/handlingunittesting
+**/
+
+(facts as list) =>
+let
+    UnitTest.Run = LibPQ("UnitTest.Run"),
+    Assert = LibPQ("UnitTest.Assert"),
+    Config = LibPQ("UnitTest.Constants"),
+
+    Tests = List.Accumulate(
+        facts,
+        [],
+        (collection, fact) =>
+            Record.AddField(
+                collection,
+                Config[Test.Prefix] & " - " & fact[name],
+                Assert[Equal](fact[expected], fact[actual])
+            )
+    ),
+    Result = UnitTest.Run(Tests)
+in
+    Result

Modules/UnitTest.Run.pq

@@ -4,9 +4,9 @@ functions or may be referenced by its module name
 
 Return a table of test results
 **/
-(suite_name as nullable text, optional suite) =>
+(suite) =>
 let
-    Suite = if suite = null then LibPQ(suite_name) else suite,
+    Suite = if Value.Is(suite, type text) then LibPQ(suite) else suite,
 
     /* Load constants */
     Config = LibPQ("UnitTest.Constants"),

RELEASES.md

@@ -9,10 +9,11 @@ changelog](http://keepachangelog.com) is important!
 
 Plans and ideas for future versions can be found in the [roadmap](ROADMAP.md).
 
-<!--
 ## Unreleased changes (currently in git master)
--->
 
+**New features**
+
+- Added support for [fact based unit tests](Docs/UnitTesting_with_Facts.md)
 
 
 ## Version 1.1.0 (2018-07-11)