Low test coverage of normalizeWithM

[sjakobi]

If I wrap the entire implementation of Dhall.Core.normalizeWithM with an undefined, only 8 tests fail:

Dhall Tests
  normalization
    customization
      simpleCustomization:                                                                      FAIL
        Exception: Prelude.undefined
        CallStack (from HasCallStack):
          error, called at libraries/base/GHC/Err.hs:78:14 in base:GHC.Err
          undefined, called at src/Dhall/Core.hs:1297:25 in dhall-1.24.0-Hhvu3RMIrBsIeoMyZcG2FL:Dhall.Core
      doubleReduction:                                                                          FAIL
        Exception: Prelude.undefined
        CallStack (from HasCallStack):
          error, called at libraries/base/GHC/Err.hs:78:14 in base:GHC.Err
          undefined, called at src/Dhall/Core.hs:1297:25 in dhall-1.24.0-Hhvu3RMIrBsIeoMyZcG2FL:Dhall.Core
  tutorial
    Functions
      Example #0:                                                                               FAIL
        Exception: Prelude.undefined
        CallStack (from HasCallStack):
          error, called at libraries/base/GHC/Err.hs:78:14 in base:GHC.Err
          undefined, called at src/Dhall/Core.hs:1297:25 in dhall-1.24.0-Hhvu3RMIrBsIeoMyZcG2FL:Dhall.Core
      Example #1:                                                                               FAIL
        Exception: Prelude.undefined
        CallStack (from HasCallStack):
          error, called at libraries/base/GHC/Err.hs:78:14 in base:GHC.Err
          undefined, called at src/Dhall/Core.hs:1297:25 in dhall-1.24.0-Hhvu3RMIrBsIeoMyZcG2FL:Dhall.Core
      Example #2:                                                                               FAIL
        Exception: Prelude.undefined
        CallStack (from HasCallStack):
          error, called at libraries/base/GHC/Err.hs:78:14 in base:GHC.Err
          undefined, called at src/Dhall/Core.hs:1297:25 in dhall-1.24.0-Hhvu3RMIrBsIeoMyZcG2FL:Dhall.Core
  Input
    Handle union literals
      Pass through:                                                                             FAIL
        Exception: Prelude.undefined
        CallStack (from HasCallStack):
          error, called at libraries/base/GHC/Err.hs:78:14 in base:GHC.Err
          undefined, called at src/Dhall/Core.hs:1297:25 in dhall-1.24.0-Hhvu3RMIrBsIeoMyZcG2FL:Dhall.Core
      Pass through:                                                                             FAIL
        Exception: Prelude.undefined
        CallStack (from HasCallStack):
          error, called at libraries/base/GHC/Err.hs:78:14 in base:GHC.Err
          undefined, called at src/Dhall/Core.hs:1297:25 in dhall-1.24.0-Hhvu3RMIrBsIeoMyZcG2FL:Dhall.Core
      Pass through:                                                                             FAIL
        Exception: Prelude.undefined
        CallStack (from HasCallStack):
          error, called at libraries/base/GHC/Err.hs:78:14 in base:GHC.Err
          undefined, called at src/Dhall/Core.hs:1297:25 in dhall-1.24.0-Hhvu3RMIrBsIeoMyZcG2FL:Dhall.Core

8 out of 1200 tests failed (4.75s)

If I insert undefineds in just some of the Expr alternatives, the testsuite isn't affected at all.

Given that we have some fairly complex logic in normalizeWithM, I wonder whether some of it is already broken.

To increase test coverage, could we simply run the normalization tests through normalizeWithM too? The "trivial" Normalizer appears to be simply (Identity . Just).

A assume we cannot simply get rid of normalizeWithM because someone's actually using it?!

[sjakobi]

The "trivial" Normalizer appears to be simply (Identity . Just).

Ah, no. It's probably something like \_e -> Identity Nothing.

[Gabriella439]

@sjakobi: We really only need one test, which is just a property test that ensures that normalizeWithM behaves the same as normalize when using the Identity Monad