Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tasty_multiple_requests_accepted property test failures #1243

Open
dhess opened this issue Apr 17, 2024 · 2 comments
Open

tasty_multiple_requests_accepted property test failures #1243

dhess opened this issue Apr 17, 2024 · 2 comments
Assignees
Labels
blocked/need-info ❌ Blocked, need more information CI CI-related testing Related to tests/testing

Comments

@dhess
Copy link
Member

dhess commented Apr 17, 2024


Tests
--
  | Action
  | Available
  | multiple requests accepted:                       FAIL (41.78s)
  | ✗ multiple requests accepted failed at test/Tests/Action/Available.hs:481:42
  | after 473 tests, 6 shrinks and 1558 discards.
  | shrink path: 473/1558:aIhBsB
  | AddTm                                   36% ███████▏············
  | AddTy                                   23% ████▋···············
  | AvailAct                                68% █████████████▋······
  | Eval1                                    5% ▉···················
  | EvalBoundedInterp                       10% █▉··················
  | EvalFull                                10% ██··················
  | Question                                 5% ▉···················
  | Redo                                     1% ▏···················
  | RenameModule                            16% ███▎················
  | Undo                                    12% ██▍·················
  | Just (Input AddBranchInt)                0% ····················
  | Just (Input AddCon)                     13% ██▌·················
  | Just (Input AddTypeParam)                2% ▍···················
  | Just (Input MakeChar)                    1% ····················
  | Just (Input MakeCon)                     0% ····················
  | Just (Input MakeForall)                  1% ▏···················
  | Just (Input MakeLAM)                     2% ▍···················
  | Just (Input MakeLam)                     8% █▌··················
  | Just (Input MakeLet)                     1% ····················
  | Just (Input MakeLetRec)                  1% ····················
  | Just (Input MakeTCon)                    5% ▉···················
  | Just (Input MakeTVar)                    0% ····················
  | Just (Input MakeVar)                     1% ▏···················
  | Just (Input MakeVarSat)                  0% ····················
  | Just (Input RenameCon)                   2% ▍···················
  | Just (Input RenameDef)                   2% ▍···················
  | Just (Input RenameLam)                   1% ····················
  | Just (Input RenameLet)                   0% ····················
  | Just (Input RenameType)                  9% █▋··················
  | Just (Input RenameTypeParam)             2% ▎···················
  | Just (NoInput AddConField)               2% ▍···················
  | Just (NoInput AddInput)                  0% ····················
  | Just (NoInput DeleteCon)                 1% ▎···················
  | Just (NoInput DeleteConField)            1% ▏···················
  | Just (NoInput DeleteDef)                 2% ▍···················
  | Just (NoInput DeleteExpr)                6% █▏··················
  | Just (NoInput DeleteKind)                2% ▌···················
  | Just (NoInput DeleteType)                4% ▉···················
  | Just (NoInput DeleteTypeDef)            11% ██▎·················
  | Just (NoInput DuplicateDef)              2% ▍···················
  | Just (NoInput EnterHole)                 2% ▎···················
  | Just (NoInput LetToRec)                  0% ····················
  | Just (NoInput MakeAPP)                   1% ▏···················
  | Just (NoInput MakeAnn)                   1% ····················
  | Just (NoInput MakeApp)                   3% ▌···················
  | Just (NoInput MakeCase)                  3% ▌···················
  | Just (NoInput MakeFun)                   8% █▋··················
  | Just (NoInput MakeKFun)                  2% ▍···················
  | Just (NoInput MakeTApp)                  2% ▎···················
  | Just (NoInput Raise)                     3% ▋···················
  | Just (NoInput RaiseType)                 1% ····················
  | Just (NoInput RemoveAnn)                 1% ▏···················
  | Just (NoInput RemoveHole)                1% ····················
  | ignoring - clash                         1% ▎···················
  | name-capture with entered name           4% ▊···················
  | rename def name clash with entered name  1% ····················
  |  
  | ┏━━ test/Tests/Action/Available.hs ━━━
  | 336 ┃ tasty_multiple_requests_accepted :: Property
  | 337 ┃ tasty_multiple_requests_accepted = withTests 500
  | 338 ┃   $ withDiscards 2000
  | 339 ┃   $ propertyWT []
  | 340 ┃   $ do
  | 341 ┃     l <- forAllT $ Gen.element enumerate
  | ┃     │ Beginner
  | 342 ┃     cxt <- forAllT $ Gen.choice $ map sequence [[], [builtinModule], [builtinModule, primitiveModule]]
  | ┃     │ []
  | 343 ┃     -- We only test SmartHoles mode (which is the only supported student-facing
  | 344 ┃     -- mode - NoSmartHoles is only used for internal sanity testing etc)
  | 345 ┃     app0 <- forAllT $ genApp SmartHoles cxt
  | ┃     │ App
  | ┃     │   { currentState =
  | ┃     │       AppState
  | ┃     │         { idCounter = 12
  | ┃     │         , nameCounter = NC 312
  | ┃     │         , prog =
  | ┃     │             Prog
  | ┃     │               { progImports = []
  | ┃     │               , progModules =
  | ┃     │                   [ Module
  | ┃     │                       { moduleName = ModuleName { unModuleName = "M" :\| [ "0" ] }
  | ┃     │                       , moduleTypes =
  | ┃     │                           fromList
  | ┃     │                             [ ( "a"
  | ┃     │                               , TypeDefAST
  | ┃     │                                   ASTTypeDef
  | ┃     │                                     { astTypeDefParameters = []
  | ┃     │                                     , astTypeDefConstructors = []
  | ┃     │                                     , astTypeDefNameHints = []
  | ┃     │                                     }
  | ┃     │                               )
  | ┃     │                             , ( "a1"
  | ┃     │                               , TypeDefAST
  | ┃     │                                   ASTTypeDef
  | ┃     │                                     { astTypeDefParameters = []
  | ┃     │                                     , astTypeDefConstructors =
  | ┃     │                                         [ ValCon
  | ┃     │                                             { valConName =
  | ┃     │                                                 GlobalName
  | ┃     │                                                   { qualifiedModule =
  | ┃     │                                                       ModuleName { unModuleName = "M" :\| [ "0" ] }
  | ┃     │                                                   , baseName = "a2"
  | ┃     │                                                   }
  | ┃     │                                             , valConArgs = []
  | ┃     │                                             }
  | ┃     │                                         ]
  | ┃     │                                     , astTypeDefNameHints = []
  | ┃     │                                     }
  | ┃     │                               )
  | ┃     │                             ]
  | ┃     │                       , moduleDefs =
  | ┃     │                           fromList
  | ┃     │                             [ ( "a3"
  | ┃     │                               , DefAST
  | ┃     │                                   ASTDef
  | ┃     │                                     { astDefExpr =
  | ┃     │                                         Hole
  | ┃     │                                           (Meta
  | ┃     │                                              11
  | ┃     │                                              (Just
  | ┃     │                                                 (TCEmb
  | ┃     │                                                    TCBoth
  | ┃     │                                                      { tcChkedAt =
  | ┃     │                                                          TApp () (TEmptyHole ()) (TEmptyHole ())
  | ┃     │                                                      , tcSynthed = TEmptyHole ()
  | ┃     │                                                      }))
  | ┃     │                                              Nothing)
  | ┃     │                                           (Con
  | ┃     │                                              (Meta
  | ┃     │                                                 5
  | ┃     │                                                 (Just
  | ┃     │                                                    (TCChkedAt
  | ┃     │                                                       (TCon
  | ┃     │                                                          ()
  | ┃     │                                                          GlobalName
  | ┃     │                                                            { qualifiedModule =
  | ┃     │                                                                ModuleName
  | ┃     │                                                                  { unModuleName = "M" :\| [ "0" ] }
  | ┃     │                                                            , baseName = "a1"
  | ┃     │                                                            })))
  | ┃     │                                                 Nothing)
  | ┃     │                                              GlobalName
  | ┃     │                                                { qualifiedModule =
  | ┃     │                                                    ModuleName { unModuleName = "M" :\| [ "0" ] }
  | ┃     │                                                , baseName = "a2"
  | ┃     │                                                }
  | ┃     │                                              [])
  | ┃     │                                     , astDefType =
  | ┃     │                                         TApp
  | ┃     │                                           (Meta 8 (Just (KHole ())) Nothing)
  | ┃     │                                           (TEmptyHole (Meta 9 (Just (KHole ())) Nothing))
  | ┃     │                                           (TEmptyHole (Meta 10 (Just (KHole ())) Nothing))
  | ┃     │                                     }
  | ┃     │                               )
  | ┃     │                             ]
  | ┃     │                       }
  | ┃     │                   ]
  | ┃     │               , progSelection = Nothing
  | ┃     │               , progSmartHoles = SmartHoles
  | ┃     │               , progLog = Log { unlog = [] }
  | ┃     │               , redoLog = Log { unlog = [] }
  | ┃     │               }
  | ┃     │         }
  | ┃     │   , initialState =
  | ┃     │       AppState
  | ┃     │         { idCounter = 12
  | ┃     │         , nameCounter = NC 312
  | ┃     │         , prog =
  | ┃     │             Prog
  | ┃     │               { progImports = []
  | ┃     │               , progModules =
  | ┃     │                   [ Module
  | ┃     │                       { moduleName = ModuleName { unModuleName = "M" :\| [ "0" ] }
  | ┃     │                       , moduleTypes =
  | ┃     │                           fromList
  | ┃     │                             [ ( "a"
  | ┃     │                               , TypeDefAST
  | ┃     │                                   ASTTypeDef
  | ┃     │                                     { astTypeDefParameters = []
  | ┃     │                                     , astTypeDefConstructors = []
  | ┃     │                                     , astTypeDefNameHints = []
  | ┃     │                                     }
  | ┃     │                               )
  | ┃     │                             , ( "a1"
  | ┃     │                               , TypeDefAST
  | ┃     │                                   ASTTypeDef
  | ┃     │                                     { astTypeDefParameters = []
  | ┃     │                                     , astTypeDefConstructors =
  | ┃     │                                         [ ValCon
  | ┃     │                                             { valConName =
  | ┃     │                                                 GlobalName
  | ┃     │                                                   { qualifiedModule =
  | ┃     │                                                       ModuleName { unModuleName = "M" :\| [ "0" ] }
  | ┃     │                                                   , baseName = "a2"
  | ┃     │                                                   }
  | ┃     │                                             , valConArgs = []
  | ┃     │                                             }
  | ┃     │                                         ]
  | ┃     │                                     , astTypeDefNameHints = []
  | ┃     │                                     }
  | ┃     │                               )
  | ┃     │                             ]
  | ┃     │                       , moduleDefs =
  | ┃     │                           fromList
  | ┃     │                             [ ( "a3"
  | ┃     │                               , DefAST
  | ┃     │                                   ASTDef
  | ┃     │                                     { astDefExpr =
  | ┃     │                                         Hole
  | ┃     │                                           (Meta
  | ┃     │                                              11
  | ┃     │                                              (Just
  | ┃     │                                                 (TCEmb
  | ┃     │                                                    TCBoth
  | ┃     │                                                      { tcChkedAt =
  | ┃     │                                                          TApp () (TEmptyHole ()) (TEmptyHole ())
  | ┃     │                                                      , tcSynthed = TEmptyHole ()
  | ┃     │                                                      }))
  | ┃     │                                              Nothing)
  | ┃     │                                           (Con
  | ┃     │                                              (Meta
  | ┃     │                                                 5
  | ┃     │                                                 (Just
  | ┃     │                                                    (TCChkedAt
  | ┃     │                                                       (TCon
  | ┃     │                                                          ()
  | ┃     │                                                          GlobalName
  | ┃     │                                                            { qualifiedModule =
  | ┃     │                                                                ModuleName
  | ┃     │                                                                  { unModuleName = "M" :\| [ "0" ] }
  | ┃     │                                                            , baseName = "a1"
  | ┃     │                                                            })))
  | ┃     │                                                 Nothing)
  | ┃     │                                              GlobalName
  | ┃     │                                                { qualifiedModule =
  | ┃     │                                                    ModuleName { unModuleName = "M" :\| [ "0" ] }
  | ┃     │                                                , baseName = "a2"
  | ┃     │                                                }
  | ┃     │                                              [])
  | ┃     │                                     , astDefType =
  | ┃     │                                         TApp
  | ┃     │                                           (Meta 8 (Just (KHole ())) Nothing)
  | ┃     │                                           (TEmptyHole (Meta 9 (Just (KHole ())) Nothing))
  | ┃     │                                           (TEmptyHole (Meta 10 (Just (KHole ())) Nothing))
  | ┃     │                                     }
  | ┃     │                               )
  | ┃     │                             ]
  | ┃     │                       }
  | ┃     │                   ]
  | ┃     │               , progSelection = Nothing
  | ┃     │               , progSmartHoles = SmartHoles
  | ┃     │               , progLog = Log { unlog = [] }
  | ┃     │               , redoLog = Log { unlog = [] }
  | ┃     │               }
  | ┃     │         }
  | ┃     │   }
  | 346 ┃     numActions <- forAllT $ Gen.int $ Range.linear 1 20
  | ┃     │ 3
  | 347 ┃     let appDefs = foldMap' (M.keys . moduleDefsQualified) . progModules . appProg
  | 348 ┃         genAction' a' =
  | 349 ┃           Gen.frequency
  | 350 ┃             $ second pure
  | 351 ┃             <$> catMaybes
  | 352 ┃               [ Just (3, AddTm)
  | 353 ┃               , Just (2, AddTy)
  | 354 ┃               , Just (1, Eval1)
  | 355 ┃               , if null $ appDefs a' then Nothing else Just (1, EvalFull)
  | 356 ┃               , Just (1, Question)
  | 357 ┃               , if null $ appDefs a' then Nothing else Just (1, EvalBoundedInterp)
  | 358 ┃               , if undoLogEmpty $ appProg a' then Nothing else Just (2, Undo)
  | 359 ┃               , if redoLogEmpty $ appProg a' then Nothing else Just (2, Redo)
  | 360 ┃               , Just (1, RenameModule)
  | 361 ┃               , Just (10, AvailAct)
  | 362 ┃               ]
  | 363 ┃     void $ iterateNM numActions app0 $ \appN ->
  | 364 ┃       forAllT (genAction' appN) >>= \a -> do
  | ┃       │ AvailAct
  | ┃       │ AddTy
  | ┃       │ AvailAct
  | 365 ┃         collect a
  | 366 ┃         case a of
  | 367 ┃           AddTm -> do
  | 368 ┃             m <- forAllT $ Gen.element $ fmap moduleName $ progModules $ appProg appN
  | 369 ┃             n <- forAllT $ Gen.choice [Just . unName <$> genName, pure Nothing]
  | 370 ┃             actionSucceedsOrCapture (handleMutationRequest $ Edit [CreateDef m n]) appN
  | 371 ┃               >>= ignoreCaptureClash appN
  | 372 ┃           AddTy -> do
  | 373 ┃             m <- forAllT $ Gen.element $ fmap moduleName $ progModules $ appProg appN
  | ┃             │ ModuleName { unModuleName = "M" :\| [ "0" ] }
  | 374 ┃             n <- qualifyName m <$> forAllT genName
  | ┃             │ "x"
  | 375 ┃             actionSucceedsOrCapture (handleMutationRequest $ Edit [AddTypeDef n $ ASTTypeDef [] [] []]) appN
  | 376 ┃               >>= ignoreCaptureClash appN
  | 377 ┃           Eval1 -> do
  | 378 ┃             (e', _) <- forAllT genSyn
  | 379 ┃             e <- generateIDs e'
  | 380 ┃             i <- forAllT $ Gen.element $ e ^.. exprIDs
  | 381 ┃             actionSucceedsOrNotRedex (readerToState $ handleEvalRequest $ EvalReq e i) appN
  | 382 ┃           EvalFull -> do
  | 383 ┃             g <- forAllT $ Gen.element $ appDefs appN
  | 384 ┃             tld <- gvar g
  | 385 ┃             steps <- forAllT $ Gen.integral $ Range.linear 0 100
  | 386 ┃             optsN <- forAllT $ Gen.element @[] [StopAtBinders, UnderBinders]
  | 387 ┃             actionSucceeds (readerToState $ handleEvalFullRequest $ EvalFullReq tld Chk steps optsN) appN
  | 388 ┃           EvalBoundedInterp -> do
  | 389 ┃             g <- forAllT $ Gen.element $ appDefs appN
  | 390 ┃             tld <- gvar g
  | 391 ┃             usec <- forAllT $ Gen.integral $ Range.linear 0 1000
  | 392 ┃             actionSucceeds (readerToState $ handleEvalBoundedInterpRequest $ EvalBoundedInterpReq tld Chk (MicroSec usec)) appN
  | 393 ┃           Question -> do
  | 394 ┃             -- Obtain a non-exhaustive case warning if we add a new question
  | 395 ┃             let _w :: Question q -> ()
  | 396 ┃                 _w = \case
  | 397 ┃                   VariablesInScope{} -> ()
  | 398 ┃                   GenerateName{} -> ()
  | 399 ┃             (_, e, w) <- forAllT $ genLoc appN
  | 400 ┃             -- We only support questions in editable modules
  | 401 ┃             case e of
  | 402 ┃               Editable -> pure ()
  | 403 ┃               NonEditable -> discard
  | 404 ┃             (def, node) <- case w of
  | 405 ┃               SelectionDef (DefSelection{def, node = Just node}) -> pure (def, node.meta)
  | 406 ┃               SelectionDef (DefSelection{node = Nothing}) -> discard
  | 407 ┃               -- We don't currently support questions on typedefs
  | 408 ┃               SelectionTypeDef _ -> discard
  | 409 ┃             (_, q) <-
  | 410 ┃               forAllWithT fst
  | 411 ┃                 $ Gen.choice
  | 412 ┃                   [ pure ("VariablesInScope", void $ handleQuestion (VariablesInScope def node))
  | 413 ┃                   , ("GenerateName",) . void . handleQuestion . GenerateName def node <$> do
  | 414 ┃                       k <- genWTKind
  | 415 ┃                       Gen.choice
  | 416 ┃                         [ Left <$> Gen.maybe (genWTType k)
  | 417 ┃                         , pure $ Right $ Just k
  | 418 ┃                         , pure $ Right Nothing
  | 419 ┃                         ]
  | 420 ┃                   ]
  | 421 ┃             actionSucceeds (readerToState q) appN
  | 422 ┃           Undo -> actionSucceeds (handleMutationRequest App.Undo) appN
  | 423 ┃           Redo -> actionSucceeds (handleMutationRequest App.Redo) appN
  | 424 ┃           AvailAct -> do
  | 425 ┃             (def'', loc, action) <- forAllT $ genAction l appN
  | ┃             │ ( Left
  | ┃             │     ( GlobalName
  | ┃             │         { qualifiedModule = ModuleName { unModuleName = "M" :\| [ "0" ] }
  | ┃             │         , baseName = "a1"
  | ┃             │         }
  | ┃             │     , ASTTypeDef
  | ┃             │         { astTypeDefParameters = []
  | ┃             │         , astTypeDefConstructors =
  | ┃             │             [ ValCon
  | ┃             │                 { valConName =
  | ┃             │                     GlobalName
  | ┃             │                       { qualifiedModule = ModuleName { unModuleName = "M" :\| [ "0" ] }
  | ┃             │                       , baseName = "a2"
  | ┃             │                       }
  | ┃             │                 , valConArgs = []
  | ┃             │                 }
  | ┃             │             ]
  | ┃             │         , astTypeDefNameHints = []
  | ┃             │         }
  | ┃             │     )
  | ┃             │ , SelectionTypeDef
  | ┃             │     TypeDefSelection
  | ┃             │       { def =
  | ┃             │           GlobalName
  | ┃             │             { qualifiedModule = ModuleName { unModuleName = "M" :\| [ "0" ] }
  | ┃             │             , baseName = "a1"
  | ┃             │             }
  | ┃             │       , node =
  | ┃             │           Just
  | ┃             │             (TypeDefConsNodeSelection
  | ┃             │                TypeDefConsSelection
  | ┃             │                  { con =
  | ┃             │                      GlobalName
  | ┃             │                        { qualifiedModule = ModuleName { unModuleName = "M" :\| [ "0" ] }
  | ┃             │                        , baseName = "a2"
  | ┃             │                        }
  | ┃             │                  , field = Nothing
  | ┃             │                  })
  | ┃             │       }
  | ┃             │ , Just (NoInput DeleteCon)
  | ┃             │ )
  | ┃             │ ( Right
  | ┃             │     ( GlobalName
  | ┃             │         { qualifiedModule = ModuleName { unModuleName = "M" :\| [ "0" ] }
  | ┃             │         , baseName = "a3"
  | ┃             │         }
  | ┃             │     , DefAST
  | ┃             │         ASTDef
  | ┃             │           { astDefExpr =
  | ┃             │               Hole
  | ┃             │                 (Meta
  | ┃             │                    11
  | ┃             │                    (Just
  | ┃             │                       (TCEmb
  | ┃             │                          TCBoth
  | ┃             │                            { tcChkedAt = TApp () (TEmptyHole ()) (TEmptyHole ())
  | ┃             │                            , tcSynthed = TEmptyHole ()
  | ┃             │                            }))
  | ┃             │                    Nothing)
  | ┃             │                 (EmptyHole (Meta 12 Nothing Nothing))
  | ┃             │           , astDefType =
  | ┃             │               TApp
  | ┃             │                 (Meta 8 (Just (KHole ())) Nothing)
  | ┃             │                 (TEmptyHole (Meta 9 (Just (KHole ())) Nothing))
  | ┃             │                 (TEmptyHole (Meta 10 (Just (KHole ())) Nothing))
  | ┃             │           }
  | ┃             │     )
  | ┃             │ , SelectionDef
  | ┃             │     DefSelection
  | ┃             │       { def =
  | ┃             │           GlobalName
  | ┃             │             { qualifiedModule = ModuleName { unModuleName = "M" :\| [ "0" ] }
  | ┃             │             , baseName = "a3"
  | ┃             │             }
  | ┃             │       , node = Nothing
  | ┃             │       }
  | ┃             │ , Just (NoInput DuplicateDef)
  | ┃             │ )
  | 426 ┃             annotateShow def''
  | ┃             │ Left
  | ┃             │   ( GlobalName
  | ┃             │       { qualifiedModule = ModuleName { unModuleName = "M" :\| [ "0" ] }
  | ┃             │       , baseName = "a1"
  | ┃             │       }
  | ┃             │   , ASTTypeDef
  | ┃             │       { astTypeDefParameters = []
  | ┃             │       , astTypeDefConstructors =
  | ┃             │           [ ValCon
  | ┃             │               { valConName =
  | ┃             │                   GlobalName
  | ┃             │                     { qualifiedModule = ModuleName { unModuleName = "M" :\| [ "0" ] }
  | ┃             │                     , baseName = "a2"
  | ┃             │                     }
  | ┃             │               , valConArgs = []
  | ┃             │               }
  | ┃             │           ]
  | ┃             │       , astTypeDefNameHints = []
  | ┃             │       }
  | ┃             │   )
  | ┃             │ Right
  | ┃             │   ( GlobalName
  | ┃             │       { qualifiedModule = ModuleName { unModuleName = "M" :\| [ "0" ] }
  | ┃             │       , baseName = "a3"
  | ┃             │       }
  | ┃             │   , DefAST
  | ┃             │       ASTDef
  | ┃             │         { astDefExpr =
  | ┃             │             Hole
  | ┃             │               (Meta
  | ┃             │                  11
  | ┃             │                  (Just
  | ┃             │                     (TCEmb
  | ┃             │                        TCBoth
  | ┃             │                          { tcChkedAt = TApp () (TEmptyHole ()) (TEmptyHole ())
  | ┃             │                          , tcSynthed = TEmptyHole ()
  | ┃             │                          }))
  | ┃             │                  Nothing)
  | ┃             │               (EmptyHole (Meta 12 Nothing Nothing))
  | ┃             │         , astDefType =
  | ┃             │             TApp
  | ┃             │               (Meta 8 (Just (KHole ())) Nothing)
  | ┃             │               (TEmptyHole (Meta 9 (Just (KHole ())) Nothing))
  | ┃             │               (TEmptyHole (Meta 10 (Just (KHole ())) Nothing))
  | ┃             │         }
  | ┃             │   )
  | 427 ┃             annotateShow loc
  | ┃             │ SelectionTypeDef
  | ┃             │   TypeDefSelection
  | ┃             │     { def =
  | ┃             │         GlobalName
  | ┃             │           { qualifiedModule = ModuleName { unModuleName = "M" :\| [ "0" ] }
  | ┃             │           , baseName = "a1"
  | ┃             │           }
  | ┃             │     , node =
  | ┃             │         Just
  | ┃             │           (TypeDefConsNodeSelection
  | ┃             │              TypeDefConsSelection
  | ┃             │                { con =
  | ┃             │                    GlobalName
  | ┃             │                      { qualifiedModule = ModuleName { unModuleName = "M" :\| [ "0" ] }
  | ┃             │                      , baseName = "a2"
  | ┃             │                      }
  | ┃             │                , field = Nothing
  | ┃             │                })
  | ┃             │     }
  | ┃             │ SelectionDef
  | ┃             │   DefSelection
  | ┃             │     { def =
  | ┃             │         GlobalName
  | ┃             │           { qualifiedModule = ModuleName { unModuleName = "M" :\| [ "0" ] }
  | ┃             │           , baseName = "a3"
  | ┃             │           }
  | ┃             │     , node = Nothing
  | ┃             │     }
  | 428 ┃             annotateShow action
  | ┃             │ Just (NoInput DeleteCon)
  | ┃             │ Just (NoInput DuplicateDef)
  | 429 ┃             let def = bimap snd snd def''
  | 430 ┃             collect action
  | 431 ┃             pa <- toProgAction l appN (def, loc, action)
  | 432 ┃             case pa of
  | 433 ┃               NoOpt progActs -> actionSucceeds (handleEditRequest progActs) appN
  | 434 ┃               OptOffered _ progActs -> actionSucceeds (handleEditRequest progActs) appN
  | 435 ┃               OptGen _ progActs ->
  | 436 ┃                 actionSucceedsOrCapture (handleEditRequest progActs) appN
  | 437 ┃                   >>= ignoreCaptureClash appN
  | 438 ┃               NoOfferedOpts -> annotate "ignoring - no options" >> pure appN
  | 439 ┃           RenameModule -> do
  | 440 ┃             m <- forAllT $ moduleName <$> Gen.element (progModules $ appProg appN)
  | 441 ┃             n <- forAllT (Gen.nonEmpty (Range.linear 1 5) genName)
  | 442 ┃             actionSucceedsOrCapture (handleMutationRequest $ Edit [App.RenameModule m $ unName <$> n]) appN >>= ignoreCaptureClash appN
  | 443 ┃   where
  | 444 ┃     ignoreCaptureClash a = \case
  | 445 ┃       Succeed b -> pure b
  | 446 ┃       Capture -> label "ignoring - capture" >> pure a
  | 447 ┃       Clash -> label "ignoring - clash" >> pure a
  | 448 ┃     iterateNM :: Monad m => Int -> a -> (a -> m a) -> m a
  | 449 ┃     iterateNM n a f
  | 450 ┃       \| n <= 0 = pure a
  | 451 ┃       \| otherwise = f a >>= \fa -> iterateNM (n - 1) fa f
  |  
  | ┏━━ test/Tests/Action/Available.hs ━━━
  | 478 ┃ actionSucceeds :: HasCallStack => EditAppM (PureLogT (WithSeverity ()) WT) ProgError a -> App -> PropertyT WT App
  | 479 ┃ actionSucceeds m a =
  | 480 ┃   runEditAppMLogs m a >>= \case
  | 481 ┃     (Left err, _) -> annotateShow err >> failure
  | ┃     │ ActionError (IDNotFound 11)
  | ┃     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  | 482 ┃     (Right _, a') -> ensureSHNormal a' $> a'
  |  
  | This failure can be reproduced by running:
  | > recheckAt (Seed 2724545197382348245 15980768592615361839) "473/1558:aIhBsB" multiple requests accepted
  |  
  | Use "--pattern '$NF ~ /multiple requests accepted/' --hedgehog-replay '473/1558:aIhBsB Seed 2724545197382348245 15980768592615361839'" to reproduce from the command-line.
  |  
  | Use -p '/multiple requests accepted/' to rerun this test only.
  |  
  | 1 out of 862 tests failed (76.47s)


Ref:

Logs:

@dhess dhess added testing Related to tests/testing blocked/need-info ❌ Blocked, need more information CI CI-related labels Apr 17, 2024
@dhess dhess self-assigned this Apr 17, 2024
@dhess
Copy link
Member Author

dhess commented Apr 17, 2024

Also this one:


Tests
--
  | EvalFullInterp
  | two interp agree:                                   FAIL (11.74s)
  | ✗ two interp agree failed at test/Tests/EvalFullInterp.hs:467:60
  | after 725 tests, 5 shrinks and 623 discards.
  | shrink path: 725/623:cA2cJ
  | both terminated         99% ███████████████████▉
  | one failed to terminate  0% ····················
  |  
  | ┏━━ test/Tests/Eval/Utils.hs ━━━
  | 51 ┃ genDirTm :: PropertyT WT (Dir, Expr, Type' () ())
  | 52 ┃ genDirTm = do
  | 53 ┃   dir <- forAllT $ Gen.element @[] [Chk, Syn]
  | ┃   │ Chk
  | 54 ┃   (t', ty) <- case dir of
  | 55 ┃     Chk -> do
  | 56 ┃       ty' <- forAllT $ genWTType $ KType ()
  | ┃       │ TApp () (TEmptyHole ()) (TEmptyHole ())
  | 57 ┃       t' <- forAllT $ genChk ty'
  | ┃       │ Letrec
  | ┃       │   ()
  | ┃       │   LocalName { unLocalName = "y" }
  | ┃       │   (EmptyHole ())
  | ┃       │   (TEmptyHole ())
  | ┃       │   (Let
  | ┃       │      ()
  | ┃       │      LocalName { unLocalName = "y" }
  | ┃       │      (EmptyHole ())
  | ┃       │      (Hole
  | ┃       │         ()
  | ┃       │         (Case
  | ┃       │            ()
  | ┃       │            (Ann
  | ┃       │               ()
  | ┃       │               (EmptyHole ())
  | ┃       │               (TCon
  | ┃       │                  ()
  | ┃       │                  GlobalName
  | ┃       │                    { qualifiedModule =
  | ┃       │                        ModuleName { unModuleName = "Primitives" :\| [] }
  | ┃       │                    , baseName = "Char"
  | ┃       │                    }))
  | ┃       │            []
  | ┃       │            (CaseFallback
  | ┃       │               (Var
  | ┃       │                  ()
  | ┃       │                  (GlobalVarRef
  | ┃       │                     GlobalName
  | ┃       │                       { qualifiedModule = ModuleName { unModuleName = "M" :\| [] }
  | ┃       │                       , baseName = "idChar"
  | ┃       │                       }))))))
  | 58 ┃       pure (t', ty')
  | 59 ┃     Syn -> forAllT genSyn
  | 60 ┃   t <- generateIDs t'
  | 61 ┃   pure (dir, t, ty)
  |  
  | ┏━━ test/Tests/EvalFullInterp.hs ━━━
  | 454 ┃ tasty_two_interp_agree :: Property
  | 455 ┃ tasty_two_interp_agree = withTests 1000
  | 456 ┃   $ withDiscards 2000
  | 457 ┃   $ propertyWT testModules
  | 458 ┃   $ do
  | 459 ┃     let globs = foldMap' moduleDefsQualified $ create' $ sequence testModules
  | 460 ┃     tds <- asks typeDefs
  | 461 ┃     (dir, t, _ty) <- genDirTm
  | 462 ┃     let optsV = ViewRedexOptions{groupedLets = True, aggressiveElision = True, avoidShadowing = False}
  | 463 ┃     let optsR = RunRedexOptions{pushAndElide = True}
  | 464 ┃     (_, ss) <- failWhenSevereLogs $ evalFullStepCount @EvalLog UnderBinders optsV optsR tds globs 100 dir t
  | 465 ┃     si <- liftIO (evalFullTest' (MicroSec 10_000) tds globs dir $ forgetMetadata t)
  | 466 ┃     case (ss, si) of
  | 467 ┃       (Right ss', Right si') -> label "both terminated" >> Hedgehog.diff (forgetMetadata ss') alphaEq si'
  | ┃       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  | ┃       │ ━━━ Failed (- lhs) (+ rhs) ━━━
  | ┃       │   Hole
  | ┃       │     ()
  | ┃       │ -   Ann
  | ┃       │ -     ()
  | ┃       │ -     (Lam
  | ┃       │ -        ()
  | ┃       │ -        LocalName { unLocalName = "x" }
  | ┃       │ -        (Var () (LocalVarRef LocalName { unLocalName = "x" })))
  | ┃       │ -     (TFun
  | ┃       │ -        ()
  | ┃       │ -        (TCon
  | ┃       │ -           ()
  | ┃       │ -           GlobalName
  | ┃       │ -             { qualifiedModule =
  | ┃       │ -                 ModuleName { unModuleName = "Primitives" :\| [] }
  | ┃       │ -             , baseName = "Char"
  | ┃       │ -             })
  | ┃       │ -        (TCon
  | ┃       │ -           ()
  | ┃       │ -           GlobalName
  | ┃       │ -             { qualifiedModule =
  | ┃       │ -                 ModuleName { unModuleName = "Primitives" :\| [] }
  | ┃       │ -             , baseName = "Char"
  | ┃       │ -             }))
  | ┃       │ +   Lam
  | ┃       │ +     ()
  | ┃       │ +     LocalName { unLocalName = "x" }
  | ┃       │ +     (Var () (LocalVarRef LocalName { unLocalName = "x" }))
  | 468 ┃       _ -> label "one failed to terminate"
  |  
  | This failure can be reproduced by running:
  | > recheckAt (Seed 267242377111036614 4435999987528114339) "725/623:cA2cJ" two interp agree
  |  
  | Use "--pattern '$NF ~ /two interp agree/' --hedgehog-replay '725/623:cA2cJ Seed 267242377111036614 4435999987528114339'" to reproduce from the command-line.
  |  
  | Use -p '/two interp agree/' to rerun this test only.
  |  
  | 1 out of 862 tests failed (91.94s)


Ref:

Logs: primer-ci_build_65_linux-x86-64-linux-primer-test-primer-test-0-dot-7-2-dot-0-check.log

@dhess
Copy link
Member Author

dhess commented Jun 2, 2024

Another:

https://buildkite.com/hackworthltd/primer-ci/builds/113#018fdaf9-1d16-4455-b88a-aac16645e4ff/16-33

             This failure can be reproduced by running:
              > recheckAt (Seed 12448919876317083484 8517633342112724275) "88/249:cDeYbA2" multiple requests accepted
          Use "--pattern '$NF ~ /multiple requests accepted/' --hedgehog-replay '88/249:cDeYbA2 Seed 12448919876317083484 8517633342112724275'" to reproduce from the command-line.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked/need-info ❌ Blocked, need more information CI CI-related testing Related to tests/testing
Projects
None yet
Development

No branches or pull requests

1 participant