Skip to content

Commit

Permalink
log server errors as critical log messages when an exception is raise…
Browse files Browse the repository at this point in the history
…d by

  It took us long enough to get to identify the source of the failure because the wallet is silently raising an exception and terminating nodes of the cluster, but there are some ghost threads which remain hanging and prevent other bracket functions to fully move forward and stop.
  At least now, an error message is printed to help debugging the root cause of the issue.
  • Loading branch information
KtorZ committed Dec 3, 2020
1 parent 5da7d5a commit ec173c5
Showing 1 changed file with 9 additions and 4 deletions.
13 changes: 9 additions & 4 deletions lib/shelley/test/integration/Main.hs
Expand Up @@ -74,7 +74,7 @@ import Control.Concurrent.Async
import Control.Concurrent.MVar
( newEmptyMVar, putMVar, takeMVar )
import Control.Exception
( throwIO )
( SomeException, handle, throwIO )
import Control.Monad.IO.Class
( liftIO )
import Control.Tracer
Expand Down Expand Up @@ -275,12 +275,12 @@ specWithServer (tr, tracers) = aroundAll withContext
concatMap genRewardAccounts mirMnemonics
moveInstantaneousRewardsTo tr' dir rewards

onClusterStart
action dir dbDecorator (RunningNode socketPath block0 (gp, vData)) = do
onClusterStart action dir dbDecorator node = do
-- NOTE: We may want to keep a wallet running across the fork, but
-- having three callbacks like this might not work well for that.
withTempDir tr' dir "wallets" $ \db -> do
serveWallet @(IO ShelleyEra)
handle (\e -> traceWith tr (MsgServerError e) >> throwIO e)
$ serveWallet @(IO ShelleyEra)
(SomeNetworkDiscriminant $ Proxy @'Mainnet)
tracers
(SyncTolerance 10)
Expand All @@ -294,6 +294,8 @@ specWithServer (tr, tracers) = aroundAll withContext
block0
(gp, vData)
(action gp)
where
RunningNode socketPath block0 (gp, vData) = node

{-------------------------------------------------------------------------------
Logging
Expand All @@ -305,6 +307,7 @@ data TestsLog
| MsgSettingUpFaucet
| MsgCluster ClusterLog
| MsgPoolGarbageCollectionEvent PoolGarbageCollectionEvent
| MsgServerError SomeException
deriving (Show)

instance ToText TestsLog where
Expand All @@ -324,6 +327,7 @@ instance ToText TestsLog where
, T.unwords (T.pack . show <$> ps)
]
]
MsgServerError e -> T.pack (show e)

instance HasPrivacyAnnotation TestsLog
instance HasSeverityAnnotation TestsLog where
Expand All @@ -333,6 +337,7 @@ instance HasSeverityAnnotation TestsLog where
MsgBaseUrl _ -> Notice
MsgCluster msg -> getSeverityAnnotation msg
MsgPoolGarbageCollectionEvent _ -> Info
MsgServerError{} -> Critical

withTracers
:: ((Tracer IO TestsLog, Tracers IO) -> IO a)
Expand Down

0 comments on commit ec173c5

Please sign in to comment.