Improvements to slotting conversions / TimeInterpreter #1943

Anviking · 2020-07-23T14:34:21Z

Issue Number

#1869 / #1960

Overview

Block instead of using a singleEraInterpreter if we don't yet have a proper TimeInterpreter.
Remove TODO; explain why delegation time conversion never should fail
Replace TimeInterpreter IO with TimeInterpreter (ExceptT IO ErrPastHorizon)
Add, test and use endTimeOfEpoch e instead of timeOf =<< firstSlotInEpoch (e+1). This is needed to stay inside the forecast range when showing next_epoch when the node is in-sync, in Byron. Fixes GET http://localhost:8090/v2/network/information returns 500 on mainnet while wallet is syncing through Byron era. #1960
Make next_epoch and network_tip optional in the API. When the node is still syncing the Byron chain, it cannot know any slotting info close to the current time. Solving related problem to GET http://localhost:8090/v2/network/information returns 500 on mainnet while wallet is syncing through Byron era. #1960
Revise the names of the constructors of TimeInterpreter

Comments

I thought queries in Shelley could fail. This is wrong. Once in Shelley (for now the final era), they have unlimited forecast range. However, if the node is in Byron, we need to be careful.

lib/shelley/src/Cardano/Wallet/Shelley/Network.hs

Anviking · 2020-07-27T16:40:43Z

lib/shelley/src/Cardano/Wallet/Shelley/Network.hs

            Left e -> do
-                traceWith tr $ MsgInterpreterPastHorizon (pretty query) e
-                throwIO e
+                liftIO $ traceWith tr $ MsgInterpreterPastHorizon (pretty query) e


MsgInterpreterPastHorizon shouldn't be critical anymore, as it might be expected.

I lowered to Info and removed the log line about it being unexpected.

Anviking · 2020-07-27T17:09:31Z

bors try

iohk-bors · 2020-07-28T01:58:12Z

try

Build succeeded

rvl

There is so much unsafeRunExceptT here. It makes our code look unsafe.
This is replacing an actual exception type PastHorizonException with userError and a string message, which is worse.

Why not let all the time/slot conversion functions throw PastHorizonException, wherever unsafeRunExceptT is currently used. In the other "safe" cases, catch the exception.

Alternatively, replace unsafeRunExceptT with a new util runNonFailingQuery (or some name like that). This will be basically runNonFailingQuery q = q >>= either throwIO pure. The advantage of this is that it's obvious where the exception could occur, and userError is not used.

And also we should sprinkle HasCallStack liberally - otherwise tracking down errors will be hard.

KtorZ · 2020-07-28T08:57:59Z

lib/shelley/src/Cardano/Wallet/Shelley/Network.hs

@@ -1024,4 +1037,4 @@ instance HasSeverityAnnotation (NetworkLayerLog b) where
        MsgWatcherUpdate{}                 -> Debug
        MsgChainSyncCmd cmd                -> getSeverityAnnotation cmd
        MsgInterpreter{}                   -> Debug
-        MsgInterpreterPastHorizon{}        -> Critical
+        MsgInterpreterPastHorizon{}        -> Info


Is this change really sound 🤔 ?

I think the reasoning is that we have actually some cases where the PastHorizonException is "normal" and somewhat expected.

But perhaps we can divide slot/time conversions into two categories:

Where it's our bug if the conversion fails.

Where any conversion errors are because of the user inputting a date which is beyond the safe zone. So an error not a bug.

On launch, the TimeInterpreter may not have been fetched from the node. Instead returning the singleEraInterpreter for the first era, it seems safer to block until fetched. I imagine there could be race conditions where we would sometimes return completely wrong time data when in Shelley, just after starting the node. Not completely sure, but I hope there shouldn't be any drawbacks with blocking.

KtorZ · 2020-07-28T10:56:18Z

bors try

iohk-bors · 2020-07-28T11:09:21Z

try

Build failed

buildkite/cardano-wallet

If the node is not enough in-sync, we cannot know them.

Anviking · 2020-07-28T11:36:45Z

bors try

But rather respond with NotResponding.

It fails without the recent fix

Instead, we do not push the exception down to every caller but rather, throw it as an exception in the network layer. The rationale is that, this exception can only occur when both conditions are met: a) The node is still syncing and doesn't yet know about any hard-fork. b) A time beyond the node's foreseeable future is queried. While syncing in Byron these two conditions can't be met (times referenced in blocks are neccessarily before the node's tip and can't be beyond its foreseeable future. There's the case of delegation certificates and or transaction TTL but these only exists in Shelley, where the foreseeable future is so far, unlimited. Yet, there are points in the API where a time that is far beyond the node's tip can be provided and that is: - As filtering parameter when listing transactions. - As current time when looking at network parameters.

iohk-bors · 2020-07-28T12:38:47Z

try

Build succeeded

KtorZ · 2020-07-28T12:40:36Z

Waited for the "trying" branch to pass. Then pushed a little update that reshaping the git history and, lowering the "PastHorizon" error from Critical to Error. Here was the diff:

$ git diff HEAD origin/anviking/1869/improve-time-interpreter 
diff --git a/lib/shelley/src/Cardano/Wallet/Shelley/Network.hs b/lib/shelley/src/Cardano/Wallet/Shelley/Network.hs
index 833ff04f2..971cfbfc4 100644
--- a/lib/shelley/src/Cardano/Wallet/Shelley/Network.hs
+++ b/lib/shelley/src/Cardano/Wallet/Shelley/Network.hs
@@ -1051,4 +1051,4 @@ instance HasSeverityAnnotation (NetworkLayerLog b) where
         MsgWatcherUpdate{}                 -> Debug
         MsgChainSyncCmd cmd                -> getSeverityAnnotation cmd
         MsgInterpreter{}                   -> Debug
-        MsgInterpreterPastHorizon{}        -> Error
+        MsgInterpreterPastHorizon{}        -> Critical

Anviking self-assigned this Jul 23, 2020

Anviking added this to the (ADP-356) Integration with Hard Fork Combinator milestone Jul 23, 2020

Anviking linked an issue Jul 23, 2020 that may be closed by this pull request

Support SlotNo <-> UTCTime conversions #1869

Closed

Anviking removed a link to an issue Jul 23, 2020

Support SlotNo <-> UTCTime conversions #1869

Closed

KtorZ reviewed Jul 23, 2020

View reviewed changes

lib/shelley/src/Cardano/Wallet/Shelley/Network.hs Outdated Show resolved Hide resolved

KtorZ force-pushed the rvl/1870/bump-deps branch 3 times, most recently from c8b3323 to 8f756c0 Compare July 24, 2020 07:06

Anviking force-pushed the anviking/1869/improve-time-interpreter branch from 9392ea0 to 1d99265 Compare July 24, 2020 09:55

KtorZ force-pushed the rvl/1870/bump-deps branch 3 times, most recently from f580554 to 3ade904 Compare July 24, 2020 11:04

Anviking force-pushed the anviking/1869/improve-time-interpreter branch from f928fd0 to a2d93ed Compare July 24, 2020 11:27

KtorZ force-pushed the rvl/1870/bump-deps branch 2 times, most recently from 8539d1a to a05514b Compare July 24, 2020 20:41

Anviking force-pushed the anviking/1869/improve-time-interpreter branch from a2d93ed to 563dc47 Compare July 24, 2020 22:40

KtorZ force-pushed the rvl/1870/bump-deps branch from a05514b to 11f35b4 Compare July 25, 2020 10:11

Base automatically changed from rvl/1870/bump-deps to master July 25, 2020 17:08

Anviking force-pushed the anviking/1869/improve-time-interpreter branch from 563dc47 to 376376a Compare July 25, 2020 20:28

Anviking mentioned this pull request Jul 25, 2020

Provide more information on failure to determine current epoch. #1939

Closed

1 task

Anviking force-pushed the anviking/1869/improve-time-interpreter branch 2 times, most recently from cea9eb0 to a1cc8c6 Compare July 27, 2020 14:52

Anviking changed the title ~~WIP: Improvements to slotting conversions / TimeInterpreter~~ Improvements to slotting conversions / TimeInterpreter Jul 27, 2020

Anviking mentioned this pull request Jul 27, 2020

GET http://localhost:8090/v2/network/information returns 500 on mainnet while wallet is syncing through Byron era. #1960

Closed

paweljakubas mentioned this pull request Jul 27, 2020

Hardfork countdown support #1932

Merged

7 tasks

Anviking marked this pull request as ready for review July 27, 2020 16:00

Anviking force-pushed the anviking/1869/improve-time-interpreter branch from ae685c2 to 1e232f8 Compare July 27, 2020 16:30

Anviking commented Jul 27, 2020

View reviewed changes

iohk-bors bot added a commit that referenced this pull request Jul 27, 2020

Try #1943:

4855ec1

rvl reviewed Jul 28, 2020

View reviewed changes

KtorZ reviewed Jul 28, 2020

View reviewed changes

Anviking added 5 commits July 28, 2020 12:47

Explain why delegation time conversion never should fail

c1e8b17

Simplify constructors of TimeInterpreter

e4a754e

Use TimeInterpreter (ExceptT PastTimeHorizon IO)

c5794e7

Add endTimeOfEpoch and test

3e9ed4e

KtorZ force-pushed the anviking/1869/improve-time-interpreter branch from 15ec465 to 351a1b1 Compare July 28, 2020 10:55

iohk-bors bot added a commit that referenced this pull request Jul 28, 2020

Try #1943:

6bb4366

Anviking added 2 commits July 28, 2020 13:35

Make network_tip and next_epoch optional

8845de8

If the node is not enough in-sync, we cannot know them.

Use TimeInterpreter for pool retirements

b30a596

Anviking force-pushed the anviking/1869/improve-time-interpreter branch from bfeefe5 to 78561c1 Compare July 28, 2020 11:36

Anviking added 4 commits July 28, 2020 13:36

Lower MsgInterpreterPastHorizon to Error

5d0bd90

*If* the time of the tip ever fails, don't crash

fd3af1c

But rather respond with NotResponding.

Fix overflow in syncProgress calculation

8954723

Add regression test for syncProgress overflow

39c4106

It fails without the recent fix

iohk-bors bot added a commit that referenced this pull request Jul 28, 2020

Try #1943:

b91b91f

Anviking and others added 2 commits July 28, 2020 13:41

handle plausible PastTimeHorizon exception in the API server.

c46082b

KtorZ force-pushed the anviking/1869/improve-time-interpreter branch from 78561c1 to c46082b Compare July 28, 2020 12:38

KtorZ approved these changes Jul 28, 2020

View reviewed changes

KtorZ merged commit aab4ddf into master Jul 28, 2020

KtorZ deleted the anviking/1869/improve-time-interpreter branch July 28, 2020 12:39

Anviking mentioned this pull request Jul 28, 2020

Support SlotNo <-> UTCTime conversions #1869

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improvements to slotting conversions / TimeInterpreter #1943

Improvements to slotting conversions / TimeInterpreter #1943

Anviking commented Jul 23, 2020 •

edited

Loading

Anviking Jul 27, 2020 •

edited

Loading

Anviking Jul 27, 2020

Anviking commented Jul 27, 2020

iohk-bors bot commented Jul 28, 2020

rvl left a comment

KtorZ Jul 28, 2020

rvl Jul 28, 2020

KtorZ commented Jul 28, 2020

iohk-bors bot commented Jul 28, 2020

Anviking commented Jul 28, 2020

iohk-bors bot commented Jul 28, 2020

KtorZ commented Jul 28, 2020

Improvements to slotting conversions / TimeInterpreter #1943

Improvements to slotting conversions / TimeInterpreter #1943

Conversation

Anviking commented Jul 23, 2020 • edited Loading

Issue Number

Overview

Comments

Anviking Jul 27, 2020 • edited Loading

Choose a reason for hiding this comment

Anviking Jul 27, 2020

Choose a reason for hiding this comment

Anviking commented Jul 27, 2020

iohk-bors bot commented Jul 28, 2020

try

Build succeeded

rvl left a comment

Choose a reason for hiding this comment

KtorZ Jul 28, 2020

Choose a reason for hiding this comment

rvl Jul 28, 2020

Choose a reason for hiding this comment

KtorZ commented Jul 28, 2020

iohk-bors bot commented Jul 28, 2020

try

Build failed

Anviking commented Jul 28, 2020

iohk-bors bot commented Jul 28, 2020

try

Build succeeded

KtorZ commented Jul 28, 2020

Anviking commented Jul 23, 2020 •

edited

Loading

Anviking Jul 27, 2020 •

edited

Loading