How to handle builtins before they're costed (PLT-9686) (#5874)

* Updates to default cost models and documentation

* Test for identically-zero costing functions

* Test for identically-zero costing functions

* Test for identically-zero costing functions

* Test for identically-zero costing functions

* Test for identically-zero costing functions

* Add comments in plutus-ledger-api

* Add comments in plutus-ledger-api

* Comment

* Minor corrections

* Fix generator for empty list

* Typeclass for unimplemented costing functions

plutus-core/cost-model/CostModelGeneration.md

@@ -63,7 +63,7 @@ costing functions involves a number of steps.
 
 * When the rest of the `plutus-core` package is compiled, the contents of
   `builtCostModel.json` are read and used by some Template Haskell code to
-  construct Haskell functions which implement the cost models.  
+  construct Haskell functions which implement the cost models.
 
 * To ensure consistency, `cabal bench plutus-core:cost-model-test` runs some
   QuickCheck tests to run the R models and the Haskell models and checks that
@@ -136,11 +136,12 @@ codebase.
 
     ```
     toBuiltinMeaning XorByteString =
-        makeBuiltinMeaning
-            xorByteString
-            mempty
-            where xorByteString a b =
+        let xorByteStringDenotation :: Data.ByteString.ByteString -> Data.ByteString.ByteString -> Data.ByteString.ByteString
+            xorByteStringDenotation a b =
                       Data.ByteString.pack $ zipWith (Data.Bits.xor) (Data.ByteString.unpack a) (Data.ByteString.unpack b)
+        in makeBuiltinMeaning
+               xorByteStringDenotation
+               (runCostingFunTwoArguments . unimplementedCostingFun)
     ```
 
     This assumes that the appropriate modules have been imported.  The
@@ -150,9 +151,11 @@ codebase.
     a check that the inputs are the same length.  For more complicated functions
     one might also put the implementation in a separate file.
 
-    The final argument of `makeBuiltinMeaning` contains the costing functions for
-    the relevant builtin.  Initially this should be set to `mempty`; we'll come
-    back and fix it later.
+    The final argument of `makeBuiltinMeaning` contains the costing functions
+    for the relevant builtin.  Initially this should be set to
+    `unimplementedCostingFun`; we'll come back and fix it later.  This assigns a
+    very large cost to prevent the uncosted version from being accidentally used
+    in situations where precise costs are important.
 
     Note that there are certain restrictions on built-in functions: for example,
     the function should be deterministic, it **must not throw any exceptions**,
@@ -181,10 +184,10 @@ to check that the semantics of the new builtin are correct.
 
 After the above steps have been carried out the new builtin will be available in
 Plutus Core, but will not incur any charges when it is called.  To fix this we
-have to add a costing function of a suitable shape and replace the `mempty` in
+have to add a costing function of a suitable shape and replace the `unimplementedCostingFun` in
 the definition of the function.
 
-#### Step 1: add the basic type of the costing function to the cost model type 
+#### Step 1: add the basic type of the costing function to the cost model type
 
 Firstly, add a new entry to the `BuiltinCostModelBase` type in
 [`PlutusCore.Evaluation.Machine.BuiltinCostModel`](../plutus-core/src/PlutusCore/Evaluation/Machine/BuiltinCostModel.hs).
@@ -207,8 +210,9 @@ in this case you should add new cases to the appropriate
 
 For `xorByteString` it would be reasonable to expect the time taken to be linear
 in the minimum of the argument sizes (the function stops when it gets to the end
-of the smaller bytestring), so we should use the `ModelTwoArgumentsMinSize`
-constructor: see Step 6 for this, and Step 7 for a caveat.
+of the smaller bytestring), so we should probably use the
+`ModelTwoArgumentsMinSize` constructor: see Steps 6 and 7 for details and some
+caveats.
 
 
 #### Step 2: add a unit cost model for new function
@@ -275,18 +279,19 @@ the Cost Model" note.
 #### Step 4: add the correct costing function to the definition of the new builtin
 
 Now go back to
-[`Builtins.hs`](../plutus-core/src/PlutusCore/Default/Builtins.hs) and
-replace `mempty` in the definition of the builtin with some code to
-run the appropriate `param<builtin-name>` function:
+[`Builtins.hs`](../plutus-core/src/PlutusCore/Default/Builtins.hs) and replace
+`unimplementedCostingFun` in the definition of the builtin with the appropriate
+`param<builtin-name>` function:
 
-```
+    ```
     toBuiltinMeaning XorByteString =
-        makeBuiltinMeaning
-            xorByteString
-            (runCostingFunTwoArguments . paramXorByteString)
-            where xorByteString a b =
+        let xorByteStringDenotation :: Data.ByteString.ByteString -> Data.ByteString.ByteString -> Data.ByteString.ByteString
+            xorByteStringDenotation a b =
                       Data.ByteString.pack $ zipWith (Data.Bits.xor) (Data.ByteString.unpack a) (Data.ByteString.unpack b)
-```
+        in makeBuiltinMeaning
+               xorByteStringDenotation
+               (runCostingFunTwoArguments . paramXorByteString)
+    ```
 
 #### Step 5: add a benchmark for the new builtin and run it
 
@@ -362,7 +367,7 @@ worst-case model.
             filter.and.check.nonempty(fname) %>%
             discard.overhead ()
         m <- lm(t ~ pmin(x_mem, y_mem), filtered)
-        adjustModel(m,fname)
+        return (mk.result (m, "min_size")
     }
 ```
 
@@ -379,7 +384,7 @@ object. (That's what gets read in by the code in Step 6: `paramXorByteString`
 contains the string "xorByteStringModel" and that lets the Haskell code retrieve
 the correct thing from R.)
 
-#### Step 7: add code to read the costing function from R into Haskell 
+#### Step 7: add code to read the costing function from R into Haskell
 
 Next we have to update the code which converts benchmarking results into JSON
 models.  Go to
@@ -397,28 +402,30 @@ occur in the object, and they can sometimes be quite cryptic.)
 Also add a new clause in [`CreateBuiltinCostModel`](./create-cost-model/CreateBuiltinCostModel.hs):
 
 ```
-    paramXorByteString                   <- getParams xorByteString paramXorByteString
-```
-
-and a function to extract the cost parameters for the R code.  This should be modelled on the existing
-functions at the end of the file:
+    paramXorByteString                   <- getParams readCF2 paramXorByteString
 
 ```
-xorByteString :: MonadR m => (SomeSEXP (Region m)) -> m (CostingFun ModelTwoArguments)
-xorByteString cpuModelR = do
-  cpuModel <- ModelTwoArgumentsMinSize <$> readModelMinSize cpuModelR
-  let memModel = ModelTwoArgumentsMinSize $ ModelMinSize 0 1
-  pure $ CostingFun (cpuModel) memModel
+(in general, use `readCF<N>` function where `N` is the arity of the builtin).
+
+When the Haskell code is run it will run the R code and process the objects
+constructed by it.  For `paramXorByteString` it will read the "min_size" tag and
+create a `ModelMinSize` object in Haskell, with the constructor arguments also
+extracted from the R object.
+
+The CPU costing function is obtained from the R code, but the memory usage
+costing function is defined statically in
+[`BuiltinMemoryModels`](./create-cost-model/BuiltinMemoryModels.hs).
+Memory usage costing functions only account for memory retained after the
+function has returned and not for any working memory that may be allocated
+during its execution.  Typically this means that the memory costing function
+should measure the size of the object returned by the builtin.  For our
+`xorByteString` implementation, if the arguments have sizes `m` and `n` then the
+result will have size `min(m,n)` so we define the memory costing function to be
+`(m,n) -> 0 + 1*min(m,n)`: this is represented in the Haskell file by
 ```
+  paramXorByteString = Id $ ModelTwoArgumentsMinSize $ OneVariableLinearFunction 0 1
 
-The CPU costing function is obtained by running the R code, but the memory usage
-costing function is defined statically here.  Memory usage costing functions
-only account for memory retained after the function has returned and not for any
-working memory that may be allocated during its execution.  Typically this means
-that the memory costing function should measure the size of the object returned
-by the builtin.  For our `xorByteString` implementation, if the arguments have
-sizes `m` and `n` then the result will have size `min(m,n)` so we define the memory
-costing function to be `(m,n) -> 0 + 1*min(m,n)`.
+```
 
 
 #### Step 8: test the Haskell versions of the costing functions

plutus-core/cost-model/data/builtinCostModel.json

@@ -934,4 +934,4 @@
             "type": "constant_cost"
         }
     }
-}
+}

plutus-core/executables/src/PlutusCore/Executable/Common.hs

@@ -544,8 +544,9 @@ runPrintBuiltinSignatures = do
       (\x -> putStr (printf "%-35s: %s\n" (show $ PP.pretty x) (show $ getSignature x)))
       builtins
   where
-    getSignature (PLC.toBuiltinMeaning @_ @_ @(PlcTerm ()) def -> PLC.BuiltinMeaning sch _ _) =
-        typeSchemeToSignature sch
+    getSignature b =
+      case PLC.toBuiltinMeaning @PLC.DefaultUni @PLC.DefaultFun @(PlcTerm ()) def b of
+        PLC.BuiltinMeaning sch _ _ -> typeSchemeToSignature sch
 
 ---------------- Parse and print a PLC/UPLC source file ----------------
 

plutus-core/plutus-core.cabal

@@ -353,6 +353,7 @@ test-suite plutus-core-test
     CBOR.DataStability
     Check.Spec
     CostModelInterface.Spec
+    CostModelSafety.Spec
     Evaluation.Machines
     Evaluation.Spec
     Generators.QuickCheck.Utils
@@ -370,6 +371,7 @@ test-suite plutus-core-test
     , bytestring
     , containers
     , data-default-class
+    , extra
     , filepath
     , flat                                            ^>=0.6
     , hedgehog

plutus-core/plutus-core/src/PlutusCore/Default/Builtins.hs

@@ -1818,6 +1818,22 @@ instance uni ~ DefaultUni => ToBuiltinMeaning uni DefaultFun where
     -- See Note [Inlining meanings of builtins].
     {-# INLINE toBuiltinMeaning #-}
 
+    {- *** IMPORTANT! *** When you're adding a new builtin above you typically won't
+       be able to add a sensible costing function until the implementation is
+       complete and you can benchmark it.  It's still necessary to supply
+       `toBuiltinMeaning` with some costing function though: this **MUST** be
+       `unimplementedCostingFun`: this will assign a very large cost to any
+       invocation of the function, preventing it from being used in places where
+       costs are important (for example on testnets) until the implementation is
+       complete and a proper costing function has been defined.  Once the
+       builtin is ready for general use replace `unimplementedCostingFun` with
+       the appropriate `param<BuiltinName>` from BuiltinCostModelBase.
+
+       Please leave this comment immediately after the definition of the final
+       builtin to maximise the chances of it being seen the next time someone
+       implements a new builtin.
+    -}
+
 instance Default (BuiltinSemanticsVariant DefaultFun) where
     def = DefaultFunSemanticsVariant2
 

plutus-core/plutus-core/src/PlutusCore/Evaluation/Machine/BuiltinCostModel.hs

@@ -14,6 +14,7 @@ module PlutusCore.Evaluation.Machine.BuiltinCostModel
     ( BuiltinCostModel
     , BuiltinCostModelBase(..)
     , CostingFun(..)
+    , UnimplementedCostingFun(..)
     , Intercept(..)
     , Slope(..)
     , Coefficient0(..)

plutus-core/plutus-core/src/PlutusCore/Evaluation/Machine/CostingFun/Core.hs

@@ -3,13 +3,15 @@
 {-# LANGUAGE DeriveAnyClass        #-}
 {-# LANGUAGE FlexibleInstances     #-}
 {-# LANGUAGE MultiParamTypeClasses #-}
+{-# LANGUAGE NumericUnderscores    #-}
 {-# LANGUAGE TypeFamilies          #-}
 {-# LANGUAGE TypeOperators         #-}
 {-# LANGUAGE UndecidableInstances  #-}
 
 {-# LANGUAGE StrictData            #-}
 module PlutusCore.Evaluation.Machine.CostingFun.Core
     ( CostingFun(..)
+    , UnimplementedCostingFun(..)
     , Intercept(..)
     , Slope(..)
     , Coefficient0(..)
@@ -77,13 +79,47 @@ instance ab ~ ExBudgetStream => OnMemoryUsages ExBudgetStream ab where
     onMemoryUsages = id
     {-# INLINE onMemoryUsages #-}
 
+{- | A type of costing functions parametric over a model type.  In practice the we
+have one model type `Model<N>Arguments` for every N, where N is the arity of the
+builtin whose costs we want to model.  Each model type has a number of
+constructors defining different "shapes" of N-parameter functions which
+calculate a cost given the sizes of the builtin's arguments. -}
 data CostingFun model = CostingFun
     { costingFunCpu    :: model
     , costingFunMemory :: model
     }
     deriving stock (Show, Eq, Generic, Lift)
     deriving anyclass (Default, NFData)
 
+{- | In the initial stages of implementing a new builtin it is necessary to
+   provide a temporary costing function which is used until the builtin has been
+   properly costed: `see CostModelGeneration.md`.  Each `Model<N>Arguments` type
+   defines an instance of this class where `unimplementedCostingFun` is a
+   constant costing function which returns a very high cost for all inputs.
+   This prevents new functions from being used in situations where costs are
+   important until a sensible costing function has been implemented. -}
+class UnimplementedCostingFun a where
+  unimplementedCostingFun :: b -> CostingFun a
+
+{- | Make a very expensive pair of CPU and memory costing functions.  The name is
+   slightly misleading because it actually makes a function which returns such a
+   pair, which is what is required at the use site in `PlutusCore.Default.Builtins`,
+   where properly implemented costing functions are constructed from a
+   BuiltinCostModel object.  We can't use maxBound :: CostingInteger because then the
+   evaluator always fails; instead we assign a cost of 100,000,000,000, which is well
+   beyond the current on-chain CPU and memory limits (10,000,000,000 and 14,000,000
+   respectively) but still allows over 92,000,000 evaluations before the maximum
+   CostingInteger is reached.  This allows us to use an "uncosted" builtin for
+   testing and for running costing benchmarks, but will prevent it from being used
+   when the Plutus Core evaluator is invoked by the ledger.
+-}
+makeUnimplementedCostingFun :: (CostingInteger -> model) -> b -> CostingFun model
+makeUnimplementedCostingFun c =
+  const $ CostingFun (c k) (c k)
+  where k = 100_000_000_000
+
+---------------- Types for use within costing functions ----------------
+
 -- | A wrapped 'CostingInteger' that is supposed to be used as an intercept.
 newtype Intercept = Intercept
     { unIntercept :: CostingInteger
@@ -124,8 +160,12 @@ data ModelOneArgument =
     | ModelOneArgumentLinearCost OneVariableLinearFunction
     deriving stock (Show, Eq, Generic, Lift)
     deriving anyclass (NFData)
+
 instance Default ModelOneArgument where
-    def = ModelOneArgumentConstantCost 0
+    def = ModelOneArgumentConstantCost maxBound
+
+instance UnimplementedCostingFun ModelOneArgument where
+  unimplementedCostingFun = makeUnimplementedCostingFun ModelOneArgumentConstantCost
 
 {- Note [runCostingFun* API]
 Costing functions take unlifted values, compute the 'ExMemory' of each of them and then invoke
@@ -330,7 +370,10 @@ data ModelTwoArguments =
     deriving anyclass (NFData)
 
 instance Default ModelTwoArguments where
-    def = ModelTwoArgumentsConstantCost 0
+    def = ModelTwoArgumentsConstantCost maxBound
+
+instance UnimplementedCostingFun ModelTwoArguments where
+  unimplementedCostingFun = makeUnimplementedCostingFun ModelTwoArgumentsConstantCost
 
 -- See Note [runCostingFun* API].
 runCostingFunTwoArguments
@@ -458,7 +501,10 @@ data ModelThreeArguments =
     deriving anyclass (NFData)
 
 instance Default ModelThreeArguments where
-    def = ModelThreeArgumentsConstantCost 0
+    def = ModelThreeArgumentsConstantCost maxBound
+
+instance UnimplementedCostingFun ModelThreeArguments where
+  unimplementedCostingFun = makeUnimplementedCostingFun ModelThreeArgumentsConstantCost
 
 runThreeArgumentModel
     :: ModelThreeArguments
@@ -528,7 +574,10 @@ data ModelFourArguments =
     deriving anyclass (NFData)
 
 instance Default ModelFourArguments where
-    def = ModelFourArgumentsConstantCost 0
+    def = ModelFourArgumentsConstantCost maxBound
+
+instance UnimplementedCostingFun ModelFourArguments where
+  unimplementedCostingFun = makeUnimplementedCostingFun ModelFourArgumentsConstantCost
 
 runFourArgumentModel
     :: ModelFourArguments
@@ -570,7 +619,10 @@ data ModelFiveArguments =
     deriving anyclass (NFData)
 
 instance Default ModelFiveArguments where
-    def = ModelFiveArgumentsConstantCost 0
+    def = ModelFiveArgumentsConstantCost maxBound
+
+instance UnimplementedCostingFun ModelFiveArguments where
+  unimplementedCostingFun = makeUnimplementedCostingFun ModelFiveArgumentsConstantCost
 
 runFiveArgumentModel
     :: ModelFiveArguments
@@ -614,7 +666,10 @@ data ModelSixArguments =
     deriving anyclass (NFData)
 
 instance Default ModelSixArguments where
-    def = ModelSixArgumentsConstantCost 0
+    def = ModelSixArgumentsConstantCost maxBound
+
+instance UnimplementedCostingFun ModelSixArguments where
+  unimplementedCostingFun = makeUnimplementedCostingFun ModelSixArgumentsConstantCost
 
 runSixArgumentModel
     :: ModelSixArguments