Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Imports + Myriad Bug Fixes #585

Merged
merged 35 commits into from Sep 12, 2019
Merged
Changes from 29 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
cf1ae0c
update bnf with module guards
emilypi Jul 15, 2019
2d4d211
no more importing interfaces
emilypi Jul 15, 2019
f27997b
disallow module definition name shadowing
emilypi Jul 15, 2019
8495e20
remove stale md
emilypi Jul 15, 2019
a18eca4
remove space
emilypi Jul 15, 2019
9211d1b
undo lensiness
emilypi Jul 15, 2019
0abffe1
found a bug in tc.repl test suite
emilypi Jul 15, 2019
945f51d
disallow native name overlaps
emilypi Jul 16, 2019
99506db
update bench.pact
emilypi Jul 16, 2019
b508446
add same check for interfaces
emilypi Jul 17, 2019
7d185ff
update cabal project
emilypi Jul 17, 2019
abe19a4
hidden imports updates
emilypi Jul 18, 2019
7a3d5df
streamline our gas + name retrievals
emilypi Jul 18, 2019
0068158
updates
emilypi Jul 18, 2019
703c8de
add correct parsing form
emilypi Jul 19, 2019
389f147
remove cruft
emilypi Jul 19, 2019
12932c2
fix installModule + some updates. TODO: track imports when resolving …
emilypi Jul 20, 2019
c0dc0f9
updates, add case for def conflicting with import
emilypi Jul 20, 2019
510ea9a
remove stack spam
emilypi Jul 20, 2019
b9e4ac5
remove test that we decided was ok to have
emilypi Jul 20, 2019
b945fd7
disallow capability imports
emilypi Jul 20, 2019
2dac411
add note, switch back to foldl
emilypi Jul 20, 2019
264bd82
small undo's
emilypi Jul 21, 2019
dc43979
json instance for 'Use'
emilypi Jul 21, 2019
fa3bdd5
update docs
emilypi Jul 29, 2019
1c03b85
restrict validateImports to just functions, constants, schema
emilypi Jul 29, 2019
3bdcfef
small switchups to the API
emilypi Jul 30, 2019
cee7b33
add additional test for table symbols
emilypi Jul 30, 2019
c120470
Merge branch 'master' into emily/bugfix-571
emilypi Aug 1, 2019
927cdb5
Update tests/pact/tc.repl
emilypi Sep 12, 2019
c0859c3
Update tests/pact/tc.repl
emilypi Sep 12, 2019
21ffab6
address stuart's comments
emilypi Sep 12, 2019
f63cce3
Merge branch 'emily/bugfix-571' of github.com:kadena-io/pact into emi…
emilypi Sep 12, 2019
68705a1
bump version to 3.3.0
emilypi Sep 12, 2019
cf945b0
Merge branch 'master' into emily/bugfix-571
emilypi Sep 12, 2019
File filter...
Filter file types
Jump to…
Jump to file or symbol
Failed to load files and symbols.

Always

Just for now

@@ -2502,18 +2502,37 @@ ROLLBACK-EXPR functions as a "cancel function" to be explicitly executed by a pa
```
(use MODULE)
(use MODULE HASH)
(use MODULE IMPORTS)
(use MODULE HASH IMPORTS)
```

Import an existing MODULE into a namespace. Can only be issued at the top-level, or within a module
declaration. MODULE can be a string, symbol or bare atom. With HASH, validate that the imported module's
hash matches HASH, failing if not. Use [describe-module](pact-functions.html#describe-module) to query for the
hash of a loaded module on the chain.

An optional list of IMPORTS consisting of function, constant, and schema names may be supplied. When this explicit import list is present, only those names will be made available for use in the module body. If no list is supplied, then every name in the imported module will be brought into scope. When two modules are defined in the same transaction, all names will be in scope for all modules, and import behavior will be defaulted to the entire module.

```lisp
(use accounts)
(transfer "123" "456" 5 (time "2016-07-22T11:26:35Z"))
"Write succeeded"
```
```lisp
(use accounts "ToV3sYFMghd7AN1TFKdWk_w00HjUepVlqKL79ckHG_s")
(transfer "123" "456" 5 (time "2016-07-22T11:26:35Z"))
"Write succeeded"
```
```lisp
(use accounts [ transfer example-fun ])
(transfer "123" "456" 5 (time "2016-07-22T11:26:35Z"))
"Write succeeded"
```
```lisp
(use accounts "ToV3sYFMghd7AN1TFKdWk_w00HjUepVlqKL79ckHG_s" [ transfer example-fun ])
(transfer "123" "456" 5 (time "2016-07-22T11:26:35Z"))
"Write succeeded"
```

### module {#module}
```
@@ -26,13 +26,17 @@ module Pact.Compile
, Reserved(..)
) where

import Prelude hiding (exp)

import Bound

import Control.Applicative hiding (some,many)
import Control.Arrow ((&&&),first)
import Control.Exception hiding (try)
import Control.Lens hiding (prism)
import Control.Monad
import Control.Monad.State

import Data.Default
import qualified Data.HashMap.Strict as HM
import qualified Data.HashSet as HS
@@ -43,7 +47,7 @@ import Data.String
import Data.Text (Text,pack,unpack)
import Data.Text.Encoding (encodeUtf8)
import qualified Data.Vector as V
import Prelude hiding (exp)

import Text.Megaparsec as MP
import qualified Text.Trifecta as TF hiding (expected)

@@ -545,13 +549,16 @@ letsForm = do
useForm :: Compile (Term Name)
useForm = do
mn <- qualifiedModuleName
i <- contextInfo
u <- Use mn <$> optional hash' <*> pure i
i <- contextInfo
h <- optional hash'
l <- optional $ withList' Brackets (some userAtom <* eof)

let v = fmap (V.fromList . fmap _atomAtom) l
u = Use mn h v i
-- this is the one place module may not be present, use traversal
psUser . csModule . _Just . msImports %= (u:)
return $ TUse u i


hash' :: Compile ModuleHash
hash' = str >>= \s -> case fromText' s of
Right h -> return (ModuleHash h)
@@ -305,27 +305,45 @@ toPersistDirect' t = case toPersistDirect t of


evalUse :: Use -> Eval e ()
evalUse (Use mn h i) = do
evalUse (Use mn h mis i) = do
mm <- resolveModule i mn
case mm of
Nothing -> evalError i $ "Module " <> pretty mn <> " not found"
Just md -> do
case view mdModule md of
case _mdModule md of
MDModule Module{..} ->
case h of
Nothing -> return ()
Just mh | mh == _mHash -> return ()
| otherwise -> evalError i $ "Module " <>
pretty mn <> " does not match specified hash: " <>
pretty mh <> ", " <> pretty _mHash
MDInterface Interface{..} ->
MDInterface i' ->
case h of
Nothing -> return ()
Just _ -> evalError i $
"Interfaces should not have associated hashes: " <>
pretty _interfaceName
Just _ -> evalError i
$ "Interfaces should not have associated hashes: "
<> pretty (_interfaceName i')

installModule False md
validateImports i (_mdRefMap md) mis
installModule False md mis

validateImports :: Info -> HM.HashMap Text Ref -> Maybe (V.Vector Text) -> Eval e ()
validateImports _ _ Nothing = return ()
validateImports i rs (Just is) = traverse_ go is
where
go imp = case HM.lookup imp rs of
Nothing -> evalError i $ "imported name not found: " <> pretty imp
Just (Ref r) -> case r of
TDef d _ -> case _dDefType d of
Defcap -> evalError i $ "cannot import capabilities: " <> pretty imp
_ -> return ()
TConst{} -> return ()
TSchema{} -> return ()
_ -> evalError i
$ "invalid import - only function, schema, and constant symbols allowed: "
<> pretty imp
Just _ -> return ()
This conversation was marked as resolved by emilypi

This comment has been minimized.

Copy link
@emilypi

emilypi Jul 21, 2019

Author Collaborator

We could probably restrict this further so it fails on load before runtime. Thoughts @slpopejoy?


mangleDefs :: ModuleName -> Term Name -> Term Name
mangleDefs mn term = modifyMn term
@@ -338,71 +356,94 @@ mangleDefs mn term = modifyMn term
_ -> id

-- | Make table of module definitions for storage in namespace/RefStore.
loadModule :: Module (Term Name) -> Scope n Term Name -> Info -> Gas
-> Eval e (Gas,ModuleData Ref)
loadModule m@Module {} bod1 mi g0 = do
(g1,mdefs) <-
case instantiate' bod1 of
(TList bd _ _bi) -> do
let doDef (g,rs) t = do
dnm <- case t of
TDef {..} -> return $ Just $ asString (_dDefName _tDef)
TConst {..} -> return $ Just $ _aName _tConstArg
TSchema {..} -> return $ Just $ asString _tSchemaName
TTable {..} -> return $ Just $ asString _tTableName
TUse (Use {..}) _ -> return Nothing
_ -> evalError (_tInfo t) "Invalid module member"
case dnm of
Nothing -> return (g, rs)
Just dn -> do
g' <- computeGas (Left (_tInfo t,dn)) (GModuleMember (MDModule m))
return (g + g',(dn,t):rs)
second HM.fromList <$> foldM doDef (g0,[]) bd
t -> evalError (_tInfo t) "Malformed module"
loadModule
:: Module (Term Name)
-> Scope n Term Name
-> Info
-> Gas
-> Eval e (Gas,ModuleData Ref)
loadModule m bod1 mi g0 = do
mapM_ evalUse $ _mImports m
evaluatedDefs <- evaluateDefs mi (fmap (mangleDefs $ _mName m) mdefs)
(g1,mdefs) <- collectNames g0 (GModuleMember $ MDModule m) bod1 $ \t -> case t of
TDef d _ -> return $ Just $ asString (_dDefName d)
TConst a _ _ _ _ -> return $ Just $ _aName a
TSchema n _ _ _ _ -> return $ Just $ asString n
tt@TTable{} -> return $ Just $ asString (_tTableName tt)
TUse _ _ -> return Nothing
_ -> evalError' t "Invalid module member"
evaluatedDefs <- evaluateDefs mi $ mangleDefs (_mName m) <$> mdefs
(m', solvedDefs) <- evaluateConstraints mi m evaluatedDefs
mGov <- resolveGovernance solvedDefs m'
let md = ModuleData mGov solvedDefs
installModule True md
installModule True md Nothing
return (g1,md)

resolveGovernance :: HM.HashMap Text Ref
-> Module (Term Name) -> Eval e (ModuleDef (Def Ref))
loadInterface
:: Interface
-> Scope n Term Name
-> Info
-> Gas
-> Eval e (Gas,ModuleData Ref)
loadInterface i body info gas0 = do
mapM_ evalUse $ _interfaceImports i
(gas1,idefs) <- collectNames gas0 (GModuleMember $ MDInterface i) body $ \t -> case t of
TDef d _ -> return $ Just $ asString (_dDefName d)
TConst a _ _ _ _ -> return $ Just $ _aName a
TSchema n _ _ _ _ -> return $ Just $ asString n
TUse _ _ -> return Nothing
_ -> evalError' t "Invalid interface member"
evaluatedDefs <- evaluateDefs info $ mangleDefs (_interfaceName i) <$> idefs
let md = ModuleData (MDInterface i) evaluatedDefs
installModule True md Nothing
return (gas1,md)

-- | Retrieve map of definition names to their corresponding terms
-- and compute their gas value
--
collectNames
:: Gas
-- ^ initial gas value
-> GasArgs
-- ^ gas args (should be GModuleMember)
-> Scope n Term Name
-- ^ module body
-> (Term Name -> Eval e (Maybe Text))
-- ^ function extracting definition names
-> Eval e (Gas, HM.HashMap Text (Term Name))
collectNames g0 args body k = case instantiate' body of
TList bd _ _ -> do
ns <- view $ eeRefStore . rsNatives
foldM (go ns) (g0, mempty) bd
t -> evalError' t $ "malformed declaration"
where
go ns (g,ds) t = k t >>= \dnm -> case dnm of
Nothing -> return (g, ds)
Just dn -> do
-- disallow native overlap
when (isJust $ HM.lookup (Name dn def) ns) $
evalError' t $ "definitions cannot overlap with native names: " <> pretty dn
-- disallow conflicting members
when (isJust $ HM.lookup dn ds) $
evalError' t $ "definition name conflict: " <> pretty dn

g' <- computeGas (Left (_tInfo t,dn)) args
return (g + g',HM.insert dn t ds)


resolveGovernance
:: HM.HashMap Text Ref
-> Module (Term Name)
-> Eval e (ModuleDef (Def Ref))
resolveGovernance solvedDefs m' = fmap MDModule $ forM m' $ \g -> case g of
TVar (Name n _) _ -> case HM.lookup n solvedDefs of
Just r -> case r of
(Ref (TDef govDef _)) -> case _dDefType govDef of
Ref (TDef govDef _) -> case _dDefType govDef of
Defcap -> return govDef
_ -> evalError (_tInfo g) "Invalid module governance, must be defcap"
_ -> evalError (_tInfo g) "Invalid module governance, should be def ref"
Nothing -> evalError (_tInfo g) "Unknown module governance reference"
_ -> evalError (_tInfo g) "Invalid module governance, should be var"

loadInterface :: Interface -> Scope n Term Name -> Info -> Gas
-> Eval e (Gas,ModuleData Ref)
loadInterface i@Interface{..} body info gas0 = do
(gas1,idefs) <- case instantiate' body of
(TList bd _ _bi) -> do
let doDef (g,rs) t = do
dnm <- case t of
TDef {..} -> return $ Just $ asString (_dDefName _tDef)
TConst {..} -> return $ Just $ _aName _tConstArg
TSchema {..} -> return $ Just $ asString _tSchemaName
TUse (Use {..}) _ -> return Nothing
_ -> evalError (_tInfo t) "Invalid interface member"
case dnm of
Nothing -> return (g, rs)
Just dn -> do
g' <- computeGas (Left (_tInfo t,dn)) (GModuleMember (MDInterface i))
return (g + g',(dn,t):rs)
second HM.fromList <$> foldM doDef (gas0,[]) bd
t -> evalError (_tInfo t) "Malformed interface"
mapM_ evalUse _interfaceImports
evaluatedDefs <- evaluateDefs info (fmap (mangleDefs _interfaceName) idefs)
let md = ModuleData (MDInterface i) evaluatedDefs
installModule True md
return (gas1,md)

-- | Definitions are transformed such that all free variables are resolved either to
-- an existing ref in the refstore/namespace ('Right Ref'), or a symbol that must
@@ -414,32 +455,36 @@ loadInterface i@Interface{..} body info gas0 = do
evaluateDefs :: Info -> HM.HashMap Text (Term Name) -> Eval e (HM.HashMap Text Ref)
evaluateDefs info defs = do
cs <- traverseGraph defs
sortedDefs <- forM cs $ \c ->
case c of
AcyclicSCC v -> return v
CyclicSCC vs -> evalError (if null vs then info else _tInfo $ view _1 $ head vs) $
"Recursion detected: " <>
prettyList (vs & traverse . _1 %~ fmap mkSomeDoc
& traverse . _3 %~ (SomeDoc . prettyList))
sortedDefs <- forM cs $ \c -> case c of
AcyclicSCC v -> return v
CyclicSCC vs -> do
let i = if null vs then info else _tInfo $ view _1 $ head vs
pl = vs
& traverse . _1 %~ fmap mkSomeDoc
This conversation was marked as resolved by emilypi

This comment has been minimized.

Copy link
@slpopejoy

slpopejoy Sep 12, 2019

Contributor

if there are changes to this file let's conform this lensy mess

This comment has been minimized.

Copy link
@emilypi

emilypi Sep 12, 2019

Author Collaborator

Conformed to the style guide

& traverse . _3 %~ (SomeDoc . prettyList)

evalError i $ "Recursion detected: " <> prettyList pl

-- the order of evaluation matters for 'dresolve' - this *must* be a left fold
let dresolve ds (d,dn,_) = HM.insert dn (Ref $ unify ds <$> d) ds
unifiedDefs = foldl dresolve HM.empty sortedDefs
unifiedDefs = foldl' dresolve HM.empty sortedDefs

traverse (runSysOnly . evalConsts) unifiedDefs
where
mkSomeDoc = either (SomeDoc . pretty) (SomeDoc . pretty)

traverseGraph ds = fmap stronglyConnCompR $ forM (HM.toList ds) $ \(dn,d) -> do
d' <- forM d $ \(f :: Name) -> do
dm <- resolveRef f f
case (dm, f) of
(Just t, _) -> return (Right t)
(Nothing, Name fn _) ->
case HM.lookup fn ds of
Just _ -> return (Left fn)
Nothing -> evalError (_nInfo f) $ "Cannot resolve " <> dquotes (pretty f)
(Nothing, _) -> evalError (_nInfo f) $ "Cannot resolve " <> dquotes (pretty f)

mkSomeDoc :: (Pretty a, Pretty b) => Either a b -> SomeDoc
mkSomeDoc = either (SomeDoc . pretty) (SomeDoc . pretty)

traverseGraph :: HM.HashMap Text (Term Name) -> Eval e [SCC (Term (Either Text Ref), Text, [Text])]
traverseGraph defs = fmap stronglyConnCompR $ forM (HM.toList defs) $ \(dn,d) -> do
d' <- forM d $ \(f :: Name) -> do
dm <- resolveRef f f
case (dm, f) of
(Just t, _) -> return (Right t)
(Nothing, Name fn _) ->
case HM.lookup fn defs of
Just _ -> return (Left fn)
Nothing -> evalError (_nInfo f) $ "Cannot resolve " <> dquotes (pretty f)
(Nothing, _) -> evalError (_nInfo f) $ "Cannot resolve " <> dquotes (pretty f)
return (d', dn, mapMaybe (either Just (const Nothing)) $ toList d')
return (d', dn, mapMaybe (either Just (const Nothing)) $ toList d')
This conversation was marked as resolved by emilypi

This comment has been minimized.

Copy link
@emilypi

emilypi Jul 20, 2019

Author Collaborator

Nothing referenced these functions aside from evaluateDefs, so they were moved into a where clause


-- | Evaluate interface constraints in module.
evaluateConstraints
@@ -494,8 +539,9 @@ solveConstraint info refName (Ref t) evalMap = do
when (length args /= length args') $ evalError info $ "mismatching argument lists: "
<> prettyList args <> line <> prettyList args'
forM_ (args `zip` args') $ \((Arg n ty _), (Arg n' ty' _)) -> do
when (n /= n') $ evalError info $ "mismatching argument names: "
<> pretty n <> " and " <> pretty n'
-- FV requires exact argument names as opposed to positional info
when (n /= n') $ evalError info $ "argument names must match interface definition: "
<> pretty n <> " does not match " <> pretty n'
when (ty /= ty') $ evalError info $ "mismatching types: "
<> pretty ty <> " and " <> pretty ty'
-- the model concatenation step: we reinsert the ref back into the map with new models
@@ -515,7 +561,7 @@ moduleResolver lkp i mn = do
case md of
Just _ -> return md
Nothing -> do
case (_mnNamespace mn) of
case _mnNamespace mn of
Just {} -> pure Nothing -- explicit namespace not found
Nothing -> do
mNs <- use $ evalRefs . rsNamespace
@@ -810,17 +856,28 @@ resolveFreeVars i b = traverse r b where
Nothing -> evalError i $ "Cannot resolve " <> pretty fv
Just d -> return d

-- | Install module into local namespace. If updated/new, update loaded modules.
installModule :: Bool -> ModuleData Ref -> Eval e ()
installModule updated md@ModuleData{..} = do
(evalRefs . rsLoaded) %= HM.union (HM.fromList . map (first (`Name` def)) . HM.toList $ _mdRefMap)
when updated $
(evalRefs . rsLoadedModules) %= HM.insert (moduleDefName _mdModule) (md,updated)
-- | Install module into local namespace. If supplied a vector of qualified imports,
-- only load those references. If updated/new, update loaded modules.
This conversation was marked as resolved by emilypi

This comment has been minimized.

Copy link
@slpopejoy

slpopejoy Sep 12, 2019

Contributor

what does the bool arg address?

This comment has been minimized.

Copy link
@emilypi

emilypi Sep 12, 2019

Author Collaborator

The bool arg is the "updated" flag. I'll add a quick note

--
installModule :: Bool -> ModuleData Ref -> Maybe (V.Vector Text) -> Eval e ()
installModule updated md = go . maybe allDefs filteredDefs
where
go f = do
evalRefs . rsLoaded %= HM.union (HM.foldlWithKey' f mempty $ _mdRefMap md)
when updated $
evalRefs . rsLoadedModules %= HM.insert (moduleDefName $ _mdModule md) (md,updated)

filteredDefs is m k v =
if V.elem k is
then HM.insert (Name k def) v m
else m

allDefs m k v = HM.insert (Name k def) v m

msg :: Doc -> Term n
msg = toTerm . renderCompactText'

enscope :: Term Name -> Eval e (Term Ref)
enscope :: Term Name -> Eval e (Term Ref)
enscope t = instantiate' <$> (resolveFreeVars (_tInfo t) . abstract (const Nothing) $ t)

instantiate' :: Scope n Term a -> Term a
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.