Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unify Match/Unify modules #573

Draft
wants to merge 13 commits into
base: main
Choose a base branch
from
Draft

Conversation

goodlyrottenapple
Copy link
Member

@goodlyrottenapple goodlyrottenapple commented Apr 10, 2024

Will re-open in the haskell backend, just keeping this for the commit history for now

fixes runtimeverification/haskell-backend#3785

@goodlyrottenapple
Copy link
Member Author

Kontrol

Test sam-unify-match-refactor time master-754e6ca time (sam-unify-match-refactor/master-754e6ca) time
PlainPrankTest.test_stopPrank_notExistent() 39.26 53.04 0.7401960784313725
PlainPrankTest.test_startPrank_zeroAddress_true() 136.31 178.45 0.763855421686747
PlainPrankTest.test_prank_expectRevert() 98.25 121.9 0.805988515176374
PlainPrankTest.test_prank_zeroAddress_true() 159.16 195.31 0.8149096308432747
PlainPrankTest.test_startPrank_true() 132.49 160.96 0.8231237574552684
PlainPrankTest.test_startPrankWithOrigin_true() 134.12 161.51 0.8304129775246115
MockCallTestFoundry.testMockCallEmptyAccount() 91.83 102.98 0.8917265488444358
FreshCheatcodes.test_bool() 45.15 47.91 0.9423919849718222
PlainPrankTest.testFail_startPrank_internalCall() 48.46 50.92 0.9516889238020424
AddrTest.test_addr_true()-trace_options1 32.67 34.15 0.956661786237189
PrankTestMsgSender.test_msgsender_setup() 116.15 120.96 0.9602347883597885
AssertTest.checkFail_assert_false() 38.1 39.65 0.960907944514502
AccountParamsTest.testDealConcrete()-trace_options0 32.08 33.35 0.9619190404797601
SetUpTest.testSetupData() 75.86 78.8 0.9626903553299493
BlockParamsTest.testChainId(uint256) 46.52 48.24 0.9643449419568823
AssumeTest.testFail_assume_false(uint256,uint256) 67.06 69.42 0.9660040334197638
ERC721.sol 48.41 46.43 1.0426448416971785
Storage.sol 51.62 49.2 1.0491869918699186
src/tests/integration/test_foundry_prove.py::test_foundry_xml_report 31.26 29.77 1.0500503862949278
AssertTest.test_assert_false() 85.75 80.09 1.070670495692346
MockCallTestFoundry.testMockNested() 432.57 394.76 1.0957797142567636
BMCLoopsTest.test_countdown_concrete() 21.87 19.3 1.133160621761658
TOTAL 1964.9499999999996 2117.1000000000004 0.9281328232015489

Comment on lines 404 to 407
| -- | A sort error was detected during unification
RewriteSortError (RewriteRule k) Term SortError
| -- | A sort error was detected during unification
InternalMatchError Text
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🍝 copy-pasta

Comment on lines 633 to 638
, attributes =
SymbolAttributes
{ collectionMetadata = Nothing
, symbolType = TotalFunction
, symbolType = Function Total
, isIdem = IsNotIdem
, isAssoc = IsNotAssoc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe better to just re-generate (using the repl, and then slightly edit and auto-format) this definition. It is not up to date any more with the source.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should re-think this maybe parsing kore via TH? I would leave out of the scope of this PR at any rate...

library/Booster/Pattern/UnifiedMatcher.hs Outdated Show resolved Hide resolved
library/Booster/Pattern/UnifiedMatcher.hs Outdated Show resolved Hide resolved
library/Booster/Pattern/UnifiedMatcher.hs Outdated Show resolved Hide resolved
Comment on lines +381 to +384
| symbol1.name /= symbol2.name =
failWith
( DifferentSymbols (SymbolApplication symbol1 sorts1 args1) (SymbolApplication symbol2 sorts2 args2)
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit intricate... only correct with the invariant that we only call this case (with Rule) for ConsApplications.
Maybe we could merge the Rule and Fun cases for symbol1.name /= symbol2.name to make this more self-contained? (the Rule case shouldn't ever occur with function symbols, and would yield Indeterminate before)

| -- mirrored case above: left list fully concrete, right one isn't
Nothing <- rest1
, Just _ <- rest2 =
matchLists def heads2 rest2 heads1 rest1 -- won't loop, will fail later if unification succeeds
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about this case...
Since the intent is matching this is probably bound to fail - the idea is that the pattern being fully concrete already means we cannot match a non-concrete list part in the subject but we might have to spell this out more explicitly.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this was in the original unification code which appeared bidirectional, but we know this would eventually fail. Definitely should refactor, but maybe in a followup?

>>= \case
Just newProblems ->
enqueueRegularProblems $ (Seq.fromList $ Map.elems commonMap) >< newProblems
Nothing -> pure ()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about the commonMap in this case?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we have a Nothing that means we added an indeterminate result, in which case I thought we don't want to continue anyway, unless we want to find more indeterminate cases in the commonMap, tho that just seems like busywork.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would there be cases where the commonMap elements still cause an outright failure to match (as opposed to being indeterminate)?

Comment on lines 559 to 567
data MapList
= ConstructorKey Term Term MapList
| Rest RestMapList

pattern SingleConstructorKey :: Term -> Term -> MapList
pattern SingleConstructorKey k v = ConstructorKey k v (Rest Empty)

pattern SingleOtherKey :: Term -> Term -> MapList
pattern SingleOtherKey k v = Rest (OtherKey k v Empty)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer if we didn't bring in a whole new map matching algorithm in the same PR as the refactoring.
It should be possble to port the existing matching/"unification" code without too many adaptations, then we can still rewrite the map-matching algorithm.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unfortunately I tried a few times to make a frankenstein monster out of the existing two versions but could not get the tests to pass in both instances. I also found one or two tests that are just wrong and others that had to be oriented correctly for matching

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That being said, I have yet to run kevm/kontrol with this change so not too sure of it myself atm

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm perhaps i will just keep the two copies for now and try to introduce this in the followup

Comment on lines 766 to 772
-- Check if term is a variable, prefer target variables. Should
-- not happen given how we call it in the code above.
Var var2
| mType == Rule
&& var2 `Set.member` targets
&& not (var `Set.member` targets) ->
bindVariable mType var2 (Var var)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An assumption of the prior "unification" code... we should probably remove targets completely from the code and not care whether or not term is a variable.

@@ -397,10 +399,12 @@ data RewriteFailed k
RuleConditionUnclear (RewriteRule k) Predicate
| -- | A rewrite rule does not preserve definedness
DefinednessUnclear (RewriteRule k) Pattern [NotPreservesDefinednessReason]
| -- | A unification produced a non-match substitution
| -- | A matching produced a non-match substitution
UnificationIsNotMatch (RewriteRule k) Term Substitution
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably need to rename the UnificationIsNotMatch constructor to just IsNotMatch.

match1 _ (AndTerm t1a t1b) t2@FunctionApplication{} = enqueueRegularProblem t1a t2 >> enqueueRegularProblem t1b t2
match1 Rewrite (AndTerm t1a t1b) t2@Var{} = enqueueRegularProblem t1a t2 >> enqueueRegularProblem t1b t2
match1 Eval t1@AndTerm{} t2@Var{} = addIndeterminate t1 t2
match1 Rewrite t1@DomainValue{} (AndTerm t2a t2b) = enqueueRegularProblem t1 t2a >> enqueueRegularProblem t1 t2b
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like how this looks! I'd suggest a minor style change: let's add comments separating the groups of clauses for the different Term patterns:

Suggested change
match1 Rewrite t1@DomainValue{} (AndTerm t2a t2b) = enqueueRegularProblem t1 t2a >> enqueueRegularProblem t1 t2b
-- DomainValue
match1 Rewrite t1@DomainValue{} (AndTerm t2a t2b) = enqueueRegularProblem t1 t2a >> enqueueRegularProblem t1 t2b

match1 _ t1@DomainValue{} t2@FunctionApplication{} = addIndeterminate t1 t2
match1 Rewrite t1@DomainValue{} (Var var2) = matchVar Rewrite var2 t1
match1 Eval t1@DomainValue{} t2@Var{} = addIndeterminate t1 t2
match1 Rewrite t1@Injection{} (AndTerm t2a t2b) = enqueueRegularProblem t1 t2a >> enqueueRegularProblem t1 t2b
Copy link
Contributor

@geo2a geo2a Apr 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As suggested above, consider adding spanners like this:

Suggested change
match1 Rewrite t1@Injection{} (AndTerm t2a t2b) = enqueueRegularProblem t1 t2a >> enqueueRegularProblem t1 t2b
-- Injection
match1 Rewrite t1@Injection{} (AndTerm t2a t2b) = enqueueRegularProblem t1 t2a >> enqueueRegularProblem t1 t2b

Comment on lines +354 to +357
----- Symbol Applications
-- two constructors: fail if names differ, recurse

----- Symbol Applications
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
----- Symbol Applications
-- two constructors: fail if names differ, recurse
----- Symbol Applications
----- Symbol Applications

[] -> pure ()
(k, _) : _ -> failWith $ DuplicateKeys k m
checkDuplicateKeys _ = pure ()
matchMaps _ _ _ = undefined
Copy link
Contributor

@geo2a geo2a Apr 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably an overlooked case, or should it "never happen"? Either way, probably a good idea to return indeterminate here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is getting a refactor in the followup PR anyway

goodlyrottenapple added a commit to runtimeverification/haskell-backend that referenced this pull request Apr 15, 2024
Fixes #3785 by merging the two versions of our matching algorithm, one
used in matching rewrite rules and another matching
function/simplification rules. This is a first pass which mostly just
merges the two algorithms, prserving most of the divergent behaviour
under the `Rewrite`/`Eval` `MatchType` flag. A followup refactor should
clean up this code further.

This PR is a port of
runtimeverification/hs-backend-booster#573 where
the development and testing happened.

---------

Co-authored-by: github-actions <github-actions@github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Merge our unification and matching algorithms
3 participants