Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
implement RhoSpec contract in Rholang #2100
The RhoSpec contract is just a demo of the assert primitives for now because I don't know yet how to sign and deploy a contract the way TestSet is deployed now. Once I figure that out I will move the implementation to a new file.
Please make sure that this PR:
I'm very sorry to say this, but...
Overall, I don't like where this is going. I'm seriously worried about the safety net the current tests provide, and thus our safety.
Why aren't we adapting the TestSet contract step-by-step so that we see the changed API is usable and better?
I am very much against the total reimplementation of the casper test harness, then leading to a major rewrite of most of the tests. That's an equivalent of re-weaving our safety net while we're working 5 floors above it.
If we're making any changes to the test harness - I'd like them to be incremental and introducing changes step-by-step to the current implementation.
To me, we should start with making TestSet accept comparison descriptions. So instead of:
This alone should be enough to have meaningful assertion messages.
I see it's not your only goal and I think I agree or can relate with some of the choices I think you've deliberately taken. I'd like us to discuss them though before implementing them and then seeing that all the tests we have need to be rewritten from scratch without a remotely clear migration path.
Features of the TestSet API:
We should discuss them before discarding them.
Features I think you seem to target:
All of those should be discussed based on examples from the existing test base before implementing them. Of all of those, I think the least controversial would probably be 'specialized assertion variants'.
I guess we can do the discussion here or move to the wiki. I'm not requiring that we discuss the whole design upfront. My requests are:
I don't think there was much discussion for the old API. I wouldn't treat it as a sacred battle tested piece of code.
Over the past couple months we've been slowly tightening our safety net. At this point, the comm layer, block proposing and merging, the estimator, and contract execution in rholang is pretty solid. Bonding and safety oracle are being tightened up. Casper contracts (including wallets, REV, proof of stake) are still WIP and in some areas still not yet completely defined. Perhaps it is a matter of style but I think we can be a bit more aggressive with casper contracts.
I'm far from that. I'm just sure it's more battle-tested than starting from scratch.
And now we're seemingly on the course to losing it due to errors in a big-bang revamp of the net's foundations.
Who tests the tests, after all? How can we tell the tests still catch the bugs they used to after we've rewritten them?
To be clear these tests have nothing to do with the already solid "comm layer, block proposing and merging, the estimator, and contract execution in rholang".
Besides Either, ListOps, and NonNegativeNumber - I don't see those changing much. Depending on the outcome of the wallet and validator incentive discussions a lot of contracts could go through some drastic change.
I don't understand why there is work in this direction? @odeac haven't we had a meeting regarding this?
I second @ArturGajowy, my main concerns are following:
I'm not argueing against the feature, but timing does not seem right? You might say "well, hell, but the work is done, what's the problem?". my issue is that the work is not completed and there will be another huge effort moving this framework onto the existing test suite. or do you want to keep both?
@ArturGajowy @rabbitonweb I think we are overestimating what is at stake here. This isn't like PR #1831 where we are about to change 3000 lines on areas of code that have gone through 3~4 months of community testing. We've barely tested or even agreed on the final designs of the wallet contracts. Validator incentive contracts are still WIP. Worst case we're only losing the tests that were written for Either, ListOps, and NonNegativeNumber.
Still, the new framework lacks usage examples (not a single test was ported so far) and it's clear porting won't be straightforward due to conflicting design paradigms ('all tests must be isolated' vs 'tests reuse the previous state and can be sequenced by the user'). While clean-slate design has its merits, it only works after identifying the use cases and requirements - and verifying them by examples.
If this is to be useful, it needs a battle test of porting all the existing tests. Given what you said @KentShikama, I'm now comfortable if this means rewriting them largely. But this should be done before we deem the new framework ready. Otherwise the solution is incomplete and only increases maintenance burden.
This PR is about making the Casper Rholang tests easier to debug by reporting insights about why the tests failed because, without such information, it's very difficult to figure out why they failed.
2 times, most recently
Jan 24, 2019
referenced this pull request
Jan 25, 2019
A couple comments, but this looks mostly good.
I didn't check that tests where migrated faithfully, but they look ok and visibly more readable
The 'only concrete terms should be allowed in comparison' comment seems important, as I've no idea what's going to happen if s/b passes a term that's using variables. Neither do the tests.
Please rebase and apply comments. Please also squash the 'remove commented code' commit.