Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration tests for boolean and assertion field operations #166

Closed
5 tasks
grjte opened this issue Apr 7, 2022 · 14 comments
Closed
5 tasks

Integration tests for boolean and assertion field operations #166

grjte opened this issue Apr 7, 2022 · 14 comments
Assignees
Labels
good first issue Good for newcomers tests Related to tests

Comments

@grjte
Copy link
Collaborator

grjte commented Apr 7, 2022

Tests for boolean and assertion field operations need to be added to the integration tests.

Tests should follow the existing pattern with the following tests added for each of the boolean field operations and the assert operation:

  • a test of simple success cases
  • a test with applicable failure cases, if there are any (compilation and/or execution failure)
  • a proptest of the expected success case (removed as it's not necessary)

The u32 bitwise tests provide an example.

Specs for the operations are in the Miden Assembly note.

The field operations which are missing tests are:

  • assert
  • not
  • and
  • or
  • xor

More information can also be seen in the field_ops assembly parsers.

@grjte grjte added tests Related to tests good first issue Good for newcomers labels Apr 7, 2022
@aga7hokakological
Copy link
Contributor

I would like to take this issue if no one else is working on this.

@grjte
Copy link
Collaborator Author

grjte commented Apr 29, 2022

@aga7hokakological that would be great! Please feel free to go for it

@aga7hokakological
Copy link
Contributor

My question is regarding the following points:

a test with applicable failure cases, if there are any (compilation and/or execution failure)

Here in below image given failure condition for all the operations which requires test. So does that failure case comes under the above point?

image

@grjte
Copy link
Collaborator Author

grjte commented May 3, 2022

@aga7hokakological yep, exactly. For example, we'd like to have a test that validates that calling not with a value of a > 1 causes the appropriate failure, which should be an ExecutionError.

@grjte
Copy link
Collaborator Author

grjte commented May 17, 2022

@aga7hokakological I just wanted to check in and see if you have any other questions about this task that I can help with. It's also fine if it turns out you don't have time to work on it - please just let us know if that's the case :)

@aga7hokakological
Copy link
Contributor

aga7hokakological commented May 17, 2022

Hey @grjte sorry, I was away for a while. I had figured out about the success cases but for failed cases such as for not condition failes if a > 1. How am I supposed to write the case. because it takes only binary input so no values other than 0 and 1 are allowed.

let test = build_op_test!(asm_op, &[2]); test.expect_error(TestError::ExecutionError("ValueGreaterThanOne"));

@grjte
Copy link
Collaborator Author

grjte commented May 17, 2022

Hey @aga7hokakological no problem. Is the problem that it's throwing a compilation error so it never gets a chance to execute and therefore an execution error is never triggered? If that's the case, I think it's fine to just test for a TestError::AssemblyError here. If that's not the issue can you give me a bit more detail?

@aga7hokakological
Copy link
Contributor

aga7hokakological commented May 17, 2022

@grjte So here,
image

If I do ExecutionError it gives Some(false) != Some(true) and value is not binary value.
image

For AssemblyError it gives None and Some(right)
image

Here ExecutionError is required but as it is not binary value test is failing.

@grjte
Copy link
Collaborator Author

grjte commented May 17, 2022

@aga7hokakological Ok, I see. I think the issue is that you're using the wrong input to yourTestError::ExecutionError. The input should be a substring that will be contained in the expected error returned by the VM. In this case, the expected error is "...NotBinaryValue...", so if you do the following it should work:

build_op_test!(asm_op, &[2]).expect_error(TestError::ExecutionError("NotBinaryValue"));

@aga7hokakological
Copy link
Contributor

aga7hokakological commented May 17, 2022

@grjte Thanks. Got the silly mistake. I have corrected it. For the prop-tests too I have added tests but in there do I need to consider values as u64 or bool?
I have considered bool and all test cases pass but do I need to consider Felt data type? As field operations are different than regular values.
image

Test for assert works.
For assert.eq what is opcode? assert_eq does not work.
image

@0xkanekiken
Copy link
Contributor

Test for assert works.
For assert.eq what is opcode? assert_eq does not work.

The opcode for assert.eq is assert.eq. You can either refer it from here (mentioned in the issue statement) or here

@grjte
Copy link
Collaborator Author

grjte commented May 18, 2022

Thanks @0xkanekiken! Yep, that's correct - in fact, the contents of the assembly note have been migrated to our docs now, so you can check out the assembly docs here: https://maticnetwork.github.io/miden/user_docs/assembly/main.html

@aga7hokakological regarding the first question about the proptests:

  • I think using bool and casting to u64 as you've done would be the right way to do this.
  • You don't need to consider the Felt data type, because that's handled inside the test handler. The test just takes in integers for both inputs and expected result.

You've brought up a good point though - since the only valid inputs are boolean values, proptests are probably overkill here, and we should probably stick to unit tests, since there are only 2 or 4 options to test for each of these anyway. We use proptests for all other tests which is why I had them specified in this issue, but I shouldn't have - sorry for my mistake and the extra work

@aga7hokakological
Copy link
Contributor

aga7hokakological commented May 18, 2022

@grjte and @0xkanekiken Thank you. Below code helped me.

You can either refer it from here (mentioned in the issue statement) or here

you can check out the assembly docs here: https://maticnetwork.github.io/miden/user_docs/assembly/main.html

PR: #210
I made mistake while committing to my branch so I didn't mentioned the PR directly in here.

@grjte
Copy link
Collaborator Author

grjte commented May 21, 2022

Closed by #210

@grjte grjte closed this as completed May 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers tests Related to tests
Projects
None yet
Development

No branches or pull requests

3 participants