Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 336: Underflow not detected for range length field #337

Merged
merged 4 commits into from
Jul 13, 2020
Merged

Conversation

senier
Copy link
Member

@senier senier commented Jul 10, 2020

This fixes cases where negative fields length were not detected. The reason was a misconception of what we (need to) proof here. The attempted proof we had in the code so far was something like (H1 and H2 being the constraints from the range type):

H1: Length >= 40
H2: Length <= 1500
GOAL: Length - 50 >= 0

We then checked whether a solution could be found:

if proof.result != ProofResult.sat:
   ... # Error

What this does is checking whether one solution exists for the goal, which is clearly the case for Length >= 50. What we actually want and what is implemented now is proving that Length - 50 < 0 is unsatisfiable.

Why wouldn't our tests detect this issue? The error message for a different check (that the start of the field is non-negative) was apparently copied from the length check, but not changed. In addition to the length issue, the test case also had a negative start which triggered that second erroneous error message. As it looked as expected, the ineffectiveness of the length check remained unnoticed. The non-negative test was apparently also copied and checked for the wrong message. Hence, the invalid message remained unnoticed.

Lessons learned:

  1. Don't copy and paste tests (without careful review)
  2. Don't copy and paste checks (without careful review)
  3. Proofs show that "there exists one solution" for your formula - if you want to show that a formula never is true, show that its negation is unsatisfiable
  4. We need to review the other checks we do regarding 3.

Other changes:

Since upgrading to isort 4.3.21 via pip3 on Debian, third-party packages are not detected anymore. As a result, isort erroneously removes a lot of whitespace. I added our third-party libs to the isort config, which solves the issue for now and also is more robust if a dependency is not installed.

Close #336

Alexander Senier added 2 commits July 10, 2020 10:39
isort 4.3.21 on Debian (installed via pip3) seems to be broken and does
not detect the third party sources. As a consequence, it formats all
sources to remove the blank line between those imports and our local
imports. Specifying them in the config fixes the issue and also makes
isort more robust in cases where the dependencies are not installed.
@senier senier requested a review from treiher July 10, 2020 09:06
@senier senier added this to In progress in RecordFlux 0.4.1 via automation Jul 10, 2020
@senier senier requested a review from jklmnn July 10, 2020 09:06
jklmnn
jklmnn previously approved these changes Jul 10, 2020
.isort.cfg Outdated Show resolved Hide resolved
tests/test_verification.py Outdated Show resolved Hide resolved
tests/test_verification.py Show resolved Hide resolved
tests/test_model.py Outdated Show resolved Hide resolved
- Add more known third party packages to .isort.cfg
- Move field length test to test_verification
- Improve field length test to not only contain invalid values

Ref. #336, #337
@codecov
Copy link

codecov bot commented Jul 12, 2020

Codecov Report

Merging #337 into develop will not change coverage.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff            @@
##           develop     #337   +/-   ##
========================================
  Coverage    97.95%   97.95%           
========================================
  Files           25       25           
  Lines         4447     4447           
  Branches       746      745    -1     
========================================
  Hits          4356     4356           
  Misses          54       54           
  Partials        37       37           
Impacted Files Coverage Δ
rflx/model.py 99.74% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b5e031f...9fe9ffb. Read the comment docs.

jklmnn
jklmnn previously approved these changes Jul 13, 2020
tests/test_verification.py Outdated Show resolved Hide resolved
senier pushed a commit that referenced this pull request Jul 13, 2020
@senier senier requested a review from jklmnn July 13, 2020 14:15
@senier senier merged commit 503c7b4 into develop Jul 13, 2020
RecordFlux 0.4.1 automation moved this from In progress to Done Jul 13, 2020
@senier senier deleted the issue_336 branch July 13, 2020 15:44
senier pushed a commit that referenced this pull request Jul 13, 2020
- Add more known third party packages to .isort.cfg
- Move field length test to test_verification
- Improve field length test to not only contain invalid values

Ref. #336, #337
@treiher treiher removed this from Done in RecordFlux 0.4.1 Jul 14, 2020
treiher pushed a commit that referenced this pull request Jul 23, 2020
- Add more known third party packages to .isort.cfg
- Move field length test to test_verification
- Improve field length test to not only contain invalid values

Ref. #336, #337
treiher pushed a commit that referenced this pull request Jul 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Underflow not detected for range length field
3 participants