New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests fail #137

Closed
uvchik opened this Issue Apr 8, 2016 · 10 comments

Comments

Projects
None yet
2 participants
@uvchik
Copy link
Member

uvchik commented Apr 8, 2016

Using commit 3bea797 two test fail. I just repaired some. Without this you can use commit 1ae046b to revert my changes and get back to the last state without my changes.

I couldn't repair the two test because I'm not sure what is right. It is strange that there is still a postheat component in the actual lp-file of the Two_inputs_one_output component.

It is kind of urgent because I do not want to make further developments on wrong code.

.........F.F....
-----------------------------------------------------------------------------
1) FAIL: Test source.FixedSource with and without investment.

   Traceback (most recent call last):
    /usr/local/lib/python3.4/dist-packages/nose/case.py line 198 in runTest
      self.test(*self.arg)
    tests/constraint_tests.py line 116 in test_source_fixed
      self.compare_lp_files(self.energysystem, "source_fixed.lp")
    tests/constraint_tests.py line 72 in compare_lp_files
      ospath.join("tests", "lp_files", filename)))
   AssertionError: None

   -------------------- >> begin captured stdout << ---------------------
   Warning: Constant objective detected, replacing with a placeholder to prevent solver failure.

   --------------------- >> end captured stdout << ----------------------
   -------------------- >> begin captured logging << --------------------
   root: INFO: Building component constraints.
   root: DEBUG: Creating optimization block for omeof classes: <class 'oemof.core.network.entities.ExcessSlack'>
   root: DEBUG: Creating optimization block for omeof classes: <class 'oemof.core.network.entities.components.sources.FixedSource'>
   root: INFO: Building bus constraints.
   root: INFO: Building objective function.
   root: INFO: LP-file saved to /home/uwe/.oemof/tmp/source_fixed_tmp.lp
   root: INFO: Comparing with file: source_fixed.lp
   --------------------- >> end captured logging << ---------------------

-----------------------------------------------------------------------------
2) FAIL: constraint_tests.Constraint_Tests.test_two_inputs_one_output

   Traceback (most recent call last):
    /usr/local/lib/python3.4/dist-packages/nose/case.py line 198 in runTest
      self.test(*self.arg)
    tests/constraint_tests.py line 158 in test_two_inputs_one_output
      "two_inputs_one_output_invest.lp")
    tests/constraint_tests.py line 72 in compare_lp_files
      ospath.join("tests", "lp_files", filename)))
   AssertionError: None
   -------------------- >> begin captured logging << --------------------
   root: INFO: Building component constraints.
   root: DEBUG: Creating optimization block for omeof classes: <class 'oemof.core.network.entities.components.transformers.TwoInputsOneOutput'>
   root: DEBUG: Set bounds for <class 'oemof.core.network.entities.components.transformers.TwoInputsOneOutput'>.
   root: DEBUG: Set bounds for <class 'oemof.core.network.entities.components.transformers.TwoInputsOneOutput'>.
   root: INFO: Building component constraints.
   root: DEBUG: Creating optimization block for omeof classes: <class 'oemof.core.network.entities.components.transformers.TwoInputsOneOutput'>
   root: DEBUG: Set bounds for <class 'oemof.core.network.entities.components.transformers.TwoInputsOneOutput'>.
   root: DEBUG: Set bounds for <class 'oemof.core.network.entities.components.transformers.TwoInputsOneOutput'>.
   root: INFO: Building bus constraints.
   root: INFO: Building objective function.
   root: INFO: LP-file saved to /home/uwe/.oemof/tmp/two_inputs_one_output_invest_tmp.lp
   root: INFO: Comparing with file: two_inputs_one_output_invest.lp
   --------------------- >> end captured logging << ---------------------


-----------------------------------------------------------------------------
16 tests run in 1.3 seconds. 
2 FAILED (14 tests passed)

@uvchik uvchik added the bug label Apr 8, 2016

@uvchik uvchik added this to the April 2016 release milestone Apr 8, 2016

@uvchik

This comment has been minimized.

Copy link
Member Author

uvchik commented Apr 18, 2016

Here are the version numbers of the main packages involved in the tests (office pc):

matplotlib (1.5.1)
nose (1.3.7)
nose2 (0.4.7)
numpy (1.11.0)
pandas (0.18.0)
Pyomo (4.3.11329)

If I am right the only real difference is in the file for the fixed source. The other "fails" are whitespaces.

@uvchik

This comment has been minimized.

Copy link
Member Author

uvchik commented Apr 19, 2016

At home the versions differ a little but the result is the same:

matplotlib (1.4.2)
nose2 (0.5.0)
Pyomo (4.3.11323)

The rest versions are the same.

I removed some white-spaces, which causes difference between the lp-files (I wonder if it is possible to compare files and ignore additional white-spaces). Maybe now the test will fail for you, after I removed the white-spaces.

@gnn : You can find my changes in the fix/issue_137 branch.

@uvchik

This comment has been minimized.

Copy link
Member Author

uvchik commented Apr 19, 2016

The actual output of the test:

.........F.F....
-----------------------------------------------------------------------------
1) FAIL: Test source.FixedSource with and without investment.

   Traceback (most recent call last):
    /usr/local/lib/python3.4/dist-packages/nose/case.py line 198 in runTest
      self.test(*self.arg)
    tests/constraint_tests.py line 116 in test_source_fixed
      self.compare_lp_files(self.energysystem, "source_fixed.lp")
    tests/constraint_tests.py line 72 in compare_lp_files
      ospath.join("tests", "lp_files", filename)))
   AssertionError: None

   -------------------- >> begin captured stdout << ---------------------
   Warning: Constant objective detected, replacing with a placeholder to prevent solver failure.

   --------------------- >> end captured stdout << ----------------------
   -------------------- >> begin captured logging << --------------------
   root: INFO: Building component constraints.
   root: DEBUG: Creating optimization block for omeof classes: <class 'oemof.core.network.entities.ExcessSlack'>
   root: DEBUG: Creating optimization block for omeof classes: <class 'oemof.core.network.entities.components.sources.FixedSource'>
   root: INFO: Building bus constraints.
   root: INFO: Building objective function.
   root: INFO: LP-file saved to /home/uwe/.oemof/tmp/source_fixed_tmp.lp
   root: INFO: Comparing with file: source_fixed.lp
   --------------------- >> end captured logging << ---------------------

-----------------------------------------------------------------------------
2) FAIL: constraint_tests.Constraint_Tests.test_two_inputs_one_output

   Traceback (most recent call last):
    /usr/local/lib/python3.4/dist-packages/nose/case.py line 198 in runTest
      self.test(*self.arg)
    tests/constraint_tests.py line 164 in test_two_inputs_one_output
      "two_inputs_one_output.lp")
    tests/constraint_tests.py line 72 in compare_lp_files
      ospath.join("tests", "lp_files", filename)))
   AssertionError: None

   -------------------- >> begin captured stdout << ---------------------
   Warning: Constant objective detected, replacing with a placeholder to prevent solver failure.

   --------------------- >> end captured stdout << ----------------------
   -------------------- >> begin captured logging << --------------------
   root: INFO: Building component constraints.
   root: DEBUG: Creating optimization block for omeof classes: <class 'oemof.core.network.entities.components.transformers.TwoInputsOneOutput'>
   root: DEBUG: Set bounds for <class 'oemof.core.network.entities.components.transformers.TwoInputsOneOutput'>.
   root: DEBUG: Set bounds for <class 'oemof.core.network.entities.components.transformers.TwoInputsOneOutput'>.
   root: INFO: Building component constraints.
   root: DEBUG: Creating optimization block for omeof classes: <class 'oemof.core.network.entities.components.transformers.TwoInputsOneOutput'>
   root: DEBUG: Set bounds for <class 'oemof.core.network.entities.components.transformers.TwoInputsOneOutput'>.
   root: DEBUG: Set bounds for <class 'oemof.core.network.entities.components.transformers.TwoInputsOneOutput'>.
   root: INFO: Building bus constraints.
   root: INFO: Building objective function.
   root: INFO: LP-file saved to /home/uwe/.oemof/tmp/two_inputs_one_output_invest_tmp.lp
   root: INFO: Comparing with file: two_inputs_one_output_invest.lp
   root: INFO: Building component constraints.
   root: DEBUG: Creating optimization block for omeof classes: <class 'oemof.core.network.entities.components.transformers.TwoInputsOneOutput'>
   root: DEBUG: Set bounds for <class 'oemof.core.network.entities.components.transformers.TwoInputsOneOutput'>.
   root: DEBUG: Set bounds for <class 'oemof.core.network.entities.components.transformers.TwoInputsOneOutput'>.
   root: INFO: Building bus constraints.
   root: INFO: Building objective function.
   root: INFO: LP-file saved to /home/uwe/.oemof/tmp/two_inputs_one_output_tmp.lp
   root: INFO: Comparing with file: two_inputs_one_output.lp
   --------------------- >> end captured logging << ---------------------


-----------------------------------------------------------------------------
16 tests run in 1.3 seconds. 
2 FAILED (14 tests passed)
@gnn

This comment has been minimized.

Copy link
Member

gnn commented Apr 21, 2016

You are indeed correct. After your whitespace changes, the tests did indeed fail for me. I added diff output to lp-file comparison test failure messages, changed the lp-file comparison to ignore trailing whitespace and removed the logging output because it was distracting me.
After that, all tests did pass for me again. I also figured out that my pyomo version was older than yours so after updating it and re-running the tests, they started to fail:

.........F.F....
======================================================================
FAIL: Test source.FixedSource with and without investment.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/gnn/work/dev/envs/renpassgis/lib/python3.5/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/home/gnn/work/dev/code/oemof_base/tests/constraint_tests.py", line 133, in test_source_fixed
    self.compare_lp_files(self.energysystem, "source_fixed.lp")
  File "/home/gnn/work/dev/code/oemof_base/tests/constraint_tests.py", line 89, in compare_lp_files
    lineterm="")))
AssertionError: Failed matching expected with generated lp file:
--- tests/lp_files/source_fixed.lp
+++ source_fixed_tmp.lp
@@ -1,7 +1,8 @@
 \* Source Pyomo model name=unknown *\

 min
-objective: +0.0 ONE_VAR_CONSTANT
+objective:
++20000000 ONE_VAR_CONSTANT

 s.t.

-------------------- >> begin captured stdout << ---------------------
Warning: Constant objective detected, replacing with a placeholder to prevent solver failure.

--------------------- >> end captured stdout << ----------------------

======================================================================
FAIL: constraint_tests.Constraint_Tests.test_two_inputs_one_output
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/gnn/work/dev/envs/renpassgis/lib/python3.5/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/home/gnn/work/dev/code/oemof_base/tests/constraint_tests.py", line 181, in test_two_inputs_one_output
    "two_inputs_one_output.lp")
  File "/home/gnn/work/dev/code/oemof_base/tests/constraint_tests.py", line 89, in compare_lp_files
    lineterm="")))
AssertionError: Failed matching expected with generated lp file:
--- tests/lp_files/two_inputs_one_output.lp
+++ two_inputs_one_output_tmp.lp
@@ -1,7 +1,8 @@
 \* Source Pyomo model name=unknown *\

 min
-objective: +0.0 ONE_VAR_CONSTANT
+objective:
++0 ONE_VAR_CONSTANT

 s.t.

-------------------- >> begin captured stdout << ---------------------
Warning: Constant objective detected, replacing with a placeholder to prevent solver failure.

--------------------- >> end captured stdout << ----------------------

----------------------------------------------------------------------
Ran 16 tests in 0.848s

FAILED (failures=2)

So the verdict is: the code is correct but pyomo changed how it works.
Possible solutions:

  • manually add an objective function so pyomo doesn't do it or
  • detect the warning and ignore objective:... lines if present.
@uvchik

This comment has been minimized.

Copy link
Member Author

uvchik commented Apr 22, 2016

Couldn't we just remove all lines from the file that contain the phrase objective: or ONE_VAR_CONSTANT and than compare it.

Or remove everthing between min and s.t..

Is it difficult or slow to remove lines from a file in python?

@gnn

This comment has been minimized.

Copy link
Member

gnn commented Apr 22, 2016

Neither difficult, nor slow. This is exactly what the second suggestion would do. Only problem: we shouldn't for all tests, because what happens if we actually want to test that the objective function is constructed correctly. But I've got an idea.
Gonna try to implement that real quick.

@uvchik

This comment has been minimized.

Copy link
Member Author

uvchik commented Apr 22, 2016

We could pass a parameter objective=False (default: True) to the function that compares the files.

@uvchik

This comment has been minimized.

Copy link
Member Author

uvchik commented Apr 25, 2016

Will you be able to fix it until Wednesday, so that we can merge open branches until Friday (release day)?

@gnn

This comment has been minimized.

Copy link
Member

gnn commented Apr 25, 2016

Done and created a pull request. Try it out and merge if you approve and it works. Wasn't too hard, just tried to do it more complicated than necessary initially. (Narf.)

uvchik added a commit that referenced this issue Apr 26, 2016

@uvchik

This comment has been minimized.

Copy link
Member Author

uvchik commented Apr 26, 2016

See PR #145.

@uvchik uvchik closed this Apr 26, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment