Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance of solve_strongly_connected_components for models with named expressions #3186

Merged
merged 33 commits into from
Mar 13, 2024

Conversation

Robbybp
Copy link
Contributor

@Robbybp Robbybp commented Mar 7, 2024

Summary/Motivation:

Working with a small IDAES model, I noticed that solve_strongly_connected_components was much slower than I would have expected. This is for a model with about 900 variables and constraints:

Identifier                                    ncalls   cumtime   percall      %                                                                    
-------------------------------------------------------------------------------                                                                    
root                                               1    40.800    40.800  100.0                                                                    
     --------------------------------------------------------------------------                                                                    
     full model post-solve                         1     0.463     0.463    1.1                                                                    
     solve-scc                                     1    40.337    40.337   98.9                                                                    
                          -----------------------------------------------------                                                                    
                          calc-var-from-con      530    20.910     0.039   51.8                                                                    
                          generate-scc           547    11.948     0.022   29.6                                                                    
                          igraph                   1     7.371     7.371   18.3                                                                    
                          scc-subsolver           16     0.100     0.006    0.2                                                                    
                          other                  n/a     0.008       n/a    0.0                                                                    
                          =====================================================                                                                    
     other                                       n/a     0.000       n/a    0.0                                                                    
     ==========================================================================                                                                    
===============================================================================                                                                    

solve_strongly_connected_components takes 40 s, half of which is in calculate_variable_from_constraint. This prompted me to review a lot of the infrastructure that solve_strongly_connected_components relies on and look for ways to speed things up. Most of the performance gains I will demonstrate come from not repeating work for the same named expressions, although reusing incidence graphs instead of walking expressions helps as well.

Changes proposed in this PR:

  • Add an option for solve_strongly_connected_components to avoid using calculate_variable_from_constraint. This can be beneficial for models with large, nested named expressions, which the NL writer exploits but calculate_variable_from_constraint does not.
  • Use IncidenceMethod.ampl_repn when constructing incidence graphs. Again, this exploits named expressions, where IncidenceMethod.standard_repn does not.
  • Reuse the already existing incidence graph for the block triangularization in generate_strongly_connected_components
  • Update _ExternalFunctionVisitor and add_local_external_functions to exploit named expressions
  • Add an option for TemporarySubsystemManager to temporarily remove bounds on fixed variables. This was the motivation for the suggestion in Unavoidable InfeasibleConstraintException in NL writer for variable fixed outside of bounds #3179

The timing breakdown after these improvements looks like:

Identifier                                                           ncalls   cumtime   percall      %                     
------------------------------------------------------------------------------------------------------                     
root                                                                      1     4.159     4.159  100.0                     
     -------------------------------------------------------------------------------------------------                     
     full model post-solve                                                1     0.445     0.445   10.7                     
     solve-scc                                                            1     3.713     3.713   89.3                     
                          ----------------------------------------------------------------------------                     
                          generate-scc                                  547     2.431     0.004   65.5                     
                                       ---------------------------------------------------------------                     
                                       block-triang                       1     0.033     0.033    1.4                     
                                       generate-block                   547     2.397     0.004   98.6                     
                                                     -------------------------------------------------                     
                                                     block              546     0.015     0.000    0.6                     
                                                     external-fcns      546     0.341     0.001   14.2                     
                                                     identify-vars      546     1.999     0.004   83.4                     
                                                     reference         1092     0.036     0.000    1.5                     
                                                     other              n/a     0.007       n/a    0.3                     
                                                     =================================================                     
                                       other                            n/a     0.002       n/a    0.1                     
                                       ===============================================================                     
                          igraph                                          1     0.347     0.347    9.3                     
                          scc-subsolver                                 546     0.922     0.002   24.8                     
                          other                                         n/a     0.012       n/a    0.3                     
                          ============================================================================                     
     other                                                              n/a     0.000       n/a    0.0                     
     =================================================================================================                     
======================================================================================================                     

We are now bottlenecked by identify_variables, of all things.

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

  1. I agree my contributions are submitted under the BSD license.
  2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

@Robbybp Robbybp changed the title Improve performance of solve_strongly_connected_components for models with large expression trees Improve performance of solve_strongly_connected_components for models with named expressions Mar 8, 2024
Copy link
Member

@jsiirola jsiirola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. A couple questions / suggestions, but nothing that would prevent merging

pyomo/util/subsystems.py Outdated Show resolved Hide resolved
pyomo/util/subsystems.py Outdated Show resolved Hide resolved
pyomo/util/subsystems.py Outdated Show resolved Hide resolved
@jsiirola
Copy link
Member

FYI: Jenkins test failures appear to be real:

=================================== FAILURES ===================================
___ TestSubsystemBlock.test_external_function_with_potential_name_collision ____

self = <pyomo.util.tests.test_subsystems.TestSubsystemBlock testMethod=test_external_function_with_potential_name_collision>

    @unittest.skipUnless(find_GSL(), "Could not find the AMPL GSL library")
    def test_external_function_with_potential_name_collision(self):
>       m = self._make_model_with_external_functions()

pyomo/pyomo/util/tests/test_subsystems.py:412: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pyomo/pyomo/util/tests/test_subsystems.py:311: in _make_model_with_external_functions
    subexpr3 = m.subexpr[2] + m.v3**2
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pyomo.core.base.PyomoModel.ConcreteModel object at 0x7f889ebce6c0>
val = 'subexpr'

    def __getattr__(self, val):
        if val in ModelComponentFactory:
            return _component_decorator(self, ModelComponentFactory.get_class(val))
        # Since the base classes don't support getattr, we can just
        # throw the "normal" AttributeError
>       raise AttributeError(
            "'%s' object has no attribute '%s'" % (self.__class__.__name__, val)
        )
E       AttributeError: 'ConcreteModel' object has no attribute 'subexpr'

pyomo/pyomo/core/base/block.py:547: AttributeError
_____________ TestSubsystemBlock.test_identify_external_functions ______________

self = <pyomo.util.tests.test_subsystems.TestSubsystemBlock testMethod=test_identify_external_functions>

    @unittest.skipUnless(find_GSL(), "Could not find the AMPL GSL library")
    def test_identify_external_functions(self):
>       m = self._make_model_with_external_functions()

pyomo/pyomo/util/tests/test_subsystems.py:319: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pyomo/pyomo/util/tests/test_subsystems.py:311: in _make_model_with_external_functions
    subexpr3 = m.subexpr[2] + m.v3**2
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pyomo.core.base.PyomoModel.ConcreteModel object at 0x7f889ebbb930>
val = 'subexpr'

    def __getattr__(self, val):
        if val in ModelComponentFactory:
            return _component_decorator(self, ModelComponentFactory.get_class(val))
        # Since the base classes don't support getattr, we can just
        # throw the "normal" AttributeError
>       raise AttributeError(
            "'%s' object has no attribute '%s'" % (self.__class__.__name__, val)
        )
E       AttributeError: 'ConcreteModel' object has no attribute 'subexpr'

pyomo/pyomo/core/base/block.py:547: AttributeError
___ TestSubsystemBlock.test_local_external_functions_with_named_expressions ____

self = <pyomo.util.tests.test_subsystems.TestSubsystemBlock testMethod=test_local_external_functions_with_named_expressions>

    @unittest.skipUnless(find_GSL(), "Could not find the AMPL GSL library")
    def test_local_external_functions_with_named_expressions(self):
        m = self._make_model_with_external_functions(named_expressions=True)
>       variables = list(pyo.component_data_objects(pyo.Var))

pyomo/pyomo/util/tests/test_subsystems.py:344: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

name = 'component_data_objects'

    def __getattr__(name):
        info = _relocated.get(name, None)
        if info is not None:
            target_obj = _import_object(name, *info)
            f_globals[name] = target_obj
            return target_obj
        elif _mod_getattr is not None:
            return _mod_getattr(name)
>       raise AttributeError(
            "module '%s' has no attribute '%s'" % (f_globals['__name__'], name)
        )
E       AttributeError: module 'pyomo.environ' has no attribute 'component_data_objects'

pyomo/pyomo/common/deprecation.py:439: AttributeError
________________ TestSubsystemBlock.test_with_external_function ________________

self = <pyomo.util.tests.test_subsystems.TestSubsystemBlock testMethod=test_with_external_function>

    @unittest.skipUnless(find_GSL(), "Could not find the AMPL GSL library")
    @unittest.skipUnless(
        pyo.SolverFactory("ipopt").available(), "ipopt is not available"
    )
    def test_with_external_function(self):
>       m = self._make_model_with_external_functions()

pyomo/pyomo/util/tests/test_subsystems.py:361: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pyomo/pyomo/util/tests/test_subsystems.py:311: in _make_model_with_external_functions
    subexpr3 = m.subexpr[2] + m.v3**2
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pyomo.core.base.PyomoModel.ConcreteModel object at 0x7f889ebf2940>
val = 'subexpr'

    def __getattr__(self, val):
        if val in ModelComponentFactory:
            return _component_decorator(self, ModelComponentFactory.get_class(val))
        # Since the base classes don't support getattr, we can just
        # throw the "normal" AttributeError
>       raise AttributeError(
            "'%s' object has no attribute '%s'" % (self.__class__.__name__, val)
        )
E       AttributeError: 'ConcreteModel' object has no attribute 'subexpr'

pyomo/pyomo/core/base/block.py:547: AttributeError
______ TestSubsystemBlock.test_with_external_function_in_named_expression ______

self = <pyomo.util.tests.test_subsystems.TestSubsystemBlock testMethod=test_with_external_function_in_named_expression>

    @unittest.skipUnless(find_GSL(), "Could not find the AMPL GSL library")
    @unittest.skipUnless(
        pyo.SolverFactory("ipopt").available(), "ipopt is not available"
    )
    def test_with_external_function_in_named_expression(self):
        m = self._make_model_with_external_functions(named_expressions=True)
        subsystem = ([m.con2, m.con3], [m.v2, m.v3])
    
        m.v1.set_value(0.5)
        block = create_subsystem_block(*subsystem)
        ipopt = pyo.SolverFactory("ipopt")
        with TemporarySubsystemManager(to_fix=list(block.input_vars.values())):
            ipopt.solve(block)
    
        # Correct values obtained by solving with Ipopt directly
        # in another script.
        self.assertEqual(m.v1.value, 0.5)
        self.assertFalse(m.v1.fixed)
        self.assertAlmostEqual(m.v2.value, 1.04816, delta=1e-5)
        self.assertAlmostEqual(m.v3.value, 1.34356, delta=1e-5)
    
        # Result obtained by solving the full system
>       m_full = self._solve_ef_model_with_ipopt()

pyomo/pyomo/util/tests/test_subsystems.py:405: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pyomo/pyomo/util/tests/test_subsystems.py:351: in _solve_ef_model_with_ipopt
    m = self._make_model_with_external_functions()
pyomo/pyomo/util/tests/test_subsystems.py:311: in _make_model_with_external_functions
    subexpr3 = m.subexpr[2] + m.v3**2
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pyomo.core.base.PyomoModel.ConcreteModel object at 0x7f889ebf2800>
val = 'subexpr'

    def __getattr__(self, val):
        if val in ModelComponentFactory:
            return _component_decorator(self, ModelComponentFactory.get_class(val))
        # Since the base classes don't support getattr, we can just
        # throw the "normal" AttributeError
>       raise AttributeError(
            "'%s' object has no attribute '%s'" % (self.__class__.__name__, val)
        )
E       AttributeError: 'ConcreteModel' object has no attribute 'subexpr'

pyomo/pyomo/core/base/block.py:547: AttributeError
------------------------------ Captured log call -------------------------------
WARNING  pyomo.core:external.py:438 AMPL External function: ignoring AtReset call in external library.  This may result in a memory leak or other undesirable behavior.
WARNING  pyomo.core:external.py:438 AMPL External function: ignoring AtReset call in external library.  This may result in a memory leak or other undesirable behavior.
=========================== short test summary info ============================
FAILED pyomo/pyomo/util/tests/test_subsystems.py::TestSubsystemBlock::test_external_function_with_potential_name_collision - AttributeError: 'ConcreteModel' object has no attribute 'subexpr'
FAILED pyomo/pyomo/util/tests/test_subsystems.py::TestSubsystemBlock::test_identify_external_functions - AttributeError: 'ConcreteModel' object has no attribute 'subexpr'
FAILED pyomo/pyomo/util/tests/test_subsystems.py::TestSubsystemBlock::test_local_external_functions_with_named_expressions - AttributeError: module 'pyomo.environ' has no attribute 'component_data_objects'
FAILED pyomo/pyomo/util/tests/test_subsystems.py::TestSubsystemBlock::test_with_external_function - AttributeError: 'ConcreteModel' object has no attribute 'subexpr'
FAILED pyomo/pyomo/util/tests/test_subsystems.py::TestSubsystemBlock::test_with_external_function_in_named_expression - AttributeError: 'ConcreteModel' object has no attribute 'subexpr'
==== 5 failed, 13172 passed, 4631 skipped, 49 xfailed in 3428.47s (0:57:08) ====

@mrmundt
Copy link
Contributor

mrmundt commented Mar 11, 2024

Real failure again, @Robbybp

self = <pyomo.util.tests.test_subsystems.TestSubsystemBlock testMethod=test_local_external_functions_with_named_expressions>

    @unittest.skipUnless(find_GSL(), "Could not find the AMPL GSL library")
    def test_local_external_functions_with_named_expressions(self):
        m = self._make_model_with_external_functions(named_expressions=True)
>       variables = list(pyo.component_data_objects(pyo.Var))

/pyomo/pyomo/util/tests/test_subsystems.py:344: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

name = 'component_data_objects'

    def __getattr__(name):
        info = _relocated.get(name, None)
        if info is not None:
            target_obj = _import_object(name, *info)
            f_globals[name] = target_obj
            return target_obj
        elif _mod_getattr is not None:
            return _mod_getattr(name)
>       raise AttributeError(
            "module '%s' has no attribute '%s'" % (f_globals['__name__'], name)
        )
E       AttributeError: module 'pyomo.environ' has no attribute 'component_data_objects'

/pyomo/pyomo/common/deprecation.py:439: AttributeError

Copy link

codecov bot commented Mar 12, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.39%. Comparing base (b06ddea) to head (4f1e20c).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3186      +/-   ##
==========================================
- Coverage   88.39%   88.39%   -0.01%     
==========================================
  Files         846      846              
  Lines       95091    95118      +27     
==========================================
+ Hits        84058    84079      +21     
- Misses      11033    11039       +6     
Flag Coverage Δ
linux 86.33% <80.00%> (-0.01%) ⬇️
osx 76.16% <80.00%> (-0.01%) ⬇️
other 86.53% <80.00%> (-0.02%) ⬇️
win 83.82% <80.00%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Robbybp
Copy link
Contributor Author

Robbybp commented Mar 13, 2024

Attention: Patch coverage is 80.00000% with 9 lines in your changes are missing coverage. Please review.

My latest commits should address this.

@mrmundt mrmundt merged commit 1b5aa97 into Pyomo:main Mar 13, 2024
33 checks passed
@Robbybp Robbybp deleted the scc-performance-2 branch March 13, 2024 16:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants