This section gives an overview of commonly used scripting commands when working with Pyomo models. These commands must be applied to a concrete model instance or in other words an instantiated model.
>>> import pyomo.environ as pyo >>> from pyomo.opt import SolverFactory >>> model = pyo.ConcreteModel() >>> model.nVars = pyo.Param(initialize=4) >>> model.N = pyo.RangeSet(model.nVars) >>> model.x = pyo.Var(model.N, within=pyo.Binary) >>> model.obj = pyo.Objective(expr=pyo.summation(model.x)) >>> model.cuts = pyo.ConstraintList() >>> opt = SolverFactory('glpk') >>> opt.solve(model) # doctest: +SKIP
>>> # Iterate, adding a cut to exclude the previously found solution >>> for i in range(5): ... expr = 0 ... for j in model.x: ... if pyo.value(model.x[j]) < 0.5: ... expr += model.x[j] ... else: ... expr += (1 - model.x[j]) ... model.cuts.add( expr >= 1 ) ... results = opt.solve(model) ... print ("n===== iteration",i) ... model.display() # doctest: +SKIP
To illustrate Python scripts for Pyomo we consider an example that is in the file iterative1.py
and is executed using the command
python iterative1.py
Note
This is a Python script that contains elements of Pyomo, so it is executed using the python
command. The pyomo
command can be used, but then there will be some strange messages at the end when Pyomo finishes the script and attempts to send the results to a solver, which is what the pyomo
command does.
This script creates a model, solves it, and then adds a constraint to preclude the solution just found. This process is repeated, so the script finds and prints multiple solutions. The particular model it creates is just the sum of four binary variables. One does not need a computer to solve the problem or even to iterate over solutions. This example is provided just to illustrate some elementary aspects of scripting.
src/scripting/iterative1.spy
Let us now analyze this script. The first line is a comment that happens to give the name of the file. This is followed by two lines that import symbols for Pyomo. The pyomo namespace is imported as pyo
. Therefore, pyo.
must precede each use of a Pyomo name.
src/scripting/iterative1_Import_symbols_for_pyomo.spy
An object to perform optimization is created by calling SolverFactory
with an argument giving the name of the solver. The argument would be 'gurobi'
if, e.g., Gurobi was desired instead of glpk:
src/scripting/iterative1_Call_SolverFactory_with_argument.spy
The next lines after a comment create a model. For our discussion here, we will refer to this as the base model because it will be extended by adding constraints later. (The words "base model" are not reserved words, they are just being introduced for the discussion of this example). There are no constraints in the base model, but that is just to keep it simple. Constraints could be present in the base model. Even though it is an abstract model, the base model is fully specified by these commands because it requires no external data:
src/scripting/iterative1_Create_base_model.spy
The next line is not part of the base model specification. It creates an empty constraint list that the script will use to add constraints.
src/scripting/iterative1_Create_empty_constraint_list.spy
The next non-comment line creates the instantiated model and refers to the instance object with a Python variable instance
. Models run using the pyomo
script do not typically contain this line because model instantiation is done by the pyomo
script. In this example, the create
function is called without arguments because none are needed; however, the name of a file with data commands is given as an argument in many scripts.
src/scripting/iterative1_Create_instantiated_model.spy
The next line invokes the solver and refers to the object contain results with the Python variable results
.
src/scripting/iterative1_Solve_and_refer_to_results.spy
The solve function loads the results into the instance, so the next line writes out the updated values.
src/scripting/iterative1_Display_updated_value.spy
The next non-comment line is a Python iteration command that will successively assign the integers from 0 to 4 to the Python variable i
, although that variable is not used in script. This loop is what causes the script to generate five more solutions:
src/scripting/iterative1_Assign_integers.spy
An expression is built up in the Python variable named expr
. The Python variable j
will be iteratively assigned all of the indexes of the variable x
. For each index, the value of the variable (which was loaded by the load
method just described) is tested to see if it is zero and the expression in expr
is augmented accordingly. Although expr
is initialized to 0 (an integer), its type will change to be a Pyomo expression when it is assigned expressions involving Pyomo variable objects:
src/scripting/iterative1_Iteratively_assign_and_test.spy
During the first iteration (when i
is 0), we know that all values of x
will be 0, so we can anticipate what the expression will look like. We know that x
is indexed by the integers from 1 to 4 so we know that j
will take on the values from 1 to 4 and we also know that all value of x
will be zero for all indexes so we know that the value of expr
will be something like
0 + instance.x[1] + instance.x[2] + instance.x[3] + instance.x[4]
The value of j
will be evaluated because it is a Python variable; however, because it is a Pyomo variable, the value of instance.x[j]
not be used, instead the variable object will appear in the expression. That is exactly what we want in this case. When we wanted to use the current value in the if
statement, we used the value
function to get it.
The next line adds to the constraint list called c
the requirement that the expression be greater than or equal to one:
src/scripting/iterative1_Add_expression_constraint.spy
The proof that this precludes the last solution is left as an exerise for the reader.
The final lines in the outer for loop find a solution and display it:
src/scripting/iterative1_Find_and_display_solution.spy
Note
The assignment of the solve output to a results object is somewhat anachronistic. Many scripts just use
>>> opt.solve(instance) # doctest: +SKIP
since the results are moved to the instance by default, leaving the results object with little of interest. If, for some reason, you want the results to stay in the results object and not be moved to the instance, you would use
>>> results = opt.solve(instance, load_solutions=False) # doctest: +SKIP
This approach can be useful if there is a concern that the solver did not terminate with an optimal solution. For example,
>>> results = opt.solve(instance, load_solutions=False) # doctest: +SKIP >>> if results.solver.termination_condition == TerminationCondition.optimal: # doctest: +SKIP ... instance.solutions.load_from(results) # doctest: +SKIP
The iterative1.py
example above illustrates how a model can be changed and then re-solved. In that example, the model is changed by adding a constraint, but the model could also be changed by altering the values of parameters. Note, however, that in these examples, we make the changes to the concrete model instances. This is particularly important for AbstractModel
users, as this implies working with the instance
object rather than the model
object, which allows us to avoid creating a new model
object for each solve. Here is the basic idea for users of an AbstractModel
:
- Create an
AbstractModel
(suppose it is calledmodel
) - Call
model.create_instance()
to create an instance (suppose it is calledinstance
) - Solve
instance
- Change something in
instance
- Solve
instance
again
Note
Users of ConcreteModel
typically name their models model
, which can cause confusion to novice readers of documentation. Examples based on an AbstractModel
will refer to instance
where users of a ConcreteModel
would typically use the name model
.
# code from spy4scripts import pyomo.environ as pyo
instance = pyo.ConcreteModel() instance.I = pyo.Set(initialize=[1,2,3]) instance.sigma = pyo.Param(mutable=True, initialize=2.3) instance.Theta = pyo.Param(instance.I, mutable=True) for i in instance.I: instance.Theta[i] = i idx = 1 NewVal = 1134
instance.x = pyo.Var([1,2,3], initialize=0) instance.y = pyo.Var() instance.iVar = pyo.Var([1,2,3], initialize=1, domain=pyo.Boolean) instance.sVar = pyo.Var(initialize=1, domain=pyo.Boolean)
If instance
has a parameter whose name is Theta
that was declared to be mutable
(i.e., mutable=True
) with an index that contains idx
, then the value in NewVal
can be assigned to it using
>>> instance.Theta[idx] = NewVal
For a singleton parameter named sigma
(i.e., if it is not indexed), the assignment can be made using
>>> instance.sigma = NewVal
Note
If the Param
is not declared to be mutable, an error will occur if an assignment to it is attempted.
For more information about access to Pyomo parameters, see the section in this document on Param
access ParamAccess
. Note that for concrete models, the model is the instance.
Instead of changing model data, scripts are often used to fix variable values. The following example illustrates this.
src/scripting/iterative2.spy
In this example, the variables are binary. The model is solved and then the value of model.x[2]
is flipped to the opposite value before solving the model again. The main lines of interest are:
src/scripting/iterative2_Flip_value_before_solve_again.spy
This could also have been accomplished by setting the upper and lower bounds:
>>> if instance.x[2].value == 0: ... instance.x[2].setlb(1) ... instance.x[2].setub(1) ... else: ... instance.x[2].setlb(0) ... instance.x[2].setub(0)
Notice that when using the bounds, we do not set fixed
to True
because that would fix the variable at whatever value it presently has and then the bounds would be ignored by the solver.
For more information about access to Pyomo variables, see the section in this document on Var
access VarAccess
.
Note that
>>> instance.x.fix(1)
is equivalent to
>>> instance.x.value = 1 >>> instance.x.fixed = True
- and
>>> instance.x.fix()
is equivalent to
>>> instance.x.fixed = True
One can add terms to an objective function of a ConcreteModel
(or and instantiated AbstractModel
) using the expr
attribute of the objective function object. Here is a simple example:
>>> import pyomo.environ as pyo >>> from pyomo.opt import SolverFactory
>>> model = pyo.ConcreteModel()
>>> model.x = pyo.Var(within=pyo.PositiveReals) >>> model.y = pyo.Var(within=pyo.PositiveReals)
>>> model.sillybound = pyo.Constraint(expr = model.x + model.y <= 2)
>>> model.obj = pyo.Objective(expr = 20 * model.x)
>>> opt = SolverFactory('glpk') # doctest: +SKIP >>> opt.solve(model) # doctest: +SKIP
>>> model.pprint() # doctest: +SKIP
>>> print ("------------- extend obj --------------") # doctest: +SKIP >>> model.obj.expr += 10 * model.y
>>> opt.solve(model) # doctest: +SKIP >>> model.pprint() # doctest: +SKIP
Multiple objectives can be declared, but only one can be active at a time (at present, Pyomo does not support any solvers that can be given more than one objective). If both model.obj1
and model.obj2
have been declared using Objective
, then one can ensure that model.obj2
is passed to the solver as shown in this simple example:
>>> model = pyo.ConcreteModel() >>> model.obj1 = pyo.Objective(expr = 0) >>> model.obj2 = pyo.Objective(expr = 0)
>>> model.obj1.deactivate() >>> model.obj2.activate()
For abstract models this would be done prior to instantiation or else the activate
and deactivate
calls would be on the instance rather than the model.
Constraints can be temporarily disabled using the deactivate()
method. When the model is sent to a solver inactive constraints are not included. Disabled constraints can be re-enabled using the activate()
method.
>>> model = pyo.ConcreteModel() >>> model.v = pyo.Var() >>> model.con = pyo.Constraint(expr=model.v**2 + model.v >= 3) >>> model.con.deactivate() >>> model.con.activate()
Indexed constraints can be deactivated/activated as a whole or by individual index:
>>> model = pyo.ConcreteModel() >>> model.s = pyo.Set(initialize=[1,2,3]) >>> model.v = pyo.Var(model.s) >>> def _con(m, s): ... return m.v[s]**2 + m.v[s] >= 3 >>> model.con = pyo.Constraint(model.s, rule=_con) >>> model.con.deactivate() # Deactivate all indices >>> model.con[1].activate() # Activate single index
Often, the point of optimization is to get optimal values of variables. Some users may want to process the values in a script. We will describe how to access a particular variable from a Python script as well as how to access all variables from a Python script and from a callback. This should enable the reader to understand how to get the access that they desire. The Iterative example given above also illustrates access to variable values.
Assuming the model has been instantiated and solved and the results have been loaded back into the instance object, then we can make use of the fact that the variable is a member of the instance object and its value can be accessed using its value
member. For example, suppose the model contains a variable named quant
that is a singleton (has no indexes) and suppose further that the name of the instance object is instance
. Then the value of this variable can be accessed using pyo.value(instance.quant)
. Variables with indexes can be referenced by supplying the index.
Consider the following very simple example, which is similar to the iterative example. This is a concrete model. In this example, the value of x[2]
is accessed.
src/scripting/noiteration1.py
Note
If this script is run without modification, Pyomo is likely to issue a warning because there are no constraints. The warning is because some solvers may fail if given a problem instance that does not have any constraints.
As with one variable, we assume that the model has been instantiated and solved. Assuming the instance object has the name instance
, the following code snippet displays all variables and their values:
>>> for v in instance.component_objects(pyo.Var, active=True): ... print("Variable",v) # doctest: +SKIP ... for index in v: ... print (" ",index, pyo.value(v[index])) # doctest: +SKIP
Alternatively,
>>> for v in instance.component_data_objects(pyo.Var, active=True): ... print(v, pyo.value(v)) # doctest: +SKIP
This code could be improved by checking to see if the variable is not indexed (i.e., the only index value is None
), then the code could print the value without the word None
next to it.
Assuming again that the model has been instantiated and solved and the results have been loaded back into the instance object. Here is a code snippet for fixing all integers at their current value:
>>> for var in instance.component_data_objects(pyo.Var, active=True): ... if not var.is_continuous(): ... print ("fixing "+str(v)) # doctest: +SKIP ... var.fixed = True # fix the current value
Another way to access all of the variables (particularly if there are blocks) is as follows (this particular snippet assumes that instead of import pyomo.environ as pyo from pyo.environ import * was used):
src/scripting/block_iter_example_compprintloop.spy
Accessing parameter values is completely analogous to accessing variable values. For example, here is a code snippet to print the name and value of every Parameter in a model:
>>> for parmobject in instance.component_objects(pyo.Param, active=True): ... nametoprint = str(str(parmobject.name)) ... print ("Parameter ", nametoprint) # doctest: +SKIP ... for index in parmobject: ... vtoprint = pyo.value(parmobject[index]) ... print (" ",index, vtoprint) # doctest: +SKIP
Access to dual values in scripts is similar to accessing primal variable values, except that dual values are not captured by default so additional directives are needed before optimization to signal that duals are desired.
To get duals without a script, use the pyomo
option --solver-suffixes='dual'
which will cause dual values to be included in output. Note: In addition to duals (dual
) , reduced costs (rc
) and slack values (slack
) can be requested. All suffixes can be requested using the pyomo
option --solver-suffixes='.*'
Warning
Some of the duals may have the value None
, rather than 0
.
To signal that duals are desired, declare a Suffix component with the name "dual" on the model or instance with an IMPORT or IMPORT_EXPORT direction.
src/scripting/driveabs2_Create_dual_suffix_component.spy
See the section on Suffixes Suffixes
for more information on Pyomo's Suffix component. After the results are obtained and loaded into an instance, duals can be accessed in the following fashion.
src/scripting/driveabs2_Access_all_dual.spy
The following snippet will only work, of course, if there is a constraint with the name AxbConstraint
that has and index, which is the string Film
.
src/scripting/driveabs2_Access_one_dual.spy
Here is a complete example that relies on the file abstract2.py
to provide the model and the file abstract2.dat
to provide the data. Note that the model in abstract2.py
does contain a constraint named AxbConstraint
and abstract2.dat
does specify an index for it named Film
.
src/scripting/driveabs2.spy
Concrete models are slightly different because the model is the instance. Here is a complete example that relies on the file concrete1.py
to provide the model and instantiate it.
src/scripting/driveconc1.py
The functions lslack()
and uslack()
return the upper and lower slacks, respectively, for a constraint.
After a solve, the results object has a member Solution.Status
that contains the solver status. The following snippet shows an example of access via a print
statement:
src/scripting/spy4scripts_Print_solver_status.spy
The use of the Python str
function to cast the value to a be string makes it easy to test it. In particular, the value 'optimal' indicates that the solver succeeded. It is also possible to access Pyomo data that can be compared with the solver status as in the following code snippet:
src/scripting/spy4scripts_Pyomo_data_comparedwith_solver_status_1.spy
Alternatively,
src/scripting/spy4scripts_Pyomo_data_comparedwith_solver_status_2.spy
To see the output of the solver, use the option tee=True
as in
src/scripting/spy4scripts_See_solver_output.spy
This can be useful for troubleshooting solver difficulties.
Most solvers accept options and Pyomo can pass options through to a solver. In scripts or callbacks, the options can be attached to the solver object by adding to its options dictionary as illustrated by this snippet:
src/scripting/spy4scripts_Add_option_to_solver.spy
If multiple options are needed, then multiple dictionary entries should be added.
Sometimes it is desirable to pass options as part of the call to the solve function as in this snippet:
src/scripting/spy4scripts_Add_multiple_options_to_solver.spy
The quoted string is passed directly to the solver. If multiple options need to be passed to the solver in this way, they should be separated by a space within the quoted string. Notice that tee
is a Pyomo option and is solver-independent, while the string argument to options
is passed to the solver without very little processing by Pyomo. If the solver does not have a "threads" option, it will probably complain, but Pyomo will not.
There are no default values for options on a SolverFactory
object. If you directly modify its options dictionary, as was done above, those options will persist across every call to optimizer.solve(…)
unless you delete them from the options dictionary. You can also pass a dictionary of options into the opt.solve(…)
method using the options
keyword. Those options will only persist within that solve and temporarily override any matching options in the options dictionary on the solver object.
Often, the executables for solvers are in the path; however, for situations where they are not, the SolverFactory function accepts the keyword executable
, which you can use to set an absolute or relative path to a solver executable. E.g.,
src/scripting/spy4scripts_Set_path_to_solver_executable.spy
Some solvers support a warm start based on current values of variables. To use this feature, set the values of variables in the instance and pass warmstart=True
to the solve()
method. E.g.,
src/scripting/spy4scripts_Pass_warmstart_to_solver.spy
Note
The Cplex and Gurobi LP file (and Python) interfaces will generate an MST file with the variable data and hand this off to the solver in addition to the LP file.
Warning
Solvers using the NL file interface (e.g., "gurobi_ampl", "cplexamp") do not accept warmstart as a keyword to the solve() method as the NL file format, by default, includes variable initialization data (drawn from the current value of all variables).
Building and solving Pyomo models in parallel is a common requirement for many applications. We recommend using MPI for Python (mpi4py) for this purpose. For more information on mpi4py, see the mpi4py documentation (https://mpi4py.readthedocs.io/en/stable/). The example below demonstrates how to use mpi4py to solve two pyomo models in parallel. The example can be run with the following command:
mpirun -np 2 python -m mpi4py parallel.py
src/scripting/parallel.py
A "temporary" directory is used for many intermediate files. Normally, the name of the directory for temporary files is provided by the operating system, but the user can specify their own directory name. The pyomo command-line --tempdir
option propagates through to the TempFileManager service. One can accomplish the same through the following few lines of code in a script:
src/scripting/spy4scripts_Specify_temporary_directory_name.spy