Verbose logging specific to compilation

[akshayka]

Adds a keyword argument verbose to the solve method. When
verbose=True, CVXPY will log information related to both problem
compilation and solving.

For beginners, the distinction between compilation and solving is sometimes not
clear; this can lead some users to conclude that the solver is taking
too long, when compilation was the bottleneck (or vice versa). This
logging is meant to help alleviate that confusion.

For beginners and advanced users alike, when compiling very large
problems, it can be helpful to have logging output describing the
various stages of compilation and noting when they finish.

Sample output:

-------------------------------------------------------------------------------
                                     CVXPY                                      
                                    v1.1.10                                     
-------------------------------------------------------------------------------
Feb 26 05:54:27 PM: Your problem with 20 variables, 2 constraints, and 0 parameters.
Feb 26 05:54:27 PM: (If you need to solve this problem multiple times, but with different data, consider using parameters.)         
Feb 26 05:54:27 PM: It is compliant with the following grammars: DCP, DQCP
Feb 26 05:54:27 PM: CVXPY will first compile your problem; then, it will invoke a numerical solver to obtain a solution.    
Feb 26 05:54:27 PM: Compiling problem (target solver=OSQP).
Feb 26 05:54:27 PM: Reduction chain: CvxAttr2Constr, Qp2SymbolicQp, QpMatrixStuffing, OSQP
Feb 26 05:54:27 PM: Applying reduction CvxAttr2Constr                                                                                    
Feb 26 05:54:27 PM: Applying reduction Qp2SymbolicQp                                                                                     
Feb 26 05:54:27 PM: Applying reduction QpMatrixStuffing
Feb 26 05:54:27 PM: Applying reduction OSQP                                                                                                                                                                                                                                       
Feb 26 05:54:27 PM: Finished problem compilation.
Feb 26 05:54:27 PM: (Subsequent compilations of this problem, using the same arguments, should take less time.)
Feb 26 05:54:27 PM: Invoking solver OSQP to obtain a solution.
-----------------------------------------------------------------
           OSQP v0.6.0  -  Operator Splitting QP Solver
              (c) Bartolomeo Stellato,  Goran Banjac
        University of Oxford  -  Stanford University 2019
-----------------------------------------------------------------
problem:  variables n = 50, constraints m = 70
          nnz(P) + nnz(A) = 700
settings: linear system solver = qdldl,
          eps_abs = 1.0e-05, eps_rel = 1.0e-05,
          eps_prim_inf = 1.0e-04, eps_dual_inf = 1.0e-04,
          rho = 1.00e-01 (adaptive),
          sigma = 1.00e-06, alpha = 1.60, max_iter = 10000
          check_termination: on (interval 25),
          scaling: on, scaled_termination: off
          warm start: on, polish: on, time_limit: off

iter   objective    pri res    dua res    rho        time
   1   0.0000e+00   1.95e+00   6.37e+02   1.00e-01   1.63e-04s
 200   1.9831e+01   2.92e-05   5.58e-06   1.29e+00   8.10e-04s
plsh   1.9831e+01   3.35e-16   8.89e-15   --------   9.09e-04s

status:               solved
solution polish:      successful
number of iterations: 200
optimal objective:    19.8313
run time:             9.09e-04s
optimal rho estimate: 4.33e+00

Feb 26 05:54:27 PM: Summary:
Feb 26 05:54:27 PM: Problem status: optimal
Feb 26 05:54:27 PM: Optimal value: 1.983e+01
Feb 26 05:54:27 PM: Compilation took 5.548e-03 seconds
Feb 26 05:54:27 PM: Solver took 1.704e-03 seconds

[SteveDiamond]

This looks good to me!

[akshayka]

Thanks for the review, @SteveDiamond ! I made some improvements/enhancements. Here's new sample output.

=============================================================================== 
                                     CVXPY                                     
                                    v1.1.10                                    
===============================================================================
(CVXPY) Feb 26 10:30:24 PM: Your problem has 20 variables, 2 constraints, and 0 parameters.
(CVXPY) Feb 26 10:30:24 PM: It is compliant with the following grammars: DCP, DQCP
(CVXPY) Feb 26 10:30:24 PM: (If you need to solve this problem multiple times, but with different data, consider using parameters.)
(CVXPY) Feb 26 10:30:24 PM: CVXPY will first compile your problem; then, it will invoke a numerical solver to obtain a solution.
-------------------------------------------------------------------------------
                                  Compilation                                   
-------------------------------------------------------------------------------
(CVXPY) Feb 26 10:30:24 PM: Compiling problem (target solver=OSQP).
(CVXPY) Feb 26 10:30:24 PM: Reduction chain: CvxAttr2Constr -> Qp2SymbolicQp -> QpMatrixStuffing -> OSQP
(CVXPY) Feb 26 10:30:24 PM: Applying reduction CvxAttr2Constr
(CVXPY) Feb 26 10:30:24 PM: Applying reduction Qp2SymbolicQp
(CVXPY) Feb 26 10:30:24 PM: Applying reduction QpMatrixStuffing                                                                          
(CVXPY) Feb 26 10:30:24 PM: Applying reduction OSQP                                                                                      
(CVXPY) Feb 26 10:30:24 PM: Finished problem compilation (took 5.444e-03 seconds).
(CVXPY) Feb 26 10:30:24 PM: (Subsequent compilations of this problem, using the same arguments, should take less time.)
-------------------------------------------------------------------------------
                                Numerical solver                                
-------------------------------------------------------------------------------
(CVXPY) Feb 26 10:30:24 PM: Invoking solver OSQP to obtain a solution.
-----------------------------------------------------------------
           OSQP v0.6.0  -  Operator Splitting QP Solver
              (c) Bartolomeo Stellato,  Goran Banjac
        University of Oxford  -  Stanford University 2019
-----------------------------------------------------------------
problem:  variables n = 50, constraints m = 70
          nnz(P) + nnz(A) = 700
settings: linear system solver = qdldl,
          eps_abs = 1.0e-05, eps_rel = 1.0e-05,
          eps_prim_inf = 1.0e-04, eps_dual_inf = 1.0e-04,
          rho = 1.00e-01 (adaptive),
          sigma = 1.00e-06, alpha = 1.60, max_iter = 10000
          check_termination: on (interval 25),
          scaling: on, scaled_termination: off
          warm start: on, polish: on, time_limit: off

iter   objective    pri res    dua res    rho        time
   1   0.0000e+00   1.95e+00   6.37e+02   1.00e-01   1.61e-04s
 200   1.9831e+01   2.92e-05   5.58e-06   1.29e+00   7.50e-04s
plsh   1.9831e+01   3.35e-16   8.89e-15   --------   8.37e-04s

status:               solved
solution polish:      successful
number of iterations: 200
optimal objective:    19.8313
run time:             8.37e-04s
optimal rho estimate: 4.33e+00

-------------------------------------------------------------------------------
                                    Summary                                     
-------------------------------------------------------------------------------
(CVXPY) Feb 26 10:30:24 PM: Problem status: optimal
(CVXPY) Feb 26 10:30:24 PM: Optimal value: 1.983e+01
(CVXPY) Feb 26 10:30:24 PM: Compilation took 5.444e-03 seconds
(CVXPY) Feb 26 10:30:24 PM: Solver (including time spent in interface) took 1.555e-03 seconds