OG-USA:master broken

[jdebacker]

It's been since early May when I last ran OG-USA:master successfully. Since that time I have updated from Python 2.7 to 3.6. But when running OG-USA I am activating the ospcdyn environment.

This morning, I attempted to run run_ogusa_serial.py again so that I can help someone new to the project. My runs failed.

In run_ogusa_serial.py, my arguments are:

    output_base = BASELINE_DIR
    input_dir = BASELINE_DIR
    kwargs={'output_base':output_base, 'baseline_dir':BASELINE_DIR,
            'test':False, 'time_path':True, 'baseline':True,
            'analytical_mtrs':False, 'age_specific':True,
            'user_params':user_params,'guid':'',
            'run_micro':False, 'small_open': False, 'budget_balance':False,
            'baseline_spending':False}
    runner(**kwargs)

First, I got the error:

Traceback (most recent call last):
  File "run_ogusa_serial.py", line 126, in <module>
    run_micro_macro(user_params={})
  File "run_ogusa_serial.py", line 85, in run_micro_macro
    runner(**kwargs)
  File "/Users/jason.debacker/repos/dynamic/Python/ogusa/scripts/execute.py", line 50, in runner
    run_params = ogusa.parameters.get_parameters(test=test, baseline=baseline, guid=guid)
  File "/Users/jason.debacker/repos/dynamic/Python/ogusa/parameters.py", line 452, in get_parameters
    flag_graphs)
  File "/Users/jason.debacker/repos/dynamic/Python/ogusa/demographics.py", line 699, in get_pop_objs
    graph=False)
  File "/Users/jason.debacker/repos/dynamic/Python/ogusa/demographics.py", line 287, in get_mort
    mort_rates_sub = np.zeros(num_sub_bins * max_yr, dtype=np.float64)
TypeError: 'float' object cannot be interpreted as an index

I fixed this by changing line 284 of demographics.py from

num_sub_bins = float(100)

to

num_sub_bins = int(100)

This allowed me to get to SS solution algorithm. This had been quite robust, but I find that the root finders (mostly scipy. optimize.fsolve) are often hitting the constraints for the variables and eventually nan's are returned and the SS solution fails.

@rickecon and @PeterDSteinberg Can you confirm this behavior?

Before fixing, I'd like to have some idea what caused the code to break.

[jdebacker]

@PeterDSteinberg Here's the output printed from the running OG-USA with the above specifications and the fix todemographics.py:

A02655JDEBACKER:Python jason.debacker$ source activate ospcdyn
(ospcdyn) A02655JDEBACKER:Python jason.debacker$ python run_ogusa_serial.py
making dir:  ./OUTPUT_BASELINE/Saved_moments
making dir:  ./OUTPUT_BASELINE/SS
making dir:  ./OUTPUT_BASELINE/TPI
In runner, baseline is  True
using baseline tax parameters
pickle path exists
Traceback (most recent call last):
  File "run_ogusa_serial.py", line 126, in <module>
    run_micro_macro(user_params={})
  File "run_ogusa_serial.py", line 85, in run_micro_macro
    runner(**kwargs)
  File "/Users/jason.debacker/repos/dynamic/Python/ogusa/scripts/execute.py", line 50, in runner
    run_params = ogusa.parameters.get_parameters(test=test, baseline=baseline, guid=guid)
  File "/Users/jason.debacker/repos/dynamic/Python/ogusa/parameters.py", line 452, in get_parameters
    flag_graphs)
  File "/Users/jason.debacker/repos/dynamic/Python/ogusa/demographics.py", line 699, in get_pop_objs
    graph=False)
  File "/Users/jason.debacker/repos/dynamic/Python/ogusa/demographics.py", line 287, in get_mort
    mort_rates_sub = np.zeros(num_sub_bins * max_yr, dtype=np.float64)
TypeError: 'float' object cannot be interpreted as an index
(ospcdyn) A02655JDEBACKER:Python jason.debacker$ python run_ogusa_serial.py
making dir:  ./OUTPUT_BASELINE/Saved_moments
making dir:  ./OUTPUT_BASELINE/SS
making dir:  ./OUTPUT_BASELINE/TPI
In runner, baseline is  True
using baseline tax parameters
pickle path exists
/Users/jason.debacker/anaconda/envs/ospcdyn/lib/python2.7/site-packages/scipy/optimize/minpack.py:161: RuntimeWarning: The iteration is not making good progress, as measured by the 
  improvement from the last ten iterations.
  warnings.warn(msg, RuntimeWarning)
/Users/jason.debacker/anaconda/envs/ospcdyn/lib/python2.7/site-packages/scipy/optimize/minpack.py:161: RuntimeWarning: The iteration is not making good progress, as measured by the 
  improvement from the last five Jacobian evaluations.
  warnings.warn(msg, RuntimeWarning)
updating frisch and associated
updating ALPHA_T with T_shifts in first 50 periods.
updating ALPHA_G with G_shifts in first 6 periods.
r, w at outset:  0.04 1.2
/Users/jason.debacker/repos/dynamic/Python/ogusa/household.py:319: RuntimeWarning: invalid value encountered in power
  savings_ut = rho * np.exp(-sigma * g_y) * chi_b * b_splus1 ** (-sigma)
/Users/jason.debacker/repos/dynamic/Python/ogusa/household.py:114: RuntimeWarning: invalid value encountered in power
  output = c**(-sigma)
/Users/jason.debacker/repos/dynamic/Python/ogusa/SS.py:228: RuntimeWarning: invalid value encountered in less
  mask1 = n_guess < 0
/Users/jason.debacker/repos/dynamic/Python/ogusa/SS.py:229: RuntimeWarning: invalid value encountered in greater
  mask2 = n_guess > ltilde
/Users/jason.debacker/repos/dynamic/Python/ogusa/SS.py:230: RuntimeWarning: invalid value encountered in less_equal
  mask3 = b_guess <= 0
/Users/jason.debacker/repos/dynamic/Python/ogusa/SS.py:244: RuntimeWarning: invalid value encountered in less
  mask6 = cons < 0
/Users/jason.debacker/repos/dynamic/Python/ogusa/household.py:142: RuntimeWarning: invalid value encountered in power
  (1.0 / upsilon) - 1.0)) * (n / ltilde) ** (upsilon - 1.0)
/Users/jason.debacker/repos/dynamic/Python/ogusa/firm.py:98: RuntimeWarning: invalid value encountered in double_scalars
  Y = Z*(K**gamma)*(L**(1-gamma))
inner factor prices:  -0.327980380329 3.22525906219
Y:  nan
errors:  -0.367980380329 2.02525906219 nan 0.0102096961687
r, w at outset:  0.04 1.2
inner factor prices:  -0.327161929824 3.58362118021
Y:  nan
errors:  -0.367161929824 2.38362118021 nan 0.0100936128435
r, w at outset:  0.04 1.2
inner factor prices:  -0.326429291792 3.98180131135
Y:  nan
errors:  -0.366429291792 2.78180131135 nan 0.00998991835035
r, w at outset:  0.040000000596 1.2
inner factor prices:  -0.325773106547 4.42422367928
Y:  nan
errors:  -0.365773107143 3.22422367928 nan 0.00989721843208
r, w at outset:  0.04 1.20000001788
inner factor prices:  -0.325185105119 4.91580408808
Y:  nan
errors:  -0.365185105119 3.7158040702 nan 0.00981428976995
r, w at outset:  0.04 1.2
inner factor prices:  -0.324657968801 5.46200462371
Y:  nan
errors:  -0.364657968801 4.26200462371 nan 0.0097400551385
r, w at outset:  0.04 1.2
inner factor prices:  -0.324185209636 6.06889393591
Y:  nan
errors:  -0.364185209636 4.86889393591 nan 0.00967356847861
r, w at outset:  nan nan
inner factor prices:  nan nan
/Users/jason.debacker/repos/dynamic/Python/ogusa/tax.py:58: RuntimeWarning: invalid value encountered in greater
  PIA[PIA > maxpayment] = maxpayment
Y:  nan
errors:  nan nan nan nan
r, w at outset:  nan nan
inner factor prices:  nan nan
Y:  nan
errors:  nan nan nan nan
r, w at outset:  0.040000000596 1.2
inner factor prices:  -0.323038708972 8.32495738807
Y:  nan
errors:  -0.363038709568 7.12495738807 nan 0.00951267853467
r, w at outset:  0.04 1.20000001788
inner factor prices:  -0.3227318733 9.24995265341
Y:  nan
errors:  -0.3627318733 8.04995263553 nan 0.00946970279749
r, w at outset:  0.04 1.2
inner factor prices:  -0.322456289408 10.2777253236
Y:  nan
errors:  -0.362456289408 9.07772532361 nan 0.00943113285812
r, w at outset:  0.04 1.2
inner factor prices:  -0.322208722906 11.4196946338
Y:  nan
errors:  -0.362208722906 10.2196946338 nan 0.00939650937526
r, w at outset:  nan nan
inner factor prices:  nan nan
Y:  nan
errors:  nan nan nan nan
r, w at outset:  nan nan
inner factor prices:  nan nan
Y:  nan
errors:  nan nan nan nan
r, w at outset:  nan nan
inner factor prices:  nan nan
Y:  nan
errors:  nan nan nan nan
r, w at outset:  nan nan
inner factor prices:  nan nan
Y:  nan
errors:  nan nan nan nan
r, w at outset:  nan nan
inner factor prices:  nan nan
Y:  nan
errors:  nan nan nan nan
r, w at outset:  nan nan
inner factor prices:  nan nan
Y:  nan
errors:  nan nan nan nan
r, w at outset:  nan nan
inner factor prices:  nan nan
Y:  nan
errors:  nan nan nan nan
r, w at outset:  nan nan
inner factor prices:  nan nan
Y:  nan
errors:  nan nan nan nan
Traceback (most recent call last):
  File "run_ogusa_serial.py", line 126, in <module>
    run_micro_macro(user_params={})
  File "run_ogusa_serial.py", line 85, in run_micro_macro
    runner(**kwargs)
  File "/Users/jason.debacker/repos/dynamic/Python/ogusa/scripts/execute.py", line 123, in runner
    baseline_dir=baseline_dir)
  File "/Users/jason.debacker/repos/dynamic/Python/ogusa/SS.py", line 997, in run_SS
    raise RuntimeError("Steady state equilibrium not found")
RuntimeError: Steady state equilibrium not found

[rickecon]

Yep. That's the same issue I got. @PeterDSteinberg @jdebacker

[jdebacker]

Fixed via PR #305

[PeterDSteinberg]

@jdebacker Can this be closed? I can dive in tomorrow if it is still a problem.

[jdebacker]

@PeterDSteinberg Yes - I should have closed after my comment 3 days ago. Will do now.