Update criterion functions for new estimagic interface

[amageh]

Current behavior

The current criterion functions will not work for upcoming changes to estimagic's optimization capabilities. In particular, the way outputs are returned needs to be adjusted,

get_log_like_func for maximum likelihood estimation:
The following arguments control what objects will be returned:

• return_scalar: If True returns mean log-likelihood, if False returns NumPy.array of individual log-likelihood contributions.
• return_comparison_plot_data: If True returns pandas.DataFrame in tidy data format with the columns identifier, period, choice, value, and kind. value column contains likelihood contributions (see Restructure likelihood.py for #310 and estimagic's comparison plot. #313)

If both are True, the criterion will return a tuple.

get_moment_errors_func for method of simulated moments estimation:
The following arguments control what objects will be returned:

• return_scalar: If True will return the square product of weighted moment errors (float), if False will return moment errors multiplied by the root of weighting matrix.
• return_simulated_moments: If True will return simulated moments in the same data structure as input empirical_moments.
• return_comparison_plot_data: If True will return pandas.DataFrame in tidy data format with empirical and simulated moments (see Add comparison_plot_data to msm and return moments. #363)

If return_simulated_moments or return_comparison_plot_data are True, function will return tuple. Both can't be True at the same time.

Desired behavior

To fit the new estimagic optimization interface, the criterion should return either a scalar/NumPy.ndarray or a dictionary containing the additional information (contributions).

Solution / Implementation

Both functions now return either a scalar or dictionary (no tuples). I implemented this by eliminating the function arguments return_simulated_moments and return_comparison_plot_data. There are definitely other ways to do it but I feel like this could make the functions more concise. Backwards-compatibility will be compromised either way with these changes as far as I can judge.

I detailed in both function docstrings how the function outputs will look like. As for the comparison plot data, some additional details:

Likelihood function:

• I tested the new interface on the kw_94_one model (1000 agents and 40 periods). The pickled output is around 1.7 MB.

	identifier	period	choice	value	kind
0	0	0	a	-1.03995	choice
1	0	1	b	-0.844175	choice
2	0	2	b	-0.173064	choice
3	0	3	edu	-2.0254	choice
4	0	4	edu	-0.527486	choice
...	...	...	...	...	...
79995	999	35	b	-10.0324	wage
79996	999	36	b	-10.013	wage
79997	999	37	b	-9.97582	wage
79998	999	38	b	-10.4655	wage
79999	999	39	b	-14.1459	wage

The columns contain the following data types:

identifier      uint16
period           uint8
choice        category
value          float64
kind          category
dtype: object

Simulated Method of Moments:

Example of comparison plot data created for kw_94_one on a sample of 320 moments. The pickled output here is around 30 kb but will obviously vary quite a lot with the choice of moments. I imposed only few data types (the columns moment_set and kind are set to categorical) since most depend on user-defined inputs.

	moment_column	moment_index	value	moment_set	kind
0	count	0	535.000000	0	empirical
1	count	1	634.000000	0	empirical
2	count	2	722.000000	0	empirical
3	count	3	740.000000	0	empirical
4	count	4	767.000000	0	empirical
...	...	...	...	...	...
635	max	35	109416.436026	0	simulated
636	max	36	99576.614121	0	simulated
637	max	37	115929.849186	0	simulated
638	max	38	100242.059114	0	simulated
639	max	39	98889.945043	0	simulated

Todo

• Add data types for comparison plot data for both criterion functions.
• Update returned objects in likelihood module.
• Update returned objects in msm module.
• Update documentation (msm guides)
• Adjust tests.
• Add more tests (?)
• Document PR in CHANGES.rst.

[codecov]

Codecov Report

Merging #389 (43f8797) into main (a4a5ff4) will decrease coverage by 2.41%.
The diff coverage is 54.54%.

❗ Current head 43f8797 differs from pull request most recent head 6cdd6b0. Consider uploading reports for the commit 6cdd6b0 to get more accurate results

@@            Coverage Diff             @@
##             main     #389      +/-   ##
==========================================
- Coverage   76.59%   74.17%   -2.42%     
==========================================
  Files          50       50              
  Lines        3640     3636       -4     
==========================================
- Hits         2788     2697      -91     
- Misses        852      939      +87     

Flag	Coverage Δ
end_to_end	74.17% <54.54%> (+0.85%)	⬆️
integration	?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
respy/tests/test_likelihood.py	48.83% <14.28%> (-44.50%)	⬇️
respy/likelihood.py	76.43% <37.50%> (-8.54%)	⬇️
respy/tests/test_method_of_simulated_moments.py	86.66% <59.09%> (-13.34%)	⬇️
respy/method_of_simulated_moments.py	83.78% <100.00%> (+3.78%)	⬆️
respy/tests/test_flexible_choices.py	54.54% <0.00%> (-33.34%)	⬇️
respy/tests/test_solve.py	50.22% <0.00%> (-16.45%)	⬇️
respy/tests/utils.py	92.59% <0.00%> (-3.71%)	⬇️
respy/tests/test_model_processing.py	45.28% <0.00%> (-2.84%)	⬇️
... and 2 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a4a5ff4...6cdd6b0. Read the comment docs.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB