Skip to content

Releases

v0.2.0

Choose a tag to compare

@JulienPascal JulienPascal released this 07 Oct 09:21

MethodOfSimulatedMoments.jl 0.2.0
(formerly MSM.jl).

Starting with the branch julia_1.13, the bug fixes and the update to Julia
1.13 were made with the assistance of Claude Opus 5.5 (Anthropic), used through
Claude Code.

Added

  • check_problem(sMMProblem) checks that a problem is set up correctly, and
    throws an informative error otherwise. msm_optimize! and msm_multistart!
    call it first; skip it with check = false. The objective function returns
    the penalty value on any error, so such mistakes previously gave a flat
    objective function, and the optimization ran to completion without an error.
    It checks that the priors, the empirical moments, the simulation function and
    the objective function are set, and that W is k×k for k empirical moments
    (the default W is 1×1). It then simulates the moments once, at the initial
    values of the priors, outside of the objective function. It fails if the
    simulation throws (the original error is logged with its stack trace), does
    not return every empirical moment, or returns non-finite moments. It warns
    about simulated moments that are not empirical moments (they are ignored),
    and when the distance at the initial values exceeds a finite penaltyValue
    (failed simulations would then look better than successful ones).

  • MSMOptions(lambda = ...): number of points evaluated per generation by the
    natural evolution strategies (:dxnes, the default, :xnes and
    :separable_nes), and nes_lambda to compute it. The points of a generation
    are evaluated in parallel, and the next generation starts when all of them
    are done. BlackBoxOptim's default depends only on the number of parameters
    (8 for 5 parameters, 10 for 10 parameters with :dxnes), so with more
    workers than that, most of them were idle. With lambda = 0 (default), it is
    now rounded up to a multiple of the number of workers (:dxnes requires an
    even value: with an odd number of workers, one worker is idle). With
    verbose = true, msm_optimize! logs the value used and the number of
    generations it implies. With several workers, this changes the global
    optimization, and so its results.
    maxFuncEvals still counts
    evaluations: a larger lambda means fewer generations for the same budget,
    so scale maxFuncEvals with lambda. With one worker, nothing changes.

  • is_nes_optimizer(s::Symbol).

  • A notebook estimating a small DSGE model (a real business cycle model written
    with MacroModelling.jl): notebooks/models/RBC.ipynb, with its own
    environment. It recovers known parameters from simulated data, and checks the
    coverage of the confidence intervals and the size of the J-test in a Monte
    Carlo experiment.

  • A notebook estimating the same model with Dynare.jl instead of
    MacroModelling.jl to solve and simulate it:
    notebooks/models/dynare/RBCDynare.ipynb, with its own environment and the
    model file RBC.mod. Each worker loads the model in its own temporary
    folder, because Dynare.jl writes files next to the model file. An optional
    last section runs the method_of_moments command of Dynare 6 in Octave (as
    an external program, with the model file RBC_mom.mod) and compares its
    estimates with those of MethodOfSimulatedMoments.jl. It is skipped when
    Octave is not installed.

Changed

  • The package is renamed MethodOfSimulatedMoments.jl (it was MSM.jl),
    so that it can be registered in Julia's General registry, whose naming rules
    exclude short, all-caps names. Replace using MSM with
    using MethodOfSimulatedMoments. The functions and types keep their names
    (MSMProblem, MSMOptions, msm_optimize!...), and the UUID is unchanged.
    The repository is now github.com/JulienPascal/MethodOfSimulatedMoments.jl
    (the old address redirects to it), and the documentation is at
    julienpascal.github.io/MethodOfSimulatedMoments.jl.
  • populationSize is ignored by the NES optimizers (it only applies to
    differential evolution). MSMOptions now warns when it is set together with
    one of them. With verbose = true and several workers, msm_optimize! also
    notes that differential evolution evaluates about one new point at a time,
    so it gets essentially no speed-up from several workers.
  • The default penaltyValue is now Inf (it was 999999). With a finite
    penalty, a real distance can exceed it when the moments are badly scaled, and
    failed points then look better than successful ones. Optim and BlackBoxOptim
    handle Inf values. The default thresholdStartingValue is still
    penaltyValue/10, so it is now Inf too: any successful point is a valid
    starting value for msm_multistart!. Pass penaltyValue = 999999.0 to
    restore the previous behavior (e.g. to feed failed points to a surrogate).
  • Local minimizations (msm_refine_globalmin!, msm_localmin,
    msm_multistart!) now stop after maxFuncEvals evaluations of the objective
    function
    , finite-difference gradients included (at the end of the
    iteration during which the budget is reached). maxFuncEvals was passed to
    Optim as the maximum number of iterations, and with finite-difference
    gradients each iteration costs several evaluations: maxFuncEvals = 1000
    could mean tens of thousands of simulations. Optim's f_calls_limit could
    not be used, because it does not count the evaluations of finite
    differences. A local minimization stopped by the budget does not count as
    converged.
  • msm_multistart!: if none of the local minimizations converged, the best
    finite local minimum is now used, with a message saying so. Previously, no
    result was stored.
  • MethodOfSimulatedMoments.jl now requires Julia 1.10 or later (1.10 is the
    current long-term support release). It is tested on Julia 1.13.
  • Project.toml now declares compatibility bounds for every dependency, so
    that a future breaking release of a dependency cannot be installed with
    MethodOfSimulatedMoments.jl. Only the current major versions are allowed, in
    particular DataFrames 1 and BlackBoxOptim 0.6 (with the exceptions below).
  • Both Optim 1 (1.13 or later) and Optim 2 are allowed, so that
    MethodOfSimulatedMoments.jl can be installed next to packages that still
    require Optim 1 (e.g. MacroModelling.jl). The test suite passes with Optim
    1.13.3 and Optim 2.3.2. The results of local minimizations can differ
    slightly between the two. Known issue: with Optim 2 only,
    localOptimizer = :AcceleratedGradientDescent can diverge (it does on the
    Rosenbrock function; it converges with Optim 1). Prefer :LBFGS (the
    default).
  • Both OrderedCollections 1 and 2 are allowed, for the same reason:
    MacroModelling.jl's Bayesian estimation requires Turing.jl 0.30 to 0.45,
    whose dependencies require OrderedCollections 1. The test suite passes with
    OrderedCollections 1.8.2 (with Optim 1.13.3) and 2.0.1.
  • Both CSV 0.10 and CSV 1 are allowed, so that MethodOfSimulatedMoments.jl
    can be installed next to Dynare.jl (version 0.10.4 requires CSV 0.10).
    MethodOfSimulatedMoments.jl only uses CSV.File, which is the same in both.
    The test suite passes with CSV 0.10.17 and 1.1.0.
  • msm_minimizer, msm_minimum, msm_local_minimizer and msm_local_minimum
    now throw an informative error when called before the corresponding
    optimization (or after msm_multistart! if no local minimization returned a
    finite value).
    They previously returned the result of a dummy optimization of the
    Rosenbrock function. The fields bbSetup, bbResults and optimResults of
    MSMProblem are now nothing until they are set.
  • Loading MethodOfSimulatedMoments.jl no longer runs a BlackBoxOptim and an
    Optim optimization at precompile time (they were only used to create those
    dummy default values).
  • convert_to_optim_algo and convert_to_fminbox no longer use
    eval(Meta.parse(...)), and throw an error for names that are not supported
    local optimizers.

Fixed

These fixes change numerical results. If you used J_test,
calculate_pvalue, calculate_CI or summary_table, re-run your inference.

  • J_test: the critical value now comes from the chi-squared distribution
    Chisq(df). It previously came from the chi distribution Chi(df), whose
    quantiles are the square roots of the correct ones, so the test rejected far
    too often.

  • J_test: the statistic is now tData/(1 + tau)*g'Wg, with
    tau = tData/tSimData, following Lee and Ingram (1991, pp. 202 and 204). It was
    tData*(1 + tau)*g'Wg, as printed in Ruge-Murcia (2012, eq. 13, probably a
    typo), which overstates J by a factor (1 + tau)^2: 4 when the simulated and
    observed series have the same length. Combined with the previous point, a
    correctly specified model was rejected 18% to 48% of the time at the 5% level
    in simulations, instead of 5%. The test still requires W to converge to
    inv(Sigma0).

  • calculate_pvalue and the Pr(>|t|) column of summary_table: the
    two-sided p-value is now 2*ccdf(Normal(), abs(t)). It was wrong (between 1
    and 2) for negative t-statistics.

  • calculate_CI and the confidence intervals of summary_table: the interval
    now uses the 1 - alpha/2 quantile of the normal distribution. With
    alpha = 0.05, it previously reported a 90% confidence interval instead of a
    95% one.

  • msm_optimize! no longer prints a leftover debug message ("hello") on
    every worker.

  • msm_optimize!(...; verbose = false) now hides BlackBoxOptim's progress
    trace and the package's messages. The verbose keyword was previously
    ignored. set_global_optimizer! and set_bbSetup! accept the same keyword.

  • msm_multistart! with 2 or more workers and without user-provided x0:
    candidate starting values were matched with the distances of other
    candidates, because r...

Read more