Skip to content

To do list

Jump to bottom
Tim Miller edited this page Mar 20, 2024 · 24 revisions

Medium-term

  • Diagnostics as in Carvalho et al. 2021:
    1. Convergence:
      • no pars at bound
      • final gradient < 1e-4
      • pos def Hessian
      • no highly correlated pars (> 0.95)
      • no pars with v high SE
      • jittering doesn't change result (see Chris Legault's jitter ex)
    2. Fit
      • no trends in residuals (runs test, loess)
      • QQ plot
      • outliers
    3. Consistency
      • Mohn's rho SSB, F < 0.20
      • R0 profile (data conflicts?)
      • ASPM fits indices (indices are informative)
    4. Prediction skill
      • hindcast cross-validation
  • Vignettes:
    • simulation self/cross tests (wham-sim)
    • using WHAM as OM for MSE (ex10)
    • debugging wham models (do.fit=F implemented)
    • redo ex1 to explore fit model object, rm m2-m4
    • compare ASAP and WHAM models using compare_wham_models
    • describe each plot in plot_wham_output, as Liz's ASAPplots User Guide
  • Features / Functions:
    • self_test(mod, fix.re, n.sim) to simulate data and refit. Output: bias plots, process error estimates, convergence rate.
    • ensemble function to combine multiple WHAM model fits
    • extend compare_wham_models to plot multiple WHAM (and ASAP3) model fits together
    • MSE functionality (see new ex10)
    • OSA residuals for age compositions
    • modify output to be able to use functions from r4ss and ss3diags
    • age-0 recruitment (SSB_y affects R_y, frac_yr_recruit, ...)
  • Workflow
    • reduce runtime of tests (only optimize a couple models, for the rest just check NLL given par values)
    • continuous integration
    • describe how to contribute (run tests and submit pull request) (contributing guide now here)
  • Other / minor
    • fix error catching/reporting
    • prepare_wham_input() improvement: age comp model options
    • prepare_wham_input() improvement: specify options using words instead of numbers (recruit_model = "BevHolt" instead of 3)
    • re-run $fn() to make last.par = last.par.best in fit_tmb?
    • option to save summary(sdreport) vs. sdreport to make saved models MUCH smaller (e.g. 2MB vs. 35MB). New save.sdrep option (2f8875).
    • allow plotting of input data without model fit, i.e. modify plot_wham_output to work on output of fit_wham(input, do.fit=F)

Future

  • Paper: ICES time-varying selectivity
  • Paper: Single stock case studies (research tracks)
  • Paper: Multivariate Ecov (w/ EDAB)
  • Paper: Spatial EOF
  • Paper: How much info does Ecov need to have to be useful (simulation study)
  • Sparseness of models with NAA_sig = 'rec' (not all ages are RE)
  • Time-varying / Ecov effects on q
  • Misspecified catch (Perretti et al. 2020)
  • More Ecov process models (AR-k, Gaussian process, spline, ?)
  • Ecov finer time step than pop model
  • Shiny interface
  • Comply with FLOSS best practices
  • Better summary text file
  • Function to read in data from other formats (e.g. ICES, SS, etc.)

Before releasing v1.0

  1. Ability to estimate Ecov_obs_sigma (1 val shared, each obs, each obs as RE)

  2. Simulate Ecov

  3. Forecasting/projections

    • Recruitment options
    • Plotting
    • Check ref point calcs
    • Add vignette
    • Specify Ecov option

  4. Automated testing

  5. Time-varying selectivity (none, iid, ar1_a, ar1_y, 2dar1)

  6. 2D AR1 deviations on M

  7. Ecov mortality (GSI, O'Leary et al. 2019)

  8. Time-varying / Ecov effects on NAA

  9. toggle NAA options with prepare_wham_input (NAA_re = "none", "rec", or "all")

  10. Static webpage, pkgdown

  11. Paper 1: Simulation tests + model description

  12. Paper 2: 2D AR1 on NAA vs. M

  13. re-run all tests and confirm all pass

Clone this wiki locally