Seeding issue related to continuation resumes when lots of seeding removed

[saraloo]

EDIT: MOVED OVERVIEW TO #140

Project aims are to compare the effect of punctuated variant emergence vs our continuous immune ladder evolution structure on projections.

Overall plan

• Refit Round 17 phases with all the fixed boundary issues etc that were possibly there during when we ran it
• Run variant structure for calibration period and projection period
• Run variant structure until after omicron, then run immune ladder structure

Work is in: https://github.com/HopkinsIDD/COVID19_USA/tree/evol-projection

[saraloo]

Current issues with run

To run a continuation resume from 2020, adding seeding of new variants, there are errors I can't figure out.

Process now with continuation resume is:

• Seir from resume location is used
• values in seir at date = start date of new config are used as new init values. If there are new compartments you can include these as 0 with allow_missing_compartments: True
• parameters etc resumed
• seeding is resumed
  -- if you want to add new variants, create_added_seeding.R can add new seeding values to the resumed global file (eg I add alpha, delta and omicron seeding)
  -- you can add options like: stop perturbation of initial seeding (eg don't refit WILD seeding again)
• at every date in the simulation, flepimop checks if there is seeding and does the normal seeding process

Something here isn't working when I try to run a continuation resume Phase 2 from Phase 1. Config is: https://github.com/HopkinsIDD/COVID19_USA/blob/evol-projection/config_SMH_R17_noBoo_lowIE_phase2_blk1_test.yml

My seeding section looks like:

seeding:
  variant_filename: data/variant/variant_props_R17_phase2.csv
  seeding_compartments:
    ALPHA:
      source_compartment: ["S", "unvaccinated", "WILD", "age18to64"]
      destination_compartment: ["E", "unvaccinated", "ALPHA", "age18to64"]
    DELTA:
      source_compartment: ["S", "unvaccinated", "WILD", "age18to64"]
      destination_compartment: ["E", "unvaccinated", "DELTA", "age18to64"]
    OMICRON:
      source_compartment: ["S", "unvaccinated", "WILD", "age18to64"]
      destination_compartment: ["E", "unvaccinated", "OMICRON", "age18to64"]
  method: FolderDraw
  seeding_file_type: seed
  lambda_file: data/seeding_territories_R17_phase2.csv
  date_sd: 3
  amount_sd: 0
  added_seeding:
    added_lambda_file: data/seeding_territories_R17_phase2_added.csv
    fix_original_seeding: TRUE
    fix_added_seeding: FALSE

Seeding looks like it is working, but for some reason it looks like it is seeding the wrong variant? The curves here are from an seir file, and the black points are seeded amounts by variant.

Is there anything in flepimop that looks for the index of a compartment or strata rather than by name?

[saraloo]

Copying from slack from @jcblemai :
it turns out it was a nasty bug in gempyor:

• this bug has been present for more than 1y
• affects runs where there is seeding before the config t_start .
  • Seeding before t_start is discarded with a message (e.g CRITICAL:root:Seeding ignored 357 rows because they were before the start of the simulation. ). This is normal behavior
  • but because of this bug, it shifted the timing of the (not discarded) seeding (which becomes really off, depending on the # of discarded seeding rows)
  • probably most of our runs are fine because we mostly used create_seeding.R which does not seed before start_date
    • but some latter runs, that are resumed from older runs & use initial conditions could be affected. I could not remember any with a start_date change that could be affected.
    • Sara's run had hundreds of rows ignored which completely shifted the timing of the seeding runs.
      Waiting for a real fix in both main and breaking-improvment, a quickfix for main is available here (PR #121).

[jcblemai]

reopen because not fix in breaking-improvment

[jcblemai]

in fact another issue in #122 , so this one can be closed