Check for when to simplify should not skip the first generation?

[apragsdale]

Using the mutation tracker found here (https://github.com/molpopgen/fwdpy11_frequency_tracker, in the "efficiency" branch, and fwdpy11 version 0.11.0), and using the attached script to track both selected and neutral mutation frequencies, there is some strange behavior at the boundary of switching from burn-in generations to tracking generations. During burn-in, the simplification interval is 100, and then we switch it to 1 when tracking neutral mutations. However, we have to switch a few generations before burn-in is finished, because otherwise the longest neutral mutation trajectories are shorter than the longest selected mutation trajectories. We need to switch at least two generations ahead of time, it looks like.

selection_example.txt

Below, note the trajectory lengths for selected vs neutral mutations depending on that burnin buffer

In [1]: %run selection_example.py -N 1000 -s 1 --track_neutral --burnin_buffer 0
 [2021-01-12 09:40:17] Starting simulation with 22000 total generations
 [2021-01-12 09:40:27] Finished simulation, at generation 22000
Long trajectories (key, length):
(18940, 51106.2087258324, 0.0) 1999
(19212, 77528.02106551826, -0.0002690292324071674) 2001
(19725, 12291.445885784924, 0.0) 1999
(19757, 44030.960416421294, -2.7552225122793526e-08) 2001
(19989, 91481.12745024264, -2.268166778672198e-06) 2001

In [2]: %run selection_example.py -N 1000 -s 1 --track_neutral --burnin_buffer 1
 [2021-01-12 09:40:32] Starting simulation with 22000 total generations
 [2021-01-12 09:40:42] Finished simulation, at generation 22000
Long trajectories (key, length):
(18940, 51106.2087258324, 0.0) 2000
(19212, 77528.02106551826, -0.0002690292324071674) 2001
(19725, 12291.445885784924, 0.0) 2000
(19757, 44030.960416421294, -2.7552225122793526e-08) 2001
(19989, 91481.12745024264, -2.268166778672198e-06) 2001

In [3]: %run selection_example.py -N 1000 -s 1 --track_neutral --burnin_buffer 2
 [2021-01-12 09:40:44] Starting simulation with 22000 total generations
 [2021-01-12 09:40:54] Finished simulation, at generation 22000
Long trajectories (key, length):
(18940, 51106.2087258324, 0.0) 2001
(19212, 77528.02106551826, -0.0002690292324071674) 2001
(19725, 12291.445885784924, 0.0) 2001
(19757, 44030.960416421294, -2.7552225122793526e-08) 2001
(19989, 91481.12745024264, -2.268166778672198e-06) 2001

In [4]: %run selection_example.py -N 1000 -s 1 --track_neutral --burnin_buffer 3
 [2021-01-12 09:40:56] Starting simulation with 22000 total generations
 [2021-01-12 09:41:07] Finished simulation, at generation 22000
Long trajectories (key, length):
(18940, 51106.2087258324, 0.0) 2001
(19212, 77528.02106551826, -0.0002690292324071674) 2001
(19725, 12291.445885784924, 0.0) 2001
(19757, 44030.960416421294, -2.7552225122793526e-08) 2001
(19989, 91481.12745024264, -2.268166778672198e-06) 2001

In [5]: fwdpy11.__version__
Out[5]: '0.11.0'

[molpopgen]

Confirmed against current main/dev branches. Thanks @apragsdale -- will investigate further!

[molpopgen]

What @apragsdale's example is seeing is due to the following:

1. In generation when no simplification occurs AND mutation tracking is asked for, mutation counts are updated from the genomes. A definite problem with this is that only the counts for selected mutations get updated, as neutral mutations are never added to genomes. This problem means that a user shouldn't use neutral counts during such phases, which the example is not doing.
2. In a generation when simplification did occur, counts are updated via the tables. Both neutral and selected variants get updated.
3. The real culprit is that the C++ back end does not allow simplification in the very first generation. So, in the example, the second time in, when the simplification interval is set to 1, it does not happen in the first generation. The relevant line is this. The gen > 0 check is in place with the usual use case in mind of "evolve until done", and it seems inefficient to simplify the very first generation. In hindsight, this is a small inefficiency that we should just get over :).

The best thing to do is to fix the issue described in 3 above. The effect will be to shift simplification to 1 generation earlier with respect to pop.generation. This change has the potential to change output for some seeds because of the coupling of simplification and mutation recycling in infinitely-many sites models.

[molpopgen]

With the changes in #627 and a change to the frequency tracker, I get the following:

In [1]: %run selection_example.py -N 1000 -s 1 --track_neutral --burnin_buffer 0
 [2021-01-12 18:31:21] Starting simulation with 22000 total generations
 [2021-01-12 18:31:30] Finished simulation, at generation 22000
Long trajectories (key, length):
(19212, 77528.02106551826, -0.0002690292324071674) 2000
(18940, 51106.2087258324, 0.0) 2000
(19757, 44030.960416421294, -2.7552225122793526e-08) 2000
(19725, 12291.445885784924, 0.0) 2000
(19989, 91481.12745024264, -2.268166778672198e-06) 2000