Orphaned indicator periods

[zzgvh]

Some editing actions in the Results framework may lead to indicator periods in child projects becoming orphaned. This can happen in at lest two ways:

1. If an indicator period of a child project, that has inherited results from a parent, is deleted and then recreated, it will not become a part of the parent-child relation even if the dates are set to be the same as originally. Instead an error message is shown for the date fields and they then become empty and in-editable when reloading the project.
   

2. If an indicator period of a parent project is edited to change the period start and/or end dates, any imported period in a child will be similarly orphaned.

In both cases the actual value or updates made in the child will not be imported to the parent. Also the Import results button on the child becomes disabled and cannot be enabled again even after removing the Result and the parent relation in the child and then putting the relation back.

I have not yet fully investigated the cause of these problems, but they are probably connected to the fact that parent-child relations is maintained in part by filtering on the period date fields and when they don't align the link may be lost.

[punchagan]

I was trying to get a list of all the child projects of 3980 that have indicator periods with no period_start.

ids = [p.id for p in Project.objects.get(id=3980).children_all()]
for id_ in ids:
    print id_
    for indicator_period in IndicatorPeriod.objects.filter(period_start=None, indicator__result__project_id=id_):
        print indicator_period.indicator.result.title
        print indicator_period.indicator.title
        print indicator_period.parent_period().period_start

I found a bunch of indicator periods on the project 4389.

4389
Funding proposal are submitted at international and country level, in order to raise funding
Number of project proposals where partners provide input for at international level and country level
None

Notice that the parent_period doesn't yet have a period_start. Given the bug we found in IndicatorPeriod.save when period_start is changed after the import, I'm guessing that there were a lot more periods which initially didn't have a period_start, but an import was performed, and the users added data slowly both to the child project and the parent project's results framework. But, once the period_start was changed for a period on the parent project, the child periods get into the weird state where their period_start is empty, but cannot be changed - they look like child indicator periods, because the associated result has a parent_result, but they are not linked to a parent indicator, because their period_start is empty ( the old value of the parent's period_start ) but the parent has a new value for period_start.

[punchagan]

Looks like this bug was accidentally introduced in this commit that was merged with #2298 (quite recently).

There aren't too many indicators (68) where this problem is present, since the bug is pretty new. Most of them seem to be in WASH alliance projects.

for indicator_period in IndicatorPeriod.objects.filter(Q(period_start=None) | Q(period_end=None)).exclude(indicator__result__parent_result=None):
    project = indicator_period.indicator.result.project
    print "|{}|{}|{}|".format(indicator_period, project.id, project.title)

Indicator	Project id	Project
No start date - 2016-12-31	4313	WASH alliance Ghana - URBANET 2016
No start date - 2016-12-31	4348	WASH Alliance Mali - Practica 2016
No start date - 2016-12-31	4347	WASH Alliance Mali - Wetlands 2016
No start date - 2016-12-31	4362	WASH Alliance Mali - NEF 2016
No start date - No end date (target: 14000)	4389	Increased access to WASH
No start date - No end date (target: 19 million (Euro))	4389	Increased access to WASH
No start date - No end date (target: 19000)	4389	Increased access to WASH
No start date - No end date (target: 24)	4389	Increased access to WASH
No start date - No end date	4389	Increased access to WASH
No start date - No end date	4389	Increased access to WASH
No start date - No end date	4389	Increased access to WASH
No start date - No end date	4389	Increased access to WASH
No start date - No end date	4389	Increased access to WASH
No start date - No end date	4389	Increased access to WASH
No start date - No end date	4389	Increased access to WASH
No start date - No end date	4389	Increased access to WASH
No start date - No end date	4389	Increased access to WASH
No start date - No end date	4389	Increased access to WASH
No start date - 2016-12-31 (actual: 0)	4206	WASH Alliance Uganda - Amref Uganda 2016
No start date - 2016-12-31	4207	WASH Alliance Uganda - Wetlands 2016
No start date - 2016-12-31	4248	WASH Alliance Kenya - KEWASNET 2016
No start date - 2016-12-31	4246	WASH Alliance Kenya - Practical Action 2016
No start date - 2016-12-31	4382	Freshwater Ecosystem Functionality in Kajiado
No start date - 2017-12-31	5297	CommonSense (Test)
No start date - 2017-12-31	5296	G4INDO (test)
No start date - 2020-12-31	3973	Enhancing Opportunities for Women's Enterprises Vietnam
No start date - 2020-12-31 (target: 12)	3974	Enhancing Opportunities for Women's Enterprises Kenya
No start date - 2020-12-31 (target: 18)	3974	Enhancing Opportunities for Women's Enterprises Kenya
No start date - 2020-12-31 (target: 90)	3973	Enhancing Opportunities for Women's Enterprises Vietnam
No start date - 2016-12-31	4241	WASH Alliance Uganda - NAWAD 2016
No start date - 2016-12-31	487	WASH Alliance Uganda - NETWAS 2016
No start date - 2016-12-31	4271	WASH Alliance Kenya - Sasol 2016
No start date - 2016-12-31	5352	WASH Alliance Kenya - NOSIM 2016
No start date - 2016-12-31 (actual: 5 / target: 13)	4277	WASH alliance Ghana - AFORD 2016
No start date - 2016-12-31 (actual: 0 / target: 1)	4278	Inegrated Community Wash and Advocacy Project
No start date - 2016-12-31	4131	WASH Alliance Ethiopia - Amref ethiopië 2016
No start date - 2016-12-31	4078	WASH Alliance Ethiopia - Diredawa city administration 2016
No start date - 2016-12-31	4074	WASH Alliance Ethiopia - Central rift Valley 2016
No start date - 2016-12-31	3982	WASH Alliance Ethiopia - 2016
No start date - 2016-12-31	3981	WASH Alliance Kenya - 2016
No start date - 2016-12-31	3986	WASH Alliance Mali - 2016
No start date - 2016-12-31 (actual: 3)	3979	WASH Alliance Bangladesh 2016
No start date - 2016-12-31 (actual: 2 / target: 2)	4283	WASH alliance Ghana - CLIP 2016
No start date - 2016-12-10 (actual: 4)	5353	Lazos de Agua - Jinotega, Nicuragua
No start date - 2017-01-10	5353	Lazos de Agua - Jinotega, Nicuragua
No start date - 2016-12-31 (actual: 0)	3983	WASH Alliance Uganda - 2016
No start date - 2016-12-31	4321	WASH Alliance Bangladesh (Practical Action, Bangladesh)
No start date - 2016-12-31 (actual: 3)	4322	WASH Alliance Bangladesh -Hope for the poorest 2016
No start date - 2016-12-31 (actual: 7)	3988	WASH Alliance Ghana - 2016
No start date - 2016-12-31	4276	WASH alliance Ghana - RUAF 2016
No start date - No end date	5569	Lazos de Agua - Jinotega, Nicuragua
No start date - No end date (actual: 10 / target: 10)	5569	Lazos de Agua - Jinotega, Nicuragua
No start date - No end date	5569	Lazos de Agua - Jinotega, Nicuragua
No start date - No end date (actual: 500 / target: 500)	5569	Lazos de Agua - Jinotega, Nicuragua
No start date - No end date	5569	Lazos de Agua - Jinotega, Nicuragua
No start date - No end date (actual: 500 / target: 1000)	5569	Lazos de Agua - Jinotega, Nicuragua
No start date - No end date (actual: 650 / target: 500)	4272	WASH Alliance Nepal Lumanti - 2016, Kohalpur WASH Project
No start date - 2016-12-31 (actual: 116608 / target: 100,000)	4206	WASH Alliance Uganda - Amref Uganda 2016
No start date - 2016-12-31 (actual: 1,379,799 / target: 1,000,000)	4206	WASH Alliance Uganda - Amref Uganda 2016
No start date - 2016-12-31 (actual: 40 / target: 40)	4206	WASH Alliance Uganda - Amref Uganda 2016
No start date - 2016-12-31 (actual: 40 / target: 40)	4206	WASH Alliance Uganda - Amref Uganda 2016
No start date - 2016-12-31 (actual: 4 / target: 4)	4206	WASH Alliance Uganda - Amref Uganda 2016
No start date - 2016-12-31 (actual: 72 / target: 72)	4206	WASH Alliance Uganda - Amref Uganda 2016
No start date - 2016-12-31 (actual: 2 / target: 2)	4206	WASH Alliance Uganda - Amref Uganda 2016
No start date - 2016-12-31 (actual: 220 / target: 220)	4206	WASH Alliance Uganda - Amref Uganda 2016
No start date - 2016-12-31 (actual: 4000 / target: 4000)	4206	WASH Alliance Uganda - Amref Uganda 2016
No start date - 2016-12-31 (actual: 8595 / target: 6000)	4206	WASH Alliance Uganda - Amref Uganda 2016
No start date - No end date (actual: 55 / target: 50)	4273	WASH Alliance Nepal Enpho - 2016

[zzgvh]

I've made the refactoring to use FKs for Indicators and IndicatorPeriods relations to parents. A little tweaking of the tests were required to make them pass but I think they are legitimate changes.

What's missing is a thorough search of the code to see if I've missed places that fiddle with the results framework. And a migration strategy that assigns FKs to all current objects in the live DB 😛 And maybe more tests 😉

[punchagan]

Closed in #2530