Add units agnostic harmonisation

[znicholls]

Add convenience functions which make it simpler to harmonise a number of timeseries and can also handle mismatches in units.

There's a few problems this is aiming to solve:

1. There's no obvious interface that would just work with the output of iamdf.timeseries() (maybe I've missed something)
2. The fact that there's not an obvious interface to use if you just had a single timeseries you wanted to harmonise. At the moment, you'd need to instantiate a harmoniser, make sure the rc configurations are set right (maybe, depending on which class you started with) etc. and then call harmonise before extracting the output. This new convenience function lets you just throw in the timeseries to harmonise, the history and the harmonisation year and returns you the output.
3. Related to the above, if you want to harmonise lots of scenarios, you have to write your own loops to go through each model and scenario combination. This interface handles all that for you.
4. The current interfaces explode if the units aren't just right (CO2 has to be in MtCO2 / yr, there's a whole bunch of gases which are expected to be in kt etc.). The new interface handles unit conversion (although it's a bit clunky cause pyam/scmdata aren't dependencies, if they were it could be even simpler again).

All I've done is add a single module, convenience, with a single function, harmonise_all, that addresses the issues above. The new tutorial notebook gives an illustration of how to use it and demonstrates a) how it can handle unit conversion and b) how the default decision tree can be overridden.

The changes in methods are to handle the time columns being integers rather than strings. They don't break any other tests so I think they're written in a suitably flexible way. I also picked up a dependence on the gas in the default decision tree, but this appears to be undocumented. I can make a separate issue for that if helpful.

[znicholls]

@lewisjared fyi

[znicholls]

@jkikstra or @gidden this is a pretty big PR. I'm happy to split it into pieces if helpful, but the general approach is hopefully fairly clear if you look at the documentation?

[gidden]

Hey @znicholls I can try to get to this in the next day or so. It would be useful to add a bit more context and flavor to the description which would help me understand the motivation (what didn't work) and solution.

[znicholls]

I can try to get to this in the next day or so

Awesome thanks. There's not a massive rush unless @jkikstra wants to simplify the WG3 process by using something like this.

It would be useful to add a bit more context and flavor to the description which would help me understand the motivation (what didn't work) and solution.

No worries, done

[znicholls]

Suggested change

	# unclear why we don't use pyam or scmdata for filtering
	# unclear why we don't have pyam or scmdata as a dependency as this would simplify filtering and other handling

[gidden]

Hi @znicholls , starting to look at this now. It's a great addition, and if it's ok for you and @lewisjared and @jkikstra I'd like to try to take a deeper look to see how much of this can be incorporated in the guts of aneris.

So if it's ok to delay a detailed review for sometime, that'd be great.

To answer a lot of your questions very simply, aneris predates pyam, so did not use it of course..

[znicholls]

Yep no worries, this can just sit until you get time to dive into the details. Makes sense re pyam, might be a useful dependency now.

…

On Thu, 2 Sep 2021 at 2:39 am, Matthew Gidden ***@***.***> wrote: Hi @znicholls <https://github.com/znicholls> , starting to look at this now. It's a great addition, and if it's ok for you and @lewisjared <https://github.com/lewisjared> and @jkikstra <https://github.com/jkikstra> I'd like to try to take a deeper look to see how much of this can be incorporated in the guts of aneris. So if it's ok to delay a detailed review for sometime, that'd be great. To answer a lot of your questions very simply, aneris predates pyam, so did not use it of course.. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#42 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFUH5G62AX3F2MMQ5L426XLT7ZJLRANCNFSM5C7225UA> .

[gidden]

Hey @znicholls - picking this up again so WG3 can get out the door. I have not had time to do any more robust thinking/coding on this. Accordingly, I suggest the following way forward:

1. I will restart tests so we make sure nothing has affected them in the intervening time
2. can you @znicholls make an issue describing what would be needed for migrating these improvements into the body of aneris please?
3. once both are complete, I'll merge

[znicholls]

2. can you @znicholls make an issue describing what would be needed for migrating these improvements into the body of aneris please?

Brain dump in #46. Perhaps not as clear as we would hope but it should get to the issues (although you might have to wade through some waffle to get there)

[gidden]

Thanks @znicholls !