Provide access to AOGCM tunings besides the default C4MIP_BERN

[safiume]

Describe the bug
live.magicc.org provides tunings for the other bench marked atmospheric-ocean general circulation models mentioned in the main MAGICC reference paper (http://www.atmos-chem-phys.net/11/1417/2011/acp-11-1417-2011.html). These tunings appear to be defined in specific files named MAGTUNE_tuningname.CFG and MAGTUNE_FULLTUNE_tuningname.cfg. They don't seem to exist in the kit delivered by pymagicc or even the windows zip from magicc.org. These files are also used to build up the data used in bulk runs.

The names of the tunings are as follows, and taken from bulk runs from live.magicc.org:

file_tuningmodel_1 = list[
    'c4mip_bern',
    'c4mip_ccsm1',
    'c4mip_climber',
    'c4mip_frcgc',
    'c4mip_hadley',
    'c4mip_ipsal',
    'c4mip_llnl',
    'c4mip_mpi',
    'c4mip_umd2',
    'c4mip_uvic']

file_tuningmodel_2_1 = list[
    'fulltune_cccma_cgcm3_1_t47', 
    'fulltune_cnrm_cm3', 
    'fulltune_csiro_mk3_0',
    'fulltune_gfdl_cm2_0', 
    'fulltune_gfdl_cm2_1',
    'fulltune_giss_model_e_h',
    'fulltune_giss_model_e_r', 
    'fulltune_iap_fgoals1_0_g',
    'fulltune_inmcm3_0',
    'fulltune_ipsl_cm4',
    'fulltune_miroc3_2_hires',
    'fulltune_miroc3_2_medres',
    'fulltune_miub_echo_g',
    'fulltune_mpi_echam5',
    'fulltune_mri_cgcm2_3_2a',
    'fulltune_ncar_ccsm3_0',
    'fulltune_ncar_pcm1',
    'fulltune_ukmo_hadcm3',
    'fulltune_ukmo_hadgem1']

System (please complete the following information):
macOS 10.13.6, pymagicc 1.3.1
(and additionally the native Windows: windows zip file: Magicc6.801 Beta)

Additional context

Some settings are in the appendix tables for the paper mentioned above, but don't appear to match the default settings.

[rgieseke]

Pymagicc wraps the binary as provided on magicc.org.

With the MAGCFG_USER.CFG files available from magicc.org you could reproduce the 171 run combinations of CMIP3/C4MIP tunings with Pymagicc.

The 600 run is another tuning, constrained to historical data and using the C4MIP tunings for the Carbon Cycle I believe. See this paper:

https://www.nature.com/articles/nature08017

@znicholls Thoughts on whether it would make sense to include the CMIP3/C4MIP tunings in the Pymagicc package?

[rgieseke]

If you're interested in general in probabilistic ensembles you should also check out Pyhector and FAIR. The latter has a notebook showing a simple ensemble setup and Hector runs faster than MAGICC (due not copying around many files) and some people also worked on probablistic setups: https://github.com/bvegawe/hector_probabilistic

Should also be useful to include the uncertainty from using a different model ...

[znicholls]

@znicholls Thoughts on whether it would make sense to include the CMIP3/C4MIP tunings in the Pymagicc package?

It would make sense, if we're allowed...

Hector runs faster than MAGICC

Hector runs faster than Pymagicc (for now...)

Should also be useful to include the uncertainty from using a different model ...

very wise

[safiume]

Ok. Yes, as a fall back, I'd have to do all the possible combos, download all settings and subtract the likely defaults that were in user.cfg from parameters.out. That will likely work ... if the files aren't available.

I haven't noticed a performance issue with MAGICC (given it's codebase age and architecture). (Native MAGICC runs my scenarios faster than VMWare can pause Windows.) pymagicc runs a tiny bit slower, but both seem to run decently. Just scanning, Hector doesn't seem as detailed compared to MAGICC. Fair looks interesting...
Should running with the bulk tunings be possible outside of live.magicc.org, waiting the estimated hour and a bit for 600 runs seems acceptable. For what it's worth, live.magicc.org didn't seem to take an hour, and my laptop took less than live.magic did to run the single runs, it may be possible that the bulk runs are optimized within MAGICC such that it won't take under an hour and half.

# NBVAL_IGNORE_OUTPUT
%time results = pymagicc.run(rcp26) ;results = ''
%time results = pymagicc.run(scenarios['emin']); results = ''
%time results = pymagicc.run(rcp26); results = ''

CPU times: user 600 ms, sys: 76.8 ms, total: 677 ms
Wall time: 2.37 s
CPU times: user 461 ms, sys: 60 ms, total: 521 ms
Wall time: 745 ms
CPU times: user 585 ms, sys: 60.3 ms, total: 645 ms
Wall time: 863 ms

Note, I am running without the files copied around per pending fix to #86.

[rgieseke]

Cool, thanks for sharing.

Pymagicc is slower than MAGICC because it additionally copies a few more files around (including a copy of MAGICC and its config itself), because it seemed the easiest way to ensure that nothing get messed up. MAGICC is slow because of file IO.

With Pymagicc's API you can reduce this somewhat, just copy once, run a scenario, adjust config, and repeat ...

Pyhector is really faster than Pymagicc/MAGICC because all settings are set without any config file writing/reading.

magicc.org is fast because results are cached I believe.

On the tunings available from magicc.org, I think you could only reproduce the 171 run combination. To really reproduce the 600 run you'd have to reproduce the probabilistic calibration. The R code is in the supplement of the paper.

[safiume]

Interesting... Oh sure, pymagicc is slower for the first or at most the second time, after that it's fairly quick. Under 1 second for wall clock time is decent. Unfortunately my plate is a bit full with other things for more deep dives...

Just for comparison, I didn't have to wait that long for live.magicc to calculate the 600 for my 4 custom test scenarios. I don't remember exactly but it was less than an hour.

[rgieseke]

Ah, I see, that's good to know ...

[rgieseke]

It would be nice to provide these tunings as well, but as discussed not easy or possible.
Closing for now, hopefully a later Pymagicc version can also provide a probabilistic setup.

[valadilene]

Hello, could anyone here could help me to know how can we reproduce the tunings for 600 probabilistic runs? The file downloaded from the website has only name and I dont know how can I extract the information from it.
Thank you very much.

[znicholls]

could anyone here could help me to know how can we reproduce the tunings for 600 probabilistic runs?

Unfortunately it's not possible as the tunings aren't publicly available, lookout for MAGICC7.

[valadilene]

could anyone here could help me to know how can we reproduce the tunings for 600 probabilistic runs?

Unfortunately it's not possible as the tunings aren't publicly available, lookout for MAGICC

I need to get the 5% and 95% percentile regions for 600 probabilistic runs but the auto-generation function in live.magic.org can only give me the 17, 25, 75, and 83% region. Do you have any idea how possible is it to obtain it? Adjusting different combinations of parameters is difficult for me as I have little understanding of how to make a reasonable modification of those huge parameters. Could you please give me some advice? many thanks!

[znicholls]

Do you have any idea how possible is it to obtain it?

Sadly not, live.magicc.org is not something I use (otherwise we wouldn't work on Pymagicc) and I don't think you will be able to back them out as you can't get the underlying parameter sets. Perhaps you could fit some sort of distribution to the points you do have and extrapolate (although it would really be guesswork as the tails are always hard).

[valadilene]

Perhaps you could fit some sort of distribution to the points you do have and extrapolate (although it would really be guesswork as the tails are always hard).

It seems impossible to reproduce their works and I have to work it out by trying different parameterisation. Thank you for your advice!

[rgieseke]

Some of the files used in the original calibration are available here:

https://web.archive.org/web/20181106205227/https://www.pik-potsdam.de/research/climate-impacts-and-vulnerabilities/research/rd2-flagship-projects/gia/primap/nature-2degc-paper

But i'd recommend waiting for the availability of MAGICC7 ...

[valadilene]

Some of the files used in the original calibration are available here:

https://web.archive.org/web/20181106205227/https://www.pik-potsdam.de/research/climate-impacts-and-vulnerabilities/research/rd2-flagship-projects/gia/primap/nature-2degc-paper

I don't know why I cannot access this link from my PC. If possible, could you please help me to download and send it to my email address?

[valadilene]

Some of the files used in the original calibration are available here:

https://web.archive.org/web/20181106205227/https://www.pik-potsdam.de/research/climate-impacts-and-vulnerabilities/research/rd2-flagship-projects/gia/primap/nature-2degc-paper

I can access the link now, thank you for your great help!

[znicholls]

But i'd recommend waiting for the availability of MAGICC7

Just to reiterate this. MAGICC6 is using input data from 2005 and MAGICC7 will be a major improvement in terms of recreating the last 15 years of observations.

[rgieseke]

Seconded, the link is more of an explanatory illustration why this is hard (i think some files were also part of the supplementary material)!

[valadilene]

Seconded, the link is more of an explanatory illustration why this is hard (i think some files were also part of the supplementary material)!

I have to say parameterisation is very hard to understand and over-complicated as well. I don't think I have enough knowledge about climate change to use it effectively.