Incomplete CHE data processing relative to SCEC

[irm-codebase]

What happened?

Issue

Our current processing for Swiss data is incomplete relative to SCEC.
This is probably because the Swiss statistical agency has the nasty habit of moving "old" data to a different excel sheet with different format.

Here is SCEC:

year                               2000      2001      2002      2003      2004      2005      2006      2007      2008      2009      2010      2011      2012       2013      2014      2015      2016      2017      2018
cat_name                                                                                                                                                                                                                    
Chemicals Industry             8.170852  8.410216  8.210467  8.621722  8.815315  9.620488  9.347826  9.149713  8.973223  8.395083  8.291800  8.609443  9.417317  16.220007  7.821974  7.783197  7.908832  7.772848  7.652839
Food, beverages and tobacco    4.466229  4.459079  4.288786  4.179849  4.104345  4.262883  4.658903  4.579539  4.564366  5.159636  5.274459  4.969356  4.795155  10.797802  5.438774  5.082128  4.746453  4.854441  4.708051
Iron and steel                 1.941352  1.942077  2.070284  2.241083  2.146052  2.137193  2.542866  2.621545  2.450503  1.856552  2.248223  2.452570  2.068677   4.053801  2.183272  2.196144  2.126684  2.241020  2.241518
Machinery Equipment            4.778636  4.781407  4.766296  5.023178  4.980083  5.136794  5.180928  5.017178  6.259830  5.648181  6.014993  5.993196  5.988008  11.754309  5.444545  5.602672  5.458318  5.602478  5.319373
Non Ferrous Metals             1.628731  1.569771  1.201696  1.261541  1.414944  1.217512  0.626381  0.564590  0.569226  0.514247  0.615886  0.628877  0.923682   1.731871  0.956130  0.873168  0.787771  0.787682  0.762940
Non-metallic mineral products  4.374657  4.349614  4.936325  5.222827  5.428015  5.394161  5.551043  5.604038  5.575781  5.626835  5.624983  5.659034  5.275591   6.975110  5.437205  4.758991  4.851510  4.924414  4.683079
Other Industrial Sectors       6.072763  6.096098  6.015502  6.048999  6.066597  6.321599  6.456112  6.351951  5.927015  5.543818  5.714487  5.322072  5.043213   8.729175  4.312536  4.270258  4.020389  3.907816  3.827744
Pulp, paper and printing       5.583888  5.627023  6.356727  6.081129  5.917123  5.752701  5.413429  5.724151  5.458365  4.474360  4.805491  4.672642  4.232735   6.866623  3.457969  3.424348  3.136068  3.130626  2.693632
Textiles and leather           0.934428  0.921893  0.854650  0.781862  0.775360  0.673536  0.694857  0.663817  0.650127  0.540690  0.568032  0.532399  0.523548   1.031149  0.518621  0.491143  0.409526  0.422200  0.405880

And here it is in EC (please ignore the difference in cat_name, it's due to the new industry processing):

year                               2013      2014      2015      2016      2017      2018      2019      2020      2021      2022
cat_name                                                                                                                         
Food, beverages and tobacco    5.633679  5.456540  5.098095  4.761161  4.867713  4.814223  5.014412  5.025033  5.134441  4.725216
Machinery Equipment            5.921505  5.476040  5.628026  5.485988  5.624314  5.343572  5.562812  5.167518  5.552539  5.033101
Non Ferrous Metals             0.882493  0.956254  0.873205  0.787774  0.787682  0.762940  0.814884  0.756023  0.863689  0.838353
Non-metallic mineral products  5.310382  5.722890  5.041550  5.117878  5.194309  4.949282  4.965434  4.773973  4.912490  4.900071
Other Industrial Sectors       5.148847  5.031023  4.965775  4.773304  4.566765  4.666243  4.371582  4.153160  3.998016  3.816896
Pulp, paper and printing       4.117979  3.849936  3.644553  3.356705  3.386235  2.874843  2.873955  2.574619  2.787666  2.660201
Textiles and leather           0.540198  0.518811  0.491427  0.416314  0.424167  0.407720  0.419370  0.374273  0.420975  0.385799

Fix

Either add the extra processing, or pivot to a dataset provided by https://opendata.swiss/en/dataset/energiebilanz-der-schweiz as they tend to be more stable

Version

1.0.0

Relevant log output

No response