Data missing in transport timeseries when running the default config

[adrienmellot]

What happened?

While the minimal config runs well for the implementation of the transport sector, the default config fails because:

• the annual timeseries profile data from RAMP is missing CYP. This is an easy fix by adding CYP: ['ROU'] in the fill-missing-values dictionary in the config file
• some annual demand data for historic-electrification is missing for CHE. The file build/data/transport/annual-road-transport-historic-electrification.csv does not come with CHE. See log below.

Version

1.2.0.dev

Relevant log output

(euro-calliope) adrienmellot@Adriens-MBP euro-calliope % snakemake --profile profiles/conda --configfile="config/default.yaml" "build/data/transport/timeseries/timeseries-passenger-cars-historic-electrification.csv"

Config file config/default.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 2
Rules claiming more threads will be scaled down.
Job stats:
job                                                          count    min threads    max threads
---------------------------------------------------------  -------  -------------  -------------
create_road_transport_timeseries_historic_electrification        1              1              1
total                                                            1              1              1

Select jobs to execute...

[Thu Mar 21 15:50:44 2024]
Job 0: Create timeseries for historic electrified road transport demand
Reason: Missing output files: build/data/transport/timeseries/timeseries-passenger-cars-historic-electrification.csv

Activating conda environment: ../envs/snakemake/euro-calliope/532acc2ee043cde2d8eb0d26b6750c89_
Activating conda environment: ../envs/snakemake/euro-calliope/532acc2ee043cde2d8eb0d26b6750c89_
/Users/adrienmellot/Documents/GitHub/envs/snakemake/euro-calliope/532acc2ee043cde2d8eb0d26b6750c89_/lib/python3.9/site-packages/numpy/lib/arraysetops.py:583: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
  mask |= (ar1 == a)

Traceback (most recent call last):
  File "/Users/adrienmellot/Documents/GitHub/euro-calliope/.snakemake/scripts/tmp2ebh8nme.road_transport_timeseries.py", line 84, in <module>
    create_road_transport_demand_timeseries(
  File "/Users/adrienmellot/Documents/GitHub/euro-calliope/.snakemake/scripts/tmp2ebh8nme.road_transport_timeseries.py", line 70, in create_road_transport_demand_timeseries
    assert not df_timeseries.isna().any(
AssertionError: There are NaN values in the timeseries dataframe

[Thu Mar 21 15:50:54 2024]
Error in rule create_road_transport_timeseries_historic_electrification:
    jobid: 0
    input: build/data/transport/annual-road-transport-historic-electrification.csv, data/automatic/ramp-ev-consumption-profiles.csv.gz
    output: build/data/transport/timeseries/timeseries-passenger-cars-historic-electrification.csv
    conda-env: /Users/adrienmellot/Documents/GitHub/envs/snakemake/euro-calliope/532acc2ee043cde2d8eb0d26b6750c89_

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2024-03-21T155043.561813.snakemake.log

[adrienmellot]

This issue will be fixed at the same time as #313 and #314. The idea is to retrieve the EV historical electricity consumption by subtracting rail share from the total share.