quick results for Belgium seem way off for EVs

[dominiquedemunck]

Hello

Impressive tool which can really be helpful!
I didn't do a dive in the data or code, but the quick results for Belgium (new cars), give only a 20% advantages for EVs compared to Diesel:
https://carculator.psi.ch/display_quick_results/BE

This would contradict other recent estimates like:
https://www.transportenvironment.org/what-we-do/electric-cars/how-clean-are-electric-cars

or:
https://www.carbonbrief.org/factcheck-how-electric-vehicles-help-to-tackle-climate-change

and also this tool:

https://climobil.connecting-project.lu/

[floriandierickx]

Hey, I allow myself to jump into this conversation/question (in complement to our exchange on Twitterrr, including with @thomasgibon).

Excuses for the rather 'lazy' reflection (I did not delve into all the 'nitty-gritty' details of either carculator or climobil, apart from a quick scan for methodology, energy system trajectory data), but: could it be that the two approaches can be broadly distinguished by:

• Carculator: input-output/LCA matrix using Ecoinvent/... (resulting reflection: to which extent is Ecoinvent up to date?). Did you also include other databases (ex. sectoral exchanges represented in IO databases, ...) ?
• Climobil: compiling recent existing LCA studies, tailored to a specific context/supply chain

[thomasgibon]

Hi all! Thanks for the interest.

1. First off, climobil is clearly much simpler than carculator. Our website lets you manipulate various parameters that are then multiplied with fixed (e.g. in the case of emissions embodied in manufacturing) or variable emission factors. What you get is the result of a linear combination. What you get in carculator can be literally based on a full-scale LCA if you so choose.
2. Carculator models vehicle archetypes (I think? i.e. small, mid-size, SUV...) whereas Climobil lets you choose weight/emissions or specific model, did you make sure to choose equivalent cars before comparing?
3. Real-world emissions are estimated in climobil (we simply estimate that emissions are ~39% higher than NEDC-declared, following an ICCT publication), but they are actually calculated in carculator (again, @romainsacchi will probably clarify) - I am assuming this entails a potentially big difference for the emissions of the ICE.
4. We do not credit GHG emissions at any step (neither for using recycled material at production, or for providing recyclable material at the end-of-life treatment stage), this might play a role too...

There are probably other reasons, but that's already what I can think of from the top of my head.

[floriandierickx]

Hey @thomasgibon, thank you for these insights. I hope to find time to dive deeper in the models and start with, for example, comparing a similar case study and get back to this list to better understand the scope and modelling frameworks.

[romainsacchi]

Hi,

the difference in terms of GWP between the mid-size BEV and its ICEV counterpart for the quick case of Belgium is indeed not flagrant. Having had a quick look, it seems to mainly come from the electricity mix assumed to charge the battery.
Using default values, the vehicles has a lifetime of 200,000 km, with an annual mileage of 12,000 km/year, giving it a lifetime of about 17 years.

Knowing this, the model looks at how the Belgian grid is expected to develop in the future, and will calculate a lifetime-weighted electricity mix, assuming that the annual mileage is the same throughout the 17 years of lifetime.

So, starting from these projections for Belgium:

country	year	Hydro	Nuclear	Gas	Solar	Wind	Biomass	Coal	Oil	Geothermal	Waste	Biogas CCS	Biomass CCS	Coal CCS	Gas CCS	Wood CCS
BE	2020	0	0.5	0.24	0.05	0.16	0.04	0	0.01	0	0	0	0	0	0	0
BE	2025	0.01	0.06	0.56	0.06	0.24	0.06	0	0.01	0	0	0	0	0	0	0
BE	2030	0.01	0	0.58	0.06	0.27	0.07	0	0.01	0	0	0	0	0	0	0
BE	2040	0.01	0	0.61	0.05	0.26	0.07	0	0	0	0	0	0	0	0	0
BE	2050	0.01	0	0.59	0.05	0.28	0.07	0	0	0	0	0	0	0	0	0

the model will use the following mix to represent the 2020-2037 period:

Hydro	Nuclear	Gas	Solar	Wind	Biomass	Coal	Oil	Geothermal	Waste
0.008235	0.105882	0.517824	0.057	0.240529	0.061765	0	0.008765	0	0

and this is quite intensive in terms of GHG because of the natural gas power plant dominance. Of course, if you shorten the lifetime or increase the annual mileage, the mix will change, which you can directly see in the "configuration" page.

Whether these projections are accurate or not, that's indeed another issue. We get them from the EU Reference Scenario 2016, which is probably a bit outdated, but we have not found anything better with a country-wise resolution. If you have a better source, let me know.

Otherwise, I think carculator yields higher GWP scores for the battery production (25 g CO2-eq./km for a 50 kWh battery) compared to the graphs you show above. This is due to the ecoinvent update for cobalt production, which has multiplied by 3 the carbon footprint of cobalt.

Last but not least, we will soon update the model as we have received new calibration data from the European Commission's CO2 test database for 2019 vehicles, but this will mostly make ICEVs a bit more efficient (so this will decrease further the difference) than before (diesel and gasoline ICEVs are now close to 130g CO27km of direct emissions) ...

So maybe our projections for Belgium grid mix (and other countries) should be updated...

[romainsacchi]

@thomasgibon @floriandierickx @dominiquedemunck Also, there's maybe the fact that for Belgium during that period, the model considered 7% bioethanol and 5% biodiesel in the blend (reducing direct GHG emission by that much) -- I got these blends from the IEA's World Extended Energy balance.

[floriandierickx]

Hey @romainsacchi, thanks a lot for your answers (althought it was not my question, I think I can say that for all present here :) ).

I went briefly through the EU Reference Scenario 2016, and I have been looking into comparing it with other projections specifically for Belgium (although the network is interconnected and influenced by electricity exchange, another question to tackle..). I don't have a tangible result yet, but broadly speaking, I think it is a good representation of reality and future prospect (gas is currently projected to increase in the coming years because of a planned phase-out of nuclear). I will try to get other numbers to compare with (from other scenario's such as Belgium 2050: abstract online, more details upcoming), the grid manager (Elia), etc..) and experiment with how it influences the results. Also the shares of bioenergy are indeed a really important aspect, but I'm - for now - technically not sufficiently knowledgeable on that matter.

Interesting to learn about the details on the Ecoinvent update as well. I know that Ecoinvent is an authority in LCA databases, but it's only a pity that it's not open (although I might be mistaken here, as I did see an open version that enables consultation of individual processes online on the ecoinvent-website, which might be used to go through the assumptions of carculator?). I did experiment with it in the past, but curious to dive deeper... :))

Last but not least, we will soon update the model as we have received new calibration data from the European Commission's CO2 test database for 2019 vehicles, but this will mostly make ICEVs a bit more efficient (so this will decrease further the difference) than before (diesel and gasoline ICEVs are now close to 130g CO27km of direct emissions) ...

Linked with point 3 from @thomasgibon: related to the the ICCT finding (if I understood correctly) that real-world emissions are 39 % higher compared to the new WLTP cycle: an 'LCA Q&A' from Transport & Environment (2020) [1] explains that they used real-world ("ex-post") emission data from the ​spritmonitor.de​ database, in order to model the 10 most common car models on the market per category. I don't know the database, it might not be capturing all the car types of the market and neither have sufficient data for new cars (and therefore hard to use from a legislative point of view with decreasing-but-not-entirely-zero-emission norms / for a detailed LCA for each and every car on the market), but it might be interesting to compare with at some point ?

[1] T&E. “Frequently Asked Questions on T&E’s EV LCA Tool,” April 2020. https://www.transportenvironment.org/sites/te/files/downloads/FAQ%20T%26E%27s%20EV%20LCA%20tool_1.pdf, page 1:

Regulatory lab test protocols (WLTP or NEDC) are not used here because they seriously underestimate real-world fuel consumption and therefore CO2 emissions (real-world test results diverge from lab tests by 39%, according to ​the ICCT​).
For diesel and petrol cars, we used the average real-world consumption (CO2 emissions) of the 2018 top 10 most-sold vehicles for each category.
Fuel and energy consumption is based on real-world consumption declared values from millions of drivers, taken from the ​spritmonitor.de​ database.

[romainsacchi]

Hi @floriandierickx,

to be precise, we use the EU's database to calibrate the vehicles' weight and we fine tune the different drivetrain compoentns' efficiencies until we match their fuel consumption values (which they get based on the NEDC driving cycle). So we do not use their FC values as they are, but instead there's a "calibrated" energy model which calculates the FC based on the weight, road gradient, engine, gearbox and axles efficiencies, rolling resistance, etc.

We thought about the spritmonitor data, but unlike the ICCT report, we found that the results look a bit biased. When one looks at gasoline or diesel cars present there, 60% are hybrid vehicles. Once you get rid of those, their FC is suspiciously low. We thought that maybe users that register their fuel consumption are somewhat incentivized to drive more economically. So, we decided in the end not to use it. I guess if spritmonitor grants you the access officially, they get you "cleaner" data, I don't know. I do have all the scrapped data though, if you are interested. Also, spritmonitor only have a few thousands of vehicles, which is insignificant compared to the 290+ million cars in Europe.

[romainsacchi]

Here is the sort of calibration we do for the vehicle mass per size class. There's about 15 million points per year, which we reduce down to average by car model.

And then we fine-tune efficiencies to match approximately their FC values using the NEDC driving cycle (red dots), and we show the WLTC-based values as well (orange dots).

[floriandierickx]

😮 So I understand you model your own emission/driving tests to the different driving test (NECP/WLTP) values ? That's really impressive.. This with the purpose of being able to modify different aspects/parameters of a 'generic' car in all detail, in order to derive the influence of each parameter on the impact indicators? (or maybe in case somebody wants to do an electric retrofit of his/her existing car and wants to lower the impact - I would then go for a velomobile, but still :p). I have yet to look into the details of the calculations and documentation (I hope to find time for that..), thank you for the 'quick summary' and insights here.

I did find the explanation on the hybridisation (hypothesis) and how to disable it (hypothesis) also in the documentation now.

Also thanks a lot for your insights in the spritmonitor database. For hybrid vehicles a big difference might not come as a surprise, so I'll 'touch with care' conclusions for ICEVs (including resulting reports) ... :)

[romainsacchi]

Hi @floriandierickx ,

indeed, the car model combined with the energy consumption model allows to completely parametrize the vehicle (no of passengers, engine efficiency, battery mass, frontal area) and its use phase (road gradient, speed profile, etc.). In the examples notebook, we show how to do a sensitivity analysis, where we change each input parameter individually by +10% and look at the effect on the GWP score. Here for example with a mid-size BEV:

[floriandierickx]

Cool! That's really insightful.. ! If I understand correctly, this allows to do a kind of impact analysis of changing each of the variables, for example using 'best available/projected technology'-parameters (I guess a 10 % decrease in frontal area is easier to pursue compared to 10 % battery cell mass increase).

[dominiquedemunck]

@romainsacchi

This might be helpful to make the tool more acurate:

1. The EU 2016 Reference publication is way too outdated because of huge market (price of offshore wind, solar) and policy evolutions (EU Green Deal, US aiming at 0 carbon electr in 2035, ... ).

This publication offers a recent and very detailed estimate of CO2 intensities in Europe, with database available free for download (see download link at end of paper):
https://www.mdpi.com/1996-1073/14/8/2165

As referenced here https://twitter.com/SamHamels/status/1407243178294534147

Example data:

Note about why it's good to use average CO2 intensity (instead of marginal) you will find argumentation here https://twitter.com/AukeHoekstra/status/1407081827555233799

2. Since the weight of the mileage is in the first years of the car use, I would not use a fixed annual mileage, A better estimate is to use the official Dutch km values per year, which are a good EU proxy since the Netherlands have an average age fleet park.

On average, Dutch cars do about 50% of the km's when they are younger then 8 years.
You get 97% of km's driven by cars younger then 21 years. See datasheet referenced here (second tab)
https://twitter.com/DominiqueDeMunc/status/1390035588850532364

Or summary:

Cumulative part of kms	Total
4.39%	0 jaar
12.35%	1 jaar
19.64%	2 jaar
26.02%	3 jaar
33.58%	4 jaar
39.13%	5 jaar
44.56%	6 jaar
50.69%	7 jaar
57.12%	8 jaar
62.32%	9 jaar
66.33%	10 jaar
70.70%	11 jaar
74.85%	12 jaar
79.02%	13 jaar
82.65%	14 jaar
86.04%	15 jaar
89.05%	16 jaar
91.70%	17 jaar
93.86%	18 jaar
95.77%	19 jaar
97.20%	20 jaar

[romainsacchi]

Hi @dominiquedemunck ,

great find! I looked into the data and could not find the technology mix per country-year. I contacted the author to see if he'd be kind to share that with us - as we cannot directly use CO2 intensity factors. Update: by the time I'm writing this, the author replied with the electricity mixes (which are actually available here). So, I'll open an issue about integrating this.

About marginal electricity mix: it seems to me that this person is missing the point, or that he maybe lacks a system perspective. Using the marginal mix is not about finding out which electricity providers will actually charge the battery of an EV at this time of the day in this region, or whatever. In LCA, or rather consequential LCA, it is about finding out which technology will react (by reacting, we mean investing in additional production capacity in additional to the regular rate of capital renewal) as a result of increasing the demand for electricity. We are not interested in knowing the origin of the electricity that enters the battery, but rather at the system changes that result from charging the battery. As long as the forecast for the electricity system properly account for the expect increase in electricity demand from heat pumps, BEVs, etc., I personally find it completely fine. If in a given country or network of interconnected grids, the RES are supposed to grow to face an increase in demand or a phasing out of old technologies, using a marginal mix made of these RES seems fine to me. They will expand because of the battery charging. In that regard, global energy systems can be very helpful to LCA of electric mobility.

About the annual mileage vs. age of the vehicle: that's interesting and I previously found data like this (COPERT and HBEFA). But I was never quite sure of the implementation: I guess we would still need to know the expected lifetime of the vehicle in km but also in years, and integrate that over some distribution to get the annual mileage per year of age. Also, the distribution you show is a fleet average: it won't fit cars with a life expectancy of 10 or 15 years, for example. I guess only a fraction of the fleet makes it to 20 years. Ideally, we would need a simple, yet representative, distribution of the km over the years that would adapt to the life expectancy indicated (in years). I'll open an issue about this task too! If you have a precise idea about this, please let me know.

Thanks for your inputs.

[dominiquedemunck]

I don't think the kind of very detailed vehicle km data your are looking for is readily available.

In this doc you will find attempts for estimates about EU vehicle lifetime and miles driven:
https://www.avere.org/wp-content/uploads/2020/09/englisch_Studie-EAuto-versus-Verbrenner_CO2.pdf

The Dutch data remain my favorite, as this national fleet average, is quite representative for the average EU car.
If you use the Diesel data only, I think you get a good profile for high mileage drivers.

I noticed that google translate worked OK with the Dutch data:

https://translate.google.com/translate?hl=en&sl=auto&tl=en&u=https%3A%2F%2Fopendata.cbs.nl%2F%23%2FCBS%2Fnl%2Fdataset%2F83702NED%2Ftable%3Fts%3D1516822135004

Also note: cars are exported also a lot, especially the older ones out of the EU.
Even in Poland AVERAGE car age is 14 years!

https://www.acea.auto/figure/average-age-of-eu-vehicle-fleet-by-country/

[romainsacchi]

Hi @dominiquedemunck ,

regarding the electricity mix projected for Belgium, I just looked at the data from https://doi.org/10.3390/en14082165.
I normalized the generated volumes and graphed it using the baseline scenario called "National trends" (not enough to reach the Paris Agreement, but quite ambitious nevertheless). Here are the results for Belgium (% gross production mix, excluding trade):

So it does change quite a bit: here the disappearance of nuclear is not compensated by gas (as in the EU 2016 Reference Scenario) but by solar + wind (>70% of the gross production, and this is not even the ambitious scenario!). This seems huge. What does @floriandierickx think about this?

If these projections turn out to be good enough, we will use them.
We might even add an option in the configuration page where the use can choose between the baseline projection or the Paris Agreement-compliant projections.

[romainsacchi]

Hi @dominiquedemunck,

I updated the electricity mixes for European countries (incl. Belgium) based on ENTSO-E TYNPD report projections (https://2020.entsos-tyndp-scenarios.eu/), following the "National Trends" scenario (sort of baseline).
The km-weighted mix for BE (from 2020 to 2037) is now made of 55% renewable.

We now have a 35-40% GHG emissions improvement over a comparable gasoline car (https://carculator.psi.ch/display_quick_results/BE).

But this is admittedly still quite different from this T&E graph:

Hence, to be investigated further.

[dominiquedemunck]

Thanks! Closing in.

The ICCT released its LCA in July, got quite some attention, maybe here you can find other explanations?
https://theicct.org/publications/global-LCA-passenger-cars-jul2021

They also get very low BEV results, less than half of carculator BE result:

There are also refs for vehicle age and kms: