Finding electricity at-grid generation processes for residual grid mix

[dt-woods]

One of the design goals for the NetlOlca class is to support elci_to_rem Python project, which creates residual grid mix percentages by fuel type for each balancing authority (2020 data). These data are exported to CSV as shown in the table below.

The goal is to update electricity generation mixes (as found in openLCA) to reflect the new percentages (i.e., "Gen_Ratio_new" in the table).

Subregion	FuelCategory	Electricity	Generation_Ratio	BA_CODE	Electricity_new	Gen_Ratio_new
Alcoa Power Generating, Inc. - Yadkin Division	HYDRO	1196076	1	YAD	1181037.97	1
Arizona Public Service Company	BIOMASS	222976	0.012176	AZPS	193574	0.010768
Arizona Public Service Company	COAL	10748972	0.586951	AZPS	10748972	0.597912
Arizona Public Service Company	GAS	5018295	0.274025	AZPS	5018295	0.279143
Arizona Public Service Company	SOLAR	1193234	0.065157	AZPS	1035891	0.057621
Arizona Public Service Company	SOLARTHERMAL	775704	0.042358	AZPS	673418	0.037459
Arizona Public Service Company	WIND	354061	0.019334	AZPS	307374	0.017098
Arlington Valley, LLC - AVBA	GAS	2111383	1	DEAA	2111383	1

Assuming that electricity generation comes from the Federal LCA Commons, it generally falls into the following structure:

• Processes
  • 22: Utilities
    • 2211: Electric Power Generation, Transmission and Distribution

Here's where things get funny. It seems that BA electricity generation is split. There are sub-folders that are fuel-based (e.g., BIOMASS, COAL, GAS, GEOTHERMAL) and there are combined processes (e.g., Electricity; at grid; generation mix - Arizona Public Service Company), which contain all fuel categories.

The individual fuel category process appear to have all upstream and emissions data associated with them.

The Balancing Authority (at grid, generation mix) only have the fraction of 1 MWh for each fuel category. Therefore, the goal is to update these mix processes.

The first step is to find all processes associated with "Electricity; at grid; generation mix - BA NAME - BA". Perhaps query with regular expressions?

Example code (assuming IPC Server connection to aluminum_single_system.zolca).

from netlolca.NetlOlca import NetlOlca
netl = NetlOlca()
netl.connect()
netl.read()
# Check to see if electricity flows are present 
netl.print_descriptors(d_type=netl.get_spec_class("Process"))
# Yes, they are.

Next steps: go to get_descriptors and iterate over each object, checking to see if their name matches the query.

Assuming you already ran the code above to create and connect the NetlOlca to openLCA.

import re

my_d = netl.get_descriptors(netl.get_spec_class("Process"))

# Get list of process UUIDs for each generation mix
q = re.compile("^Electricity; at grid; generation mix - (.*)$")
my_matches = []
num_d = len(my_d)
for i in range(num_d):
    ref = my_d[i]
    r = q.match(ref.name)
    if r:
        # Match group 1 should be BA name
        # print(i)
        # print(r.group(1))
        my_matches.append(tuple([ref.id, r.group(1)]))

m = my_matches[0]
p = netl.query(netl.get_spec_class("Process"), m[0])
netl.print_as_dict(p)
# It looks like the data are in the "exchanges" property
# Note: there are always two or more exchanges: one output and X inputs
# The input exchange has the grid mix values
for p_ex in p.exchanges:
    if p_ex.is_input:
        print("%s %s" % (p_ex.amount, p_ex.description))
        p_ex.description # (e.g., 'from SOLARTHERMAL - BA NAME')
        p_ex.amount # THIS IS THE OLD GRID MIX (normalized to 1.0 MWh)

Next Steps

Updated, 29 August 2023

• For each input Exchange, find the Fuel category and BA name from description (e.g., "from GAS - BA NAME").
• Find the residual grid mix values from the CSV for the BA + Fuel.
• Create new process ID (to avoid overwriting)
• Edit the Process name to: "Electricity; at grid; residual generation mix - BA NAME"
• Edit the exchange property of the process root class with new mixes, then
• add back into the project (e.g., IPC or JSON-LD file)

Find Exchange Fuel Name

# Get list of process UUIDs for each generation mix
my_d = netl.get_descriptors(netl.get_spec_class("Process"))
q = re.compile("^Electricity; at grid; generation mix - (.*)$")
my_matches = []
for ref in my_d:
    r = q.match(ref.name)
    if r:
        my_matches.append(tuple([ref.id, r.group(1)]))

q2 = re.compile("^from (\\w+) - (.*)$")
for m in my_matches:
    ba_uid, ba_name = m
    p = netl.query(netl.get_spec_class("Process"), ba_uid)
    # Note: for the electricity grid mixes, there are always
    # two or more exchanges: one output and X inputs.
    # The input exchanges have the grid mix values we want.
    print("-"*len(ba_name))
    print(ba_name)
    print("-"*len(ba_name))
    for p_ex in p.exchanges:
        if p_ex.is_input:
            f_name = ""
            r = q2.match(p_ex.description)
            if r:
                f_name = r.group(1)
            print("  %s %s" % (p_ex.amount, f_name))

Produces the following output:

------------------
Homestead, City of
------------------
  1.0 GAS
----------------------------
Arlington Valley, LLC - AVBA
----------------------------
  1.0 GAS
------------------
Seattle City Light
------------------
  0.0023448010247781306 BIOMASS
  0.9976551989752219 HYDRO
----------------------------------------------
Midcontinent Independent System Operator, Inc.
----------------------------------------------
  0.1491793527781002 NUCLEAR
  1.4140634771425983e-05 OFSL
  0.0003539043515505538 SOLAR
  0.12404482693331012 MIXED
  3.4931601250110966e-05 OIL
...
-----------------------------------------------------------
Western Area Power Administration - Upper Great Plains West
-----------------------------------------------------------
  0.06384593356651642 WIND
  0.9361540664334835 HYDRO

Read Residual Grid Mix Data

import os
import pandas as pd
def get_residual_mix(u_choice, d_dir):
    csv_files = {
        1: "res-mix_2020_rec-keep_agg-area.csv",
        2: "res-mix_2020_rec-keep_agg-count.csv",
        3: "res-mix_2020_rec-zero_agg-area.csv",
        4: "res-mix_2020_rec-zero_agg-count.csv"
    }
    if u_choice not in csv_files.keys():
        raise IndexError("Choice %s not found!" % u_choice)
    csv_file = csv_files[u_choice]
    csv_path = os.path.join(d_dir, csv_file)
    if not os.path.isfile(csv_path):
        raise OSError("Missing file, %s" % csv_path)
    return pd.read_csv(csv_path)

csv_dir = os.path.join(
    os.path.expanduser("~"), "Workspace", "projects", "residual_mix")
df = get_residual_mix(1, csv_dir)

Produces the following output

                                          Subregion FuelCategory   Electricity  ...  BA_CODE Electricity_new  Gen_Ratio_new
0    Alcoa Power Generating, Inc. - Yadkin Division        HYDRO  1.196076e+06  ...      YAD    1.181038e+06       1.000000
1                    Arizona Public Service Company      BIOMASS  2.229760e+05  ...     AZPS    1.935738e+05       0.010768
2                    Arizona Public Service Company         COAL  1.074897e+07  ...     AZPS    1.074897e+07       0.597912
3                    Arizona Public Service Company          GAS  5.018295e+06  ...     AZPS    5.018295e+06       0.279143
4                    Arizona Public Service Company        SOLAR  1.193234e+06  ...     AZPS    1.035891e+06       0.057621
..                                              ...          ...           ...  ...      ...             ...            ...
327                    B.C. Hydro & Power Authority          ALL  1.000000e+00  ...     BCHA    1.000000e+00       1.000000
328                       Hydro-Quebec TransEnergie          ALL  1.000000e+00  ...      HQT    1.000000e+00       1.000000
329                                  Manitoba Hydro          ALL  1.000000e+00  ...     MHEB    1.000000e+00       1.000000
330                   New Brunswick System Operator          ALL  1.000000e+00  ...     NBSO    1.000000e+00       1.000000
331                                    Ontario IESO          ALL  1.000000e+00  ...     IESO    1.000000e+00       1.000000

[332 rows x 7 columns]

Query Residual Mix

# Get list of process UUIDs for each generation mix
my_d = netl.get_descriptors(netl.get_spec_class("Process"))
q = re.compile("^Electricity; at grid; generation mix - (.*)$")
my_matches = []
for ref in my_d:
    r = q.match(ref.name)
    if r:
        my_matches.append(tuple([ref.id, r.group(1)]))

q2 = re.compile("^from (\\w+) - (.*)$")
for m in my_matches:
    ba_uid, ba_name = m
    p = netl.query(netl.get_spec_class("Process"), ba_uid)
    for p_ex in p.exchanges:
        if p_ex.is_input:
            f_name = ""
            r = q2.match(p_ex.description)
            if r:
                f_name = r.group(1)
            a = df.query("(Subregion == '%s') & (FuelCategory == '%s')" % (
                ba_name, f_name))
            print("Found %s %s rows for %s" % (len(a), f_name, ba_name))

Using df = get_residual_mix(1, csv_dir) as shown in the code snipped above, the output produces some interesting edge cases.

Edge Cases

1. No matching Balancing Authority in df (e.g., Homestead, City of).
   • Note that this case, there is only fuel category and it is GAS, which means that it could be preserved (i.e., there is no REC to be removed).
   • Consider a switch for "keep missing BAs"
   • Same goes for:
     • New Harquahala Generating Company, LLC - HGBA
     • Gila River Power, LLC
2. Missing "other fuel" category (OTHF) from a mixed fuel region (e.g., Midcontinent Independent System Operator, Inc.)
   • Note, in this case, since there are other fuel categories (e.g., nuclear, mixed, oil, gas, coal, other fossil, biomass, wind, and hydro), it may be reasonable to simply omit this category in the residual mix
3. Missing mixed category
   • Note in this case of City of Tacoma, Department of Public Utilities, Light Division, there is only one other fuel type, hydro, and in the df, there is only one fuel category, hydro; therefore, it may be reasonable to omit the mixed category from the residual mix
   • In the case of Public Service Company of Colorado, there are several other fuel categories (mainly coal, gas, and wind), perhaps it's fine to omit the mixed category from the residual mix
   • In the case of Associated Electric Cooperative, Inc., the df shows primarily coal and gas, and the mixed category in openLCA is <0.01%
4. Missing renewable category from a region with several fuel types
   • Note in the PacifiCorp West case, the geothermal mix value is quite small (0.04%) and since it is not found in the df, it may be reasonable to omit it from the residual mix
   • In the Tampa Electric Company case, the biomass is 0.01%
5. Multiple missing fuel categories (e.g., Duke Energy Florida, Inc.)
   • In this case, hydro made up 0.05% (negligible), but coal made up 21%.
   • Based on a quick literature search, it appears that Florida is getting rid of all its coal (see here and here), so I guess it's okay to omit both hydro and coal from residual mix (same issue with coal in Florida Power & Light Co. and JEA)
6. Missing fossil category
   • In the case of Nevada Power Company, oil is ~0.00%
   • In the case of Public Service Company of New Mexico, oil is ~0.00%

Found 0 GAS rows for Homestead, City of
Found 1 NUCLEAR rows for Midcontinent Independent System Operator, Inc.
Found 0 OFSL rows for Midcontinent Independent System Operator, Inc.
Found 1 SOLAR rows for Midcontinent Independent System Operator, Inc.
Found 1 MIXED rows for Midcontinent Independent System Operator, Inc.
Found 1 OIL rows for Midcontinent Independent System Operator, Inc.
Found 1 GAS rows for Midcontinent Independent System Operator, Inc.
Found 1 COAL rows for Midcontinent Independent System Operator, Inc.
Found 1 OTHF rows for Midcontinent Independent System Operator, Inc.
Found 1 BIOMASS rows for Midcontinent Independent System Operator, Inc.
Found 1 WIND rows for Midcontinent Independent System Operator, Inc.
Found 1 HYDRO rows for Midcontinent Independent System Operator, Inc.
Found 0 MIXED rows for City of Tacoma, Department of Public Utilities, Light Division
Found 1 HYDRO rows for City of Tacoma, Department of Public Utilities, Light Division
Found 1 SOLAR rows for PacifiCorp West
Found 1 HYDRO rows for PacifiCorp West
Found 1 BIOMASS rows for PacifiCorp West
Found 0 GEOTHERMAL rows for PacifiCorp West
Found 1 GAS rows for PacifiCorp West
Found 1 WIND rows for PacifiCorp West
Found 1 OIL rows for Duke Energy Florida, Inc.
Found 1 MIXED rows for Duke Energy Florida, Inc.
Found 0 HYDRO rows for Duke Energy Florida, Inc.
Found 1 BIOMASS rows for Duke Energy Florida, Inc.
Found 1 GAS rows for Duke Energy Florida, Inc.
Found 0 COAL rows for Duke Energy Florida, Inc.
Found 1 SOLAR rows for Duke Energy Florida, Inc.
Found 1 GAS rows for Tampa Electric Company
Found 1 SOLAR rows for Tampa Electric Company
Found 0 BIOMASS rows for Tampa Electric Company
Found 1 MIXED rows for Tampa Electric Company
Found 1 OTHF rows for Tampa Electric Company
Found 1 GAS rows for Nevada Power Company
Found 1 OTHF rows for Nevada Power Company
Found 1 WIND rows for Nevada Power Company
Found 0 OIL rows for Nevada Power Company
Found 1 HYDRO rows for Nevada Power Company
Found 1 GEOTHERMAL rows for Nevada Power Company
Found 1 COAL rows for Nevada Power Company
Found 1 BIOMASS rows for Nevada Power Company
Found 1 MIXED rows for Nevada Power Company
Found 1 SOLARTHERMAL rows for Nevada Power Company
Found 1 SOLAR rows for Nevada Power Company
Found 0 GAS rows for New Harquahala Generating Company, LLC - HGBA
Found 1 HYDRO rows for Public Service Company of Colorado
Found 1 BIOMASS rows for Public Service Company of Colorado
Found 1 COAL rows for Public Service Company of Colorado
Found 1 WIND rows for Public Service Company of Colorado
Found 1 GAS rows for Public Service Company of Colorado
Found 1 SOLAR rows for Public Service Company of Colorado
Found 0 MIXED rows for Public Service Company of Colorado
Found 0 GAS rows for Gila River Power, LLC
Found 1 NUCLEAR rows for Florida Power & Light Co.
Found 1 GAS rows for Florida Power & Light Co.
Found 1 MIXED rows for Florida Power & Light Co.
Found 1 SOLAR rows for Florida Power & Light Co.
Found 1 OIL rows for Florida Power & Light Co.
Found 1 BIOMASS rows for Florida Power & Light Co.
Found 0 COAL rows for Florida Power & Light Co.
Found 1 HYDRO rows for Florida Power & Light Co.
Found 1 MIXED rows for JEA
Found 1 BIOMASS rows for JEA
Found 1 GAS rows for JEA
Found 1 SOLAR rows for JEA
Found 0 COAL rows for JEA
Found 0 COAL rows for Tucson Electric Power
Found 1 GAS rows for Tucson Electric Power
Found 1 MIXED rows for Tucson Electric Power
Found 1 SOLAR rows for Tucson Electric Power
Found 1 HYDRO rows for Associated Electric Cooperative, Inc.
Found 1 OIL rows for Associated Electric Cooperative, Inc.
Found 1 GAS rows for Associated Electric Cooperative, Inc.
Found 1 WIND rows for Associated Electric Cooperative, Inc.
Found 1 BIOMASS rows for Associated Electric Cooperative, Inc.
Found 0 MIXED rows for Associated Electric Cooperative, Inc.
Found 1 COAL rows for Associated Electric Cooperative, Inc.
Found 1 MIXED rows for Public Service Company of New Mexico
Found 1 GAS rows for Public Service Company of New Mexico
Found 1 WIND rows for Public Service Company of New Mexico
Found 1 GEOTHERMAL rows for Public Service Company of New Mexico
Found 1 SOLAR rows for Public Service Company of New Mexico
Found 1 BIOMASS rows for Public Service Company of New Mexico
Found 0 OIL rows for Public Service Company of New Mexico
Found 1 COAL rows for Public Service Company of New Mexico

Quality Control Testing

1. Find duplicate residual generation grid mixes (e.g., executing run.py on the same project multiple times)
2. Find where exchange amounts do not balance (i.e., inputs do not equal outputs)

Test 2 - Find where exchange amounts do not balance

The issue is that for some Balancing Authority areas, the fuels in 2020 are different than those in 2016 or 2018. Therefore, not only must existing fuels be updated (or removed), but also new inputs be added when needed.

For example, 'Portland General Electric Company' does not have coal or wind in the Electricity Baseline, which are both found in the 2020 dataset.