[FEATURE REQUEST] US: NEI 2016 (P. Campbell et al)

[yantosca]

This update by co-authors Patrick Campbell, Barron Henderson (@barronh), and Lyssa Freese (@lfreese) is listed as almost ready for inclusion into GEOS-Chem.

This feature request will track the progress of this item's inclusion into GEOS-Chem.

[lfreese]

@yantosca I have the benchmark analysis that @barronh and I completed for the new files if that is needed for this incorporation. I think we were also hoping to find a way to make the methods publicly accessible (with the hopes of it being used again for future inventories), but Barron may want to confirm that given that he did the majority of the work on the notebook to create the files.

[barronh]

I confirm that this is ready, but I urge initial users to look for any issues that we may have missed.

[barronh]

Lyssa Freese and I developed a Jupyter Notebook that can prepare gridded monthly NEI files for use by HEMCO. It can work in the cloud with Google Colab, or on any system that has the necessary libraries. It can be tweaked for different versions.

https://colab.research.google.com/drive/1Z7Btq_upCyCcHWaKk-NFhS9P_Rbiddne?usp=sharing

I'm not sure where all this is documented, so I wanted to tag it here.

[barronh]

Also wanting to note an ftp site for the emissions.

https://gaftp.epa.gov/Air/aqmg/global/NEI2016fh/output/

[msulprizio]

Hi @barronh. I'm working on bringing these emissions into GEOS-Chem 13.2.0 and have a few questions. Do you recommend using the files in EPA_Config.rc or EPAMERGED_Config.rc? Do you have a README for the files in NEI2016fh/output/?

Note: I was able to greatly reduce file size by chunking and compressing the files in NEI2016fh/output/ using:
nccopy -c lon/900,lat/400,lev/1,time/1 -d1 2016fh_xxx.ncf tmp.nc; mv tmp.nc 2016fh_xxx.ncf

[barronh]

I can work on a README. The merged files are useful for faster reading, but have limited utility for sectoral applications. I can address this in a README. I'm glad that the chunking reduced the files so greatly. I though we were already chunking, so I'll have to check that.

…

-Barron

On Tue, Jun 22, 2021 at 12:35 PM Melissa Sulprizio ***@***.***> wrote: Hi @barronh <https://github.com/barronh>. I'm working on bringing these emissions into GEOS-Chem 13.2.0 and have a few questions. Do you recommend using the files in EPA_Config.rc or EPAMERGED_Config.rc? Do you have a README file for the files in NEI2016fh/output/? Note: I was able to greatly reduce file size by chunking and compressing the files in NEI2016fh/output/ using: nccopy -c lon/900,lat/400,lev/1,time/1 -d1 2016fh_xxx.ncf tmp.nc; mv tmp.nc 2016fh_xxx.ncf — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#547 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJCZ3O3U326BZXNIXPCNQLTUC3VTANCNFSM4T5KA5BQ> .

[msulprizio]

Hi @barronh. Can you advise as to whether NEI2016 should be on or off by default in HEMCO_Config.rc? Right now, all regional inventories are off and CEDS is used globally by default. This is because the CEDS data are considered to be more recent and already account for many regional inventories. For GEOS-Chem 13.2.0, we've already updated to the latest CEDS emissions for 1750-2019.

Since our benchmark year is 2019, does it make sense to overwrite 2019 CEDS emissions with NEI2016 emissions for the US? The CEDS reference cites US EPA: Air Pollutant Emissions Trends Data: Average Annual
Emission. In addition, the CEDS release notes for the pre-release version dated August 25, 2019 indicates, "USA emissions now scaled to most recent EPA Trends data."

[barronh]

Great question. I do think these are newer, but they don’t yet have scaling factors for other years. That complicates using them as a default for other years or projects that use other years. I thought part of moving to CEDS was also simplicity and consistency.

[msulprizio]

I thought part of moving to CEDS was also simplicity and consistency.

That's true too.

We can leave NEI2016 off for now and have it as an option (especially for users of high-resolution simulations over North America). When we have scaling factors for to project these emissions to other years, we can run it by the GCSC as to whether the inventory should overwrite CEDS by default.

[msulprizio]

Hi @barronh. Just checking in on the status of a README for these files.

Also, in the sample HEMCO configuration files provided, the VOCs are listed with units kgC/m2/s, but the netCDF files list those units as kg/m2/s. Can you confirm if the units of those emissions is actually kg species/m2/s? As you know, we removed the use of kgC throughout HEMCO and GEOS-Chem in 13.0.0. If units from the files are in kgC/m2/s, then we need to apply the units carbon to species scale factors in HEMCO_Config.rc.

[msulprizio]

@barronh One more question: I see you use mask 1007 for the EPA2016 entries, which corresponds to:

1007 CONUS_MASK  $ROOT/MASKS/v2018-09/CONUS_Mask.01x01.nc MASK  2000/1/1/0 C xy 1 1 -140/20/-50/60

However, there are entries labeled with "can" and "mex" containing emissions over those countries. Should a different mask (e.g. North America) be applied to all EPA2016 entries or maybe just the can/mex entries?

[lfreese]

@msulprizio Barron and I were talking last week about a way to create a DOI for the code so that it is publicly accessible in a more obvious way than the current link that is just noted in this thread here. I was planning to create a zenodo with the code, but I wanted to check if the GC team has preferences for a location for storing this (eg: Pangeo or something other than zenodo)?

[msulprizio]

@lfreese That sounds like a good plan. We use zenodo as well for the GEOS-Chem DOIs, so that would be fine with us.

[barronh]

I've written a README.md and I also want to upload the Jupyter Notebook with it... I am now having trouble accessing my ftp site... So, I am posting the README.md here for now. github issues won't accept a notebook, but I'll post it shortly.

Good question about the mask. I'm trying to look it up, but if I recall that mask number changed between version. Is that right?

[barronh]

Also, good question about the kgC. We were using this with an older version and I think it needed kgC.

[msulprizio]

Thanks for the README @barronh! As for the other points:

Good question about the mask. I'm trying to look it up, but if I recall that mask number changed between version. Is that right?

That's right. I was able to look it up in the now obsolete unit tester repository. We were previously using:

1007 NEI11_MASK  $ROOT/MASKS/v2018-09/USA_LANDMASK_NEI2011_0.1x0.1.20160921.nc LANDMASK 2000/1/1/0 C xy 1 1 -140/20/-50/60

We can revert to that mask if you think that makes most sense for the NEI2016 emissions.

Also, good question about the kgC. We were using this with an older version and I think it needed kgC.

Yes, older HEMCO versions used kgC. If data were in units kg species/m2/s then HEMCO would convert to kgC for VOCs with an emitted MW of 12.0 g/mol in the species database. I just wanted to confirm that the data is in fact in kg species/m2/s in the NEI2016 files, which it seems to be from the units in the netCDF files. In this case units do not need to be converted since HEMCO now carries all emissions in kg species/m2/s.

[barronh]

In terms of the mask, I think either mask can be justified. The EPA emission modeling platform always includes our best estimate of that year's emissions for Canada and Mexico (othpt, othar, ptfire_oth). Using that estimate is likely good. However, that can lead to a discontinuity in Mexico and Canada. I think that's a small issue, but I'm curious what others' think.

The units in the file are kg species, so yes just edit the HEMCO.

[msulprizio]

@barronh I think it may make most sense to split out the Canadian and Mexican emissions provided by EPA as separate option so that users can choose to use them independent of the US emissions. That will also allow us to use the Canada and Mexico masks for those fields. I can work on adding switches for that, and in that case we can keep the CONUS_MASK as is for the US emissions.

[msulprizio]

Daniel Jacob wrote:

We still need yearly scaling factors to apply to NEI16, what should we do for this?

Barron Henderson responded:

Until there is a better option, the EPA Trends Report for Tier 1 CAPS is a good place to start. It is important to remove the wildfires from the trends before calculating scale factors. In the notes for scale factor calculations, it is a good idea to note when the file was retrieved.

[RuijunDang]

Hi all, I am trying to use NEI2016 here and a few questions came up to me.

1. Where does this NEI2016 emission come from? As far as I know, the NEI updates every three years. There are NEI2011, 2014 and 2017. I am not sure which of these the NEI2016 emission inventory is based on.
2. I noticed that the 'c1c2' and 'c3' categories are ship emissions. Perhaps it is better to split it and add it as an option in the SHIP section of HEMCO_Config.rc.

[barronh]

Short answer: There is a lot of documentation available for this version, which is nominally 2016v1.[1]

Long answer:
The naming as NEI2016 can cause confusion. "The National Emissions Inventory (NEI) is a comprehensive and detailed estimate of air emissions of criteria pollutants, criteria precursors, and hazardous air pollutants from air emissions sources."[2] This is just one step in making emissions for photochemical models. To be usable, it must be combined with temporal, vertical, and horizontal allocation methodologies -- and chemical speciation. In short, you have to make a "modeling platform."

The term "NEI" has historically been used synonymously with the EPA's emissions modeling platform. Historically, there has been a focused effort on creating a modeling platform associated with each NEI. This time, the modeling platform was even more collaborative than usual. EPA worked very collaboratively with MJOs and states -- so it's more accurate to call it the collaborative (i.e., not just EPA) emissions modeling platform. As a part of that collaborative effort, the group decided to focus on 2016. This means that some of the data is projected from NEI 2014 and other data is 2016 specific.

Just a note that 2016v2 will be released at some point in the future.

Yes, the c1c2 and c3 categories are "Commercial Marine Vehicle" emissions for different size classes that have different off-coast ranges. What ever you do with CMV, you have to be aware of possible overlap with other ship options.

[1] https://www.epa.gov/air-emissions-modeling/2016v1-platform
[2] http://views.cira.colostate.edu/wiki/wiki/10202

[RuijunDang]

Hi @barronh, thanks for your answer. I learned a lot!

[YanshunLi-washu]

Hi @barronh Will hourly emission data of NEI2016 be available?

[barronh]

There are hourly emissions that have been made publicly available.[1] Another version (V2) will be available soon. The hourly files have point sources in the typical point-source format. These can be vertically allocated similarly to what we do in our Jupyter Notebook. The data volume is, obviously, much larger.

[1] https://views.cira.colostate.edu/iwdw/Modeling/Platforms.aspx

[msulprizio]

I wanted to follow up on the inclusion of emissions over Canada and Mexico in this inventory. Since some of the emissions for Canada and Mexico are included in the standard files, I think it would be easier for users to update the mask file if they want to include those emissions rather than attempt to split them off into separate options in HEMCO_Config.rc (and further clutter that file). I've updated the HEMCO_Config.rc to read:

#==============================================================================
# --- NEI 2016v1 (USA) ---
#
# NOTES:
#  - Barron Henderson wrote, "The EPA emission modeling platform always
#    includes our best estimate of that year's emissions for Canada and Mexico
#    (othpt, othar, ptfire_oth). Using that estimate is likely good. However,
#    that can lead to a discontinuity in Mexico and Canada."
#  - By default only emissions over the CONUS are used (via Mask #1007)
#  - To include emissions over Canada and Mexico, users may revert to the old
#    US mask file $ROOT/MASKS/v2018-09/USA_LANDMASK_NEI2011_0.1x0.1.20160921.nc
#==============================================================================

[msulprizio]

Annual scaling factors for 2002-2020 based on EPA Trends Report for Tier 1 CAPS mentioned above by Barron. The version of the spreadsheet used here was downloaded from the EPA website on 21 Sep 2021. A modified version of that sheet with the scale factor calculations has been made available at http://ftp.as.harvard.edu/gcgrid/data/ExtData/HEMCO/NEI2016/v2021-06/national_tier1_caps+HEMCOscaling.xlsx. See commit 556e42d for details on the application of the annual scaling factors in HEMCO_Config.rc.

[msulprizio]

Hi @barronh @lfreese. I'm trying to run a 1-year HEMCO standalone simulation with NEI2016 emissions to evaluate emissions totals. I ran into an issue with the file 2016fh_16j_othafdust_adj_0pt1degree_month_02.ncf. That file has a time dimension but no time value set, which is causing HEMCO to crash. It also has no other fields aside from the dimension variables and the global attributes are cut off after the history field. Here is the ncdump output for that file:

netcdf \2016fh_16j_othafdust_adj_0pt1degree_month_02 {
dimensions:
        time = UNLIMITED ; // (0 currently)
        lon = 900 ;
        lat = 400 ;
        lev = 1 ;
variables:
        double time(time) ;
                time:standard_name = "time" ;
                time:units = "hours since 1970-1-1 00:00:00" ;
                time:calendar = "proleptic_gregorian" ;
                time:axis = "T" ;
        double lon(lon) ;
                lon:standard_name = "longitude" ;
                lon:long_name = "longitude" ;
                lon:units = "degrees_east" ;
                lon:axis = "X" ;
        double lat(lat) ;
                lat:standard_name = "latitude" ;
                lat:long_name = "latitude" ;
                lat:units = "degrees_north" ;
                lat:axis = "Y" ;
        double lev(lev) ;
                lev:standard_name = "hybrid_sigma_pressure" ;
                lev:formula = "hyam hybm (mlev=hyam+hybm*aps)" ;
                lev:formula_terms = "ap: hyam b: hybm ps: aps" ;
                lev:units = "level" ;
                lev:long_name = "GEOS-Chem levels" ;
                lev:positive = "up" ;
                lev:axis = "Z" ;

// global attributes:
                :CDI = "Climate Data Interface version 1.9.6 (http://mpimet.mpg.de/cdi)" ;
                :Conventions = "CF-1.6" ;
                :history = "Wed Oct 28 09:35:05 2020: cdo -O -O -f nc4 -k grid -z zip_1 -setmisstoc,0 -remapycon,latlon.grid -setgrid,EPA12US1.grid -setattribute,NOX@units=kgNO2/m2/s -setattribute,NO@units=kgNO2/m2/s -setattribute,*@units=kg/m2/s -mulc,907.185 -divc,144000000.0 -divc,2505600 -setreftime,1970-01-01,00:00:00,hours -settaxis,2016-02-01,00:00:00 -delname,TFLAG -chname,TSTEP,time -setzaxis,z1.grid -chname,LAY,lev input/2016fh_16j_othafdust_adj_12US1_month_02.ncf output/2016fh_16j_othafdust_adj_0pt1degree_month_02.ncf" ;
}

I suspect this may also be an issue with the original file available at https://gaftp.epa.gov/Air/aqmg/global/NEI2016fh/output/ because the size of that file is much smaller than the files for other months.

Could please look into that file and reprocess if necessary?

[barronh]

I'll check it out in the morning.

[barronh]

Wow, that was a long morning. This slipped through the cracks, but I have updated it and reposted.

[msulprizio]

Thanks @barron! I will close out this feature request for now since NEI2016 is being included as an option in 13.3.0. I will comment here once I have validated the emission totals with a 1-year HEMCO standalone simulation.

[msulprizio]

Hi @barronh. Could you check the reprocessed file (https://gaftp.epa.gov/Air/aqmg/global/NEI2016fh/output/2016fh_16j_othafdust_adj_0pt1degree_month_02.ncf) again? The timestamp of the file has changed, but the contents have not. It still appears to only have the dimension variables defined, but still no time value and no emissions data.

[barronh]

Hey @msulprizio,

That is super weird. The file on my computational server is right. So, I tested downloading and, of course, you were right.

At first, I thought that the problem was using reput on the sftp server. So, I removed the file and reposted. I downloaded it again and it was still bad...

So, I removed it from the sftp server and, on a whim, used my browser to download again knowing that it wasn't there... the download succeeded. I was just getting a cached file. I had to clear my cached images and files in my browser. Then, when I redownloaded the file was right.

I don't know which step fixed it, but it works for me now.

Best,
Barron

[msulprizio]

Thanks @barronh! I think it was the cache issue after all. I cleared mine, downloaded the file again, and it looks correct now. Sorry for the bother!

[barronh]

No worries. It surprised me that it cached. I wonder if that is a setting on our site that could be updated to prevent similar future caches.

[YanshunLi-washu]

There are hourly emissions that have been made publicly available.[1] Another version (V2) will be available soon. The hourly files have point sources in the typical point-source format. These can be vertically allocated similarly to what we do in our Jupyter Notebook. The data volume is, obviously, much larger.

[1] https://views.cira.colostate.edu/iwdw/Modeling/Platforms.aspx

Thanks, @barronh. Do we have the hourly emissions files ready for GEOS-Chem. Since for NEI2011, we have the option to switch between hourly and monthly. I'm wondering we also have that for NEI2016 in GEOS-Chem. Maybe should also check with @msulprizio . Do we have plan developing NEI2016 hourly emissions into GEOS-Chem. Thanks.

[barronh]

I do have these files 3d gridded, but found performance was slow (too many layers). I also believe that the most common use is monthly. So, I spent most time on the monthly.

Email me at the EPA last.first@epa.gov. We can figure out a good transfer mechanism and testing.