Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Refresh Emissions Views Database Procedures #5793

Open
25 tasks done
jwhitehead77 opened this issue Oct 18, 2023 · 15 comments · Fixed by US-EPA-CAMD/easey-db-scripts#24 · May be fixed by US-EPA-CAMD/easey-db-scripts#10
Open
25 tasks done

Refactor Refresh Emissions Views Database Procedures #5793

jwhitehead77 opened this issue Oct 18, 2023 · 15 comments · Fixed by US-EPA-CAMD/easey-db-scripts#24 · May be fixed by US-EPA-CAMD/easey-db-scripts#10
Assignees
Labels
ECMPS Emissions Phase 1 CAMPD & ECMPS 2.0 tickets

Comments

@jwhitehead77
Copy link
Contributor

jwhitehead77 commented Oct 18, 2023

The emissions view refresh procedures ported from the MS SQL client and used in the workspace do not perform well when used in the official schema due to the vast amount of data. Need to refactor the refresh procedures to be more performant overall.

There are 3 functions that pivot the derived_hrly_value, monitor_hrly_value, & hrly_param_fuel_flow tables so that the values needed for each parameter are a single row for each hour reported.

  • get_derived_hourly_values_pivoted
  • get_monitor_hourly_values_pivoted
  • get_hourly_param_fuel_flow_pivoted

NOTE: The above functions are dynamic functions that return a dynamic list of columns based on the parameter codes passed into the function from the caller. If you add a column to be returned then anything that uses it must be updated to expected that column.

Instead of joining multiple times on derived_hrly_value, monitor_hrly_value, and/or hrly_param_fuel_flow which has 100+ million records based on different parameters. Replace all these joins with one join to the appropriate pivot functions above passing in the parameters that data is needed for.

The following have been refactored to use the pivot approach and can be used as examples...

  • refresh_emission_view_all
  • refresh_emission_view_co2appd
  • refresh_emission_view_co2calc
  • refresh_emission_view_co2cems
  • refresh_emission_view_hiappd

The following had previously been refactored to use an approach that does not always work well for every case and needs to be refactored to use the pivot approach...

  • refresh_emission_view_hicems
  • refresh_emission_view_hiunitstack
  • refresh_emission_view_lme
  • refresh_emission_view_massoilcalc
  • refresh_emission_view_matshcl
  • refresh_emission_view_matshf
  • refresh_emission_view_matshg
  • refresh_emission_view_matsso2
  • refresh_emission_view_moisture
  • refresh_emission_view_noxappemixedfuel
  • refresh_emission_view_noxappesinglefuel

The following are the original ported procedures that have not been refactored at all and needs to be refactored to use the pivot approach...

  • refresh_emission_view_noxmasscems
  • refresh_emission_view_noxratecems
  • refresh_emission_view_so2appd
  • refresh_emission_view_so2cems

The following cannot use the pivot approach but need to be looked at from a performance perspective...

  • refresh_emission_view_matssorbent
  • refresh_emission_view_matsweekly
  • refresh_emission_view_co2dailyfuel
  • refresh_emission_view_dailycal
  • refresh_emission_view_otherdaily

NOTE: Refactor the camdecmps schema (official) procedures and test in the TEST or CDC database which has all of the production data and then copy them to the camdecmpswks schema (workspace) replacing the referenced schema from camdecmps to camdecmpswks

The following procedures are not persisted views and instead uses an actual view for the data so these refresh procedures are just refreshing the counts and should not need refactoring...

  • refresh_emission_view_count
  • refresh_emission_view_dailybackstop
  • refresh_emission_view_ltff
  • refresh_emission_view_nsps4t
@jwhitehead77 jwhitehead77 added the Tech Team CVP Shared Services Tech Team label Oct 18, 2023
@jwhitehead77 jwhitehead77 self-assigned this Oct 18, 2023
@jwhitehead77 jwhitehead77 changed the title Refactor Refresh Emissions Views Procedures Refactor Refresh Emissions Views Database Procedures Oct 18, 2023
@acollad1 acollad1 added the Phase 1 CAMPD & ECMPS 2.0 tickets label Oct 20, 2023
@JanellC
Copy link

JanellC commented Oct 20, 2023

CVP will not work on this ticket. DPC will complete reamaining work

@djw4erg
Copy link

djw4erg commented Oct 24, 2023

Jason, is it possible to load the Emission View data from Workspace into Official when emission reports are submitted instead of reproducing the values? This approach would need to account for the following:

  • Not allowing the failure to load Emission View data to prevent a successful submission.
  • Still providing the ability to refresh the Official Emission View data for Official data.

The second issue would eventually require improving the efficiency of the Official process, but the need might not be as urgent.

@JanellC JanellC added the DPC Tech Shared Services/ Technical Tasks DPC will work on label Oct 30, 2023
@alangmaid
Copy link
Collaborator

@hamezene hamezene added the ECMPS label Nov 3, 2023
@JanellC JanellC removed the Tech Team CVP Shared Services Tech Team label Nov 7, 2023
@ergjustin
Copy link

ergjustin commented Nov 20, 2023

  • refresh_emission_view_hiunitstack
  • refresh_emission_view_lme
  • refresh_emission_view_noxappemixedfuel
  • refresh_emission_view_noxappesinglefuel

  • refresh_emission_view_noxmasscems
  • refresh_emission_view_so2appd
  • refresh_emission_view_so2cems
  • refresh_emission_view_noxratecems

@ergjustin
Copy link

@alangmaid alangmaid assigned renaemyers and unassigned ergjustin Dec 1, 2023
@shahbazkhan1999
Copy link

need to discuss with @ergjustin

@djw4erg
Copy link

djw4erg commented Mar 12, 2024

Related Tickets:

@alangmaid alangmaid added the blocked This issue is blocked label Mar 19, 2024
@renaemyers
Copy link

Address monitor_hrly_value slowness: #6174

@djw4erg
Copy link

djw4erg commented Apr 23, 2024

Example Potential Alternate Solution (View Emissions SO2 CEMS)

As far as I can tell the pivot function method would not use indexes to join the DHV and MHV data. This might not cause an execution hit, but worries me. The following method achieves a similar result as the pivot function method, but should join using indexes.

View Emissions SO2 CEMS Query

select  cor.Oris_Code,
        cor.Facility_Name,
        cor.Locations,
        /* Real Columns */
        cor.Mon_Plan_Id,
        cor.Mon_Loc_Id,
        cor.Rpt_Period_Id,
        cor.Begin_Date as Date,
        cor.Begin_Hour as Hour,
        cor.Op_Time,
        cor.Hr_Load as Unit_Load,
        cor.Load_Uom_Cd as Load_Uom,     
        cor.So2c_Mhv_Unadjusted_Hrly_Value as Unadj_So2,
        cor.So2c_Mhv_Applicable_Bias_Adj_Factor as So2_Baf, 
        cor.So2c_Mhv_Adjusted_Hrly_Value as Rpt_Adj_So2,
        cor.So2c_Mhv_Calc_Adjusted_Hrly_Value as Calc_Adj_So2, 
        cor.So2c_Mhv_Modc_Cd as So2_Modc,
        cor.So2c_Mhv_Pct_Available as So2_Pma,   
        cor.Flow_Mhv_Unadjusted_Hrly_Value as Unadj_Flow,
        cor.Flow_Mhv_Applicable_Bias_Adj_Factor as Flow_Baf,
        cor.Flow_Mhv_Adjusted_Hrly_Value as Rpt_Adj_Flow,
        cor.Flow_Mhv_Calc_Adjusted_Hrly_Value as Adj_Flow_Used,
        cor.Flow_Mhv_Modc_Cd as Flow_Modc,
        cor.Flow_Mhv_Pct_Available as Flow_Pma,
        cor.So2_Dhv_Calc_Pct_Moisture as Pct_Wrong_Used,
        case 
            when cor.So2_Dhv_Calc_Pct_Moisture is null then null
            when cor.h2o_dhv_Modc_Cd is not null then cor.h2o_dhv_Modc_Cd
            when cor.h2o_mhv_Modc_Cd is not null then cor.h2o_mhv_Modc_Cd
            else 'DF'
        end as Source_H2o_Value,
        frm.Equation_Cd as So2_Formula_Code,
        cor.So2_Dhv_Adjusted_Hrly_Value as Rpt_So2_Mass_Rate,
        cor.So2_Dhv_Calc_Adjusted_Hrly_Value as Calc_So2_Mass_Rate,
        cor.Hi_Dhv_Calc_Adjusted_Hrly_Value,
        cor.H2o_Dhv_Modc_Cd,
        cor.So2r_Dhv_Adjusted_Hrly_Value,
        cor.H2o_Mhv_Modc_Cd,
        case (frm.Equation_Cd)
            when 'F-23' then cor.Hi_Dhv_Calc_Adjusted_Hrly_Value
        end as Calc_Hi_Rate,
        case (frm.Equation_Cd)
            when 'F-23' then 
                coalesce
                (
                    cor.So2r_Dhv_Adjusted_Hrly_Value,
                    (
                        select  max( Default_Value )
                          from  camdecmps.MONITOR_DEFAULT def
                         where  def.Mon_Loc_Id = cor.Mon_Loc_Id
                           and  def.Parameter_Cd = 'SO2R'
                           and  def.Default_Purpose_Cd = 'F23'
                           and  ( def.Begin_Date  + interval '1' HOUR * def.Begin_Hour ) <= ( cor.Begin_Date  + interval '1' HOUR * cor.Begin_Hour )
                           and  ( ( def.End_Date is null) or ( def.End_Date  + interval '1' HOUR * def.End_Hour ) <= ( cor.Begin_Date  + interval '1' HOUR * cor.Begin_Hour ) )
                    )
                )
        end as Default_So2_Emission_Rate
  from  (
            select  sel.Oris_Code,
                    sel.Facility_Name,
                    sel.Locations,
                    sel.Mon_Plan_Id,
                    sel.Rpt_Period_Id,
                    sel.Mon_Loc_Id,
                    sel.Begin_Date,
                    sel.Begin_Hour,
                    sel.Op_Time,
                    sel.Hr_Load,
                    sel.Load_Uom_Cd,
                    max( case when dhv.Parameter_Cd = 'SO2'  then dhv.Hour_Id end ) as So2_Dhv_Hour_Id,
                    max( case when dhv.Parameter_Cd = 'SO2'  then dhv.Calc_Pct_Moisture end ) as So2_Dhv_Calc_Pct_Moisture,
                    max( case when dhv.Parameter_Cd = 'SO2'  then dhv.Adjusted_Hrly_Value end ) as So2_Dhv_Adjusted_Hrly_Value,
                    max( case when dhv.Parameter_Cd = 'SO2'  then dhv.Calc_Adjusted_Hrly_Value end ) as So2_Dhv_Calc_Adjusted_Hrly_Value,
                    max( case when dhv.Parameter_Cd = 'SO2'  then dhv.Mon_Form_Id end ) as So2_Dhv_Mon_Form_Id,
                    max( case when dhv.Parameter_Cd = 'HI'   then dhv.Calc_Adjusted_Hrly_Value end ) as Hi_Dhv_Calc_Adjusted_Hrly_Value,
                    max( case when dhv.Parameter_Cd = 'H2O'  then dhv.Modc_Cd end ) as H2o_Dhv_Modc_Cd,
                    max( case when dhv.Parameter_Cd = 'H2O'  then dhv.Pct_Available end ) as H2o_Dhv_Pct_Available,
                    max( case when dhv.Parameter_Cd = 'SO2R' then dhv.Adjusted_Hrly_Value end ) as So2r_Dhv_Adjusted_Hrly_Value,   
                    max( case when mhv.Parameter_Cd = 'FLOW' then mhv.Hour_Id end ) as Flow_Mhv_Hour_Id,     
                    max( case when mhv.Parameter_Cd = 'FLOW' then mhv.Unadjusted_Hrly_Value end ) as Flow_Mhv_Unadjusted_Hrly_Value,
                    max( case when mhv.Parameter_Cd = 'FLOW' then mhv.Applicable_Bias_Adj_Factor end ) as Flow_Mhv_Applicable_Bias_Adj_Factor,
                    max( case when mhv.Parameter_Cd = 'FLOW' then mhv.Adjusted_Hrly_Value end ) as Flow_Mhv_Adjusted_Hrly_Value,
                    max( case when mhv.Parameter_Cd = 'FLOW' then mhv.Calc_Adjusted_Hrly_Value end ) as Flow_Mhv_Calc_Adjusted_Hrly_Value,
                    max( case when mhv.Parameter_Cd = 'FLOW' then mhv.Modc_Cd end ) as Flow_Mhv_Modc_Cd,
                    max( case when mhv.Parameter_Cd = 'FLOW' then mhv.Pct_Available end ) as Flow_Mhv_Pct_Available,
                    max( case when mhv.Parameter_Cd = 'SO2C' then mhv.Unadjusted_Hrly_Value end ) as So2c_Mhv_Unadjusted_Hrly_Value,
                    max( case when mhv.Parameter_Cd = 'SO2C' then mhv.Applicable_Bias_Adj_Factor end ) as So2c_Mhv_Applicable_Bias_Adj_Factor,
                    max( case when mhv.Parameter_Cd = 'SO2C' then mhv.Adjusted_Hrly_Value end ) as So2c_Mhv_Adjusted_Hrly_Value,
                    max( case when mhv.Parameter_Cd = 'SO2C' then mhv.Calc_Adjusted_Hrly_Value end ) as So2c_Mhv_Calc_Adjusted_Hrly_Value,
                    max( case when mhv.Parameter_Cd = 'SO2C' then mhv.Modc_Cd end ) as So2c_Mhv_Modc_Cd,
                    max( case when mhv.Parameter_Cd = 'SO2C' then mhv.Pct_Available end ) as So2c_Mhv_Pct_Available, 
                    max( case when mhv.Parameter_Cd = 'H2O'  then mhv.Modc_Cd end ) as H2o_Mhv_Modc_Cd
              from  (
                        select  fac.Oris_Code,
                                fac.Facility_Name,
                                (
                                    select  string_agg( coalesce( unt.Unitid, stp.Stack_Name ), ', ' order by case when unt.Unit_Id is null then 0 else 1 end, coalesce( unt.Unitid, stp.Stack_Name ) )
                                      from  camdecmps.MONITOR_PLAN_LOCATION mpl
                                            join camdecmps.MONITOR_LOCATION loc on loc.Mon_Loc_Id = mpl.Mon_Loc_Id
                                            left join camd.UNIT unt on unt.Unit_Id = loc.Unit_Id
                                            left join camdecmps.STACK_PIPE stp on stp.Stack_Pipe_Id = loc.Stack_Pipe_Id
                                     where  mpl.Mon_Plan_Id = pln.Mon_Plan_Id
                                ) as Locations,
                                hod.Begin_Date,
                                hod.Begin_Hour,
                                hod.Op_Time,
                                hod.Hr_Load,
                                hod.Load_Uom_Cd,
                                ems.Mon_Plan_Id,
                                ems.Rpt_Period_Id,
                                mpl.Mon_Loc_Id,
                                hod.Hour_Id
                          from  camdecmps.EMISSION_EVALUATION ems
                                join camdecmps.MONITOR_PLAN pln on pln.Mon_Plan_Id = ems.Mon_Plan_Id
                                join camd.PLANT fac on fac.Fac_Id = pln.Fac_Id
                                join camdecmps.MONITOR_PLAN_LOCATION mpl on mpl.Mon_Plan_Id = ems.Mon_Plan_Id
                                join camdecmps.HRLY_OP_DATA hod on hod.Mon_Loc_Id = mpl.Mon_Loc_Id and hod.Rpt_Period_Id = ems.Rpt_Period_Id
                         where  ems.Mon_Plan_Id = 'TWCORNEL5-488E42008B434177BC7D7BFF138D18EF'
                           and  ems.Rpt_Period_Id = 109
                           and  pln.End_Rpt_Period_Id is null
                    ) sel
                    join camdecmps.DERIVED_HRLY_VALUE dhv on dhv.Hour_Id = sel.Hour_Id and dhv.Parameter_Cd in ( 'SO2', 'HI', 'H2O', 'SO2R' )
                    join camdecmps.MONITOR_HRLY_VALUE mhv on mhv.Hour_Id = sel.Hour_Id and mhv.Parameter_Cd in ( 'FLOW', 'SO2C', 'H20' )
             group 
                by  sel.Oris_Code,
                    sel.Facility_Name,
                    sel.Locations,
                    sel.Mon_Plan_Id,
                    sel.Rpt_Period_Id,
                    sel.Mon_Loc_Id,
                    sel.Begin_Date,
                    sel.Begin_Hour,
                    sel.Op_Time,
                    sel.Hr_Load,
                    sel.Load_Uom_Cd
        ) cor
        left join camdecmps.MONITOR_FORMULA frm on frm.Mon_Form_Id = cor.So2_Dhv_Mon_Form_Id
 where  cor.So2_Dhv_Hour_Id is not null 
   and  cor.Flow_Mhv_Hour_Id is not null

@renaemyers
Copy link

All involved emissions tables have been analyzed on dev and the refresh procedures are running much faster. Code has been checked for all updated procedures.

@esaber76
Copy link

@djw4erg to provide scripts which can be used to first clear data from affected views prior to testing.

@esaber76
Copy link

esaber76 commented Jun 7, 2024

Testing Results

  1. For stack/pipe locations, the Hourly Heat Input CEMS and Hourly CO2 CEMS Views have counts which appear to be the sum for all locations in the monitoring plan even if HI and CO2 is not measured at all locations. Ex. ORIS 1356, CS023, 2, 3. The counts equal 6624 and should be 2208. HI and CO2 is only measured at CS023. These views are also showing data for units 2 and 3 and should not. This also affects other locations that do not have HI or CO2 CEMS (ex. LME, Appendix D with only NOXR CEMS have these views available). Affects both the public and workspace views.
  2. MATS HG View
    • HG Formula Code column is always null (both public and workspace views; ex. ORIS 1356, CS023).
    • Rptd. % Diluent, Diluent Parameter, Diluent MODC, and F-Factor columns null (both public and workspace views; ex. ORIS 564, 1).
  3. Hourly NOx Mass CEMS View does not display any data (both public and workspace views; ex. ORIS 880041, X015).
  4. %Diluent Used in Calc. and Calc. HCl Rate columns missing data in MATS HCL View in the workspace (Calc. HCl Rate column in particular should be populated before and after an evaluation; ex. ORIS 4125, 9).
  5. I've been seeing random instances where a lot of the other views are missing column data as well which may explain some of the comments above. I looked at this a bit closer and I see "CALL camdecmpswks.refresh_emissions_views..." getting stuck. In this example, it's been running for 15+ minutes. I saw others in there stuck for days but killed the processes. Does this procedure get called after an evaluation? That's when I've noticed it getting stuck and a lot of calculated columns missing data.

image

@esaber76 esaber76 assigned renaemyers and unassigned esaber76 Jun 7, 2024
@annalbrecht annalbrecht assigned djw4erg and unassigned renaemyers Jun 7, 2024
@djw4erg
Copy link

djw4erg commented Jun 28, 2024

Handling Testing Results

  1. Hourly Heat Input CEMS and Hourly CO2 CEMS Views
    1. REFRESH_EMISSION_VIEW_CO2CEMS did not ensure that a HI DHV and a FLOW MHV record exists for each location/hour included in the results.
      1. Corrected by adding checks that dhv.HI_HOUR_ID and mhv.FLOW_HOUR_ID are not null.
    2. REFRESH_EMISSION_VIEW_HICEMS
      1. Corrected by making joins to the DHV and MHV pivot function results are inner joins, and by adding checks that dhv.HI_HOUR_ID and mhv.FLOW_HOUR_ID are not null.
  2. MATS HG View
    1. REFRESH_EMISSION_VIEW_MATSHG (CAMDECMPS) linked the MONITOR_FORMULA record using the H2O DHV MON_FORM_ID instead of the PMRE/PMRH MATS DHV MON_FORM_ID.
      • Columns directly affected by incorrect join
        • HG_FORMULA_CD
      • Columns conditionally affected by incorrect join because of MONITOR_FORMULA.EQUATION_CD value.
        • DILUENT_MODC
        • DILUENT_PARAMETER
        • F_FACTOR
        • RPT_PCT_DILUENT
    2. Corrected the MONITOR_FORMULA join in REFRESH_EMISSION_VIEW_MATSHG (CAMDECMPS).
  3. Hourly NOx Mass CEMS View
    1. Read Handling Testing Results Suggested Hourly NOxM CEMS Improved Query
    2. Additionally, the RPT_NOX_MASS and CALC_NOX_MASS are incorrectly pulled from the NOxR DHV row instead of the
    3. Performance Notes:
      1. The use of temp table TEMP_HOURLY_TEST_ERRORS seems to cause a performance issue. The model "testing" script used a subquery instead of the temp table with the model script (without the error codes column) executing in 7 seconds, while the version with the temp table executing in 100 seconds. The temp table was mainly used to get the error information for each hour. To avoid the use of the temp table the query in Handling Testing Results Suggested Hourly NOxM CEMS Improved Query implements a subquery in the select clause to get the error information, and replaced the use of the temp table with a subquery as the model query did. This change maintained the performance improvement from the model query.
      2. Additionally, the original pivot method to improve performance may not have been needed. It appears that the joins on HOUR_ID that used the NOx DHV row, should have used the HOUR_ID from the TEMP_HOURLY_TEST_ERRORS table.
    4. Changed the REFRESH_EMISSION_VIEW_NOXMASSCEMS (CAMDECMPS) to use the suggested SQL in Handling Testing Results Suggested Hourly NOxM CEMS Improved Query.
      NOx DHV row.
  4. MATS HCL View
    1. Although the Refactor Refresh Emissions Views Database Procedures #5793 (comment) comment indicates that the workspace emission view should contain populated HCl Rate and %Diluent Used values both before and after the emissions are evaluated, calculated values will not exist before evaluation.
      • Presumably this assumption is based on emissions imported from Historical Data, which does not copy from Official (Historical) tables into Workspace tables, but exports to JSON and then imports from that JSON. Because the JSON does not have and the import does not handle calculated values, the initial workspace display would not contain calculated values until a evaluation occurs.
    2. Calculated values are missing after an evaluation because the emission views, including MATS HCl, are not repopulated/updated after an emission evaluation.
    3. Ticket Require Re-evaluations of QA and EM data issues #6067 includes refreshing the Emission Views after an evaluation.
    4. Will not correct the fact that calculated values do not exist before an evaluation.
  5. REFRESH_EMISSIONS_VIEWS Execution Stuck and Missing Column Data

@djw4erg
Copy link

djw4erg commented Jun 30, 2024

Handling Testing Results Suggested Hourly NOxM CEMS Improved Query

  • The existing query ran for over 2 minutes in DEV, which is not really acceptable.
  • This attempt at the suggested query ran for 4 seconds although the first attempt at the suggested query ran for 1 minute and 40 seconds.
  • The difference between the first and second attempts at the suggested query is that the hod.HOUR_ID is used for any joins on HOUR_ID in the second atempt instead of the dhv.HOUR_ID.
  • The first (failed) attempt at the suggested query matches the original version of the query, which also performed poorly.
  • The MONITOR_DEFAULT join is not needed since no data is utilized from MONITOR_DEFAULT .

Correcting the Original Problem

  • The original problem was caused by the inner join on MONITOR_FORULA. The join should have occurred from the NOx DHV row but was made from the NOxR DHV row. That resulted in now rows displaying.
  • Additionally, both RPT_NOX_MASS and CALC_NOX_MASS are incorrectly pulled from the NOxR DHV row instead of the NOx DHV row.

Suggested Replacement Query for the Hourly NOxM CEMS View Refresh

select  distinct
        hod.mon_plan_id,
        hod.mon_loc_id,
        hod.rpt_period_id,
        camdecmps.format_date_hour( hod.begin_date, hod.begin_hour, null ) AS date_hour,
        hod.op_time,
        hod.hr_load as unit_load,
        hod.load_uom_cd as load_uom,
        noxc.Unadjusted_Hrly_Value as unadjj_nox,
        noxc.Applicable_bias_adj_factor as calc_nox_baf,
        noxc.adjusted_hrly_value as rpt_adj_nox,
        noxc.calc_adjusted_hrly_value as rpt_adj_nox,
        noxc.modc_cd as nox_modc,
        noxc.pct_available as nox_p0ma,
        flow.unadjusted_hrly_value as unadj_flow,
        flow.applicable_bias_adj_factor as calc_flow_baf,
        flow.adjusted_hrly_value as rpt_adj_flow,
        flow.calc_adjusted_hrly_value as adj_flow_used,
        flow.modc_cd as flow_modc,
        flow.pct_available as flow_pma,
        nox.calc_pct_moisture as pct_h2o_used,
        case 
            when ( noxr.calc_pct_moisture is not null ) then 
                case 
                    when ( h2o_m.modc_cd is not null ) then h2o_m.modc_cd 
                    when ( h2o_d.modc_cd is not null ) then h2o_d.modc_cd 
                    else 'DF'
                end 
            else null
        end as source_h2o_value,
        frm.equation_cd as noxmass_formula_code,
        nox.adjusted_hrly_value as rpt_nox_mass,
        nox.calc_adjusted_hrly_value as calc_nox_Mass,
        (
            select  case when max( coalesce( sev.SEVERITY_LEVEL, 0 ) ) > 0 then 'Y' else NULL end
              from  camdecmps.EMISSION_EVALUATION ems
                    join camdecmpsaux.CHECK_LOG chl
                      on chl.chk_session_id = ems.chk_session_id
                     and chl.mon_loc_id = hod.mon_loc_id
                     and ( chl.op_begin_date < hod.begin_date or ( chl.op_begin_date = hod.begin_date and chl.op_begin_hour <= hod.begin_hour ) )
                     and ( chl.op_end_date > hod.begin_date or ( chl.op_end_date = hod.begin_date and chl.op_end_hour >= hod.begin_hour ) )
                    left join camdecmpsmd.SEVERITY_CODE sev
                      on sev.severity_cd = chl.severity_cd
        ) as error_codes
  from  (
            select  hod.hour_id,
                    mpl.MON_PLAN_ID, 
                    hod.MON_LOC_ID, 
                    hod.RPT_PERIOD_ID,
                    hod.BEGIN_DATE,
                    hod.BEGIN_HOUR,
                    hod.OP_TIME,
                    hod.HR_LOAD,
                    hod.LOAD_UOM_CD
              from  camdecmps.MONITOR_PLAN_LOCATION mpl
                    join camdecmps.HRLY_OP_DATA hod on hod.mon_loc_id = mpl.mon_loc_id
             where  mpl.mon_plan_id = vmonplanid
               and  hod.rpt_period_id = vRptPeriodId
         ) as hod 
         join camdecmps.DERIVED_HRLY_VALUE nox
           on nox.hour_id = hod.hour_id
          and nox.parameter_cd = 'NOX'
         join camdecmps.MONITOR_HRLY_VALUE flow
           on flow.hour_id = hod.hour_id
          and flow.parameter_cd = 'FLOW'
         join camdecmps.MONITOR_FORMULA frm
           on frm.mon_form_id = nox.mon_form_id
          and frm.equation_cd like 'F-26%'
         left join camdecmps.MONITOR_HRLY_VALUE noxc
           on noxc.hour_id = hod.hour_id
          and noxc.parameter_cd = 'NOXC'
         left join camdecmps.MONITOR_HRLY_VALUE h2o_m
           on h2o_m.hour_id = hod.hour_id
          and h2o_m.parameter_cd = 'H2O'
         left join camdecmps.DERIVED_HRLY_VALUE h2o_d
           on h2o_d.hour_id = hod.hour_id
          and h2o_d.parameter_cd = 'H2O'
         left join camdecmps.DERIVED_HRLY_VALUE noxr
           on noxr.hour_id = hod.hour_id
          and noxr.parameter_cd = 'NOXR'

@annalbrecht
Copy link
Contributor

annalbrecht commented Jul 17, 2024

Related to #6324.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ECMPS Emissions Phase 1 CAMPD & ECMPS 2.0 tickets
Projects
None yet