Use NI-specific age profiles for demographic targets

changelog.d/fixed/64.md

@@ -0,0 +1 @@
+Use country-specific age profiles for Northern Ireland instead of UK-wide mean in local area demographic targets.

policyengine_uk_data/datasets/local_areas/constituencies/targets/README.md

@@ -1,5 +1,8 @@
 # Data
 
-* Age data is from the [House of Commons Library](https://commonslibrary.parliament.uk/constituency-statistics-population-by-age/), with single-year age counts for each parliamentary constituency (2010). The data is from 2020. The raw download should be saved as `raw_age.csv`; running `fill_missing_age_demographics.py` fills in missing Scotland and NI constituencies using mean age profiles and writes the processed output to `age.csv`.
+* Age data is from the [House of Commons Library](https://commonslibrary.parliament.uk/constituency-statistics-population-by-age/), with single-year age counts for each parliamentary constituency. The data is from 2020. The raw download should be saved as `raw_age.csv`; running `fill_missing_age_demographics.py` fills in missing constituencies and writes the processed output to `age.csv`.
+  * The raw data covers **England, Wales and Scotland** constituencies. Northern Ireland constituencies are missing from this source.
+  * Missing constituencies are filled using **country-specific mean age profiles** (not UK-wide mean). For NI, the age distribution is derived from [ONS subnational population projections](https://www.ons.gov.uk/peoplepopulationandcommunity/populationandmigration/populationprojections/datasets/tablea21principalprojectionukpopulationinagegroups) via `demographics.csv`.
+  * NI constituency-level age data is also available from [NISRA mid-year estimates (small areas)](https://www.nisra.gov.uk/publications/2024-mid-year-population-estimates-small-geographical-areas-within-northern-ireland) if per-constituency precision is needed in future.
 * Employment incomes are from Nomis ASHE workplace analysis, covering all workers (part-time and full-time), from 2023.
 * HMRC total income is from 2021.

policyengine_uk_data/datasets/local_areas/constituencies/targets/fill_missing_age_demographics.py

@@ -9,32 +9,50 @@
 SCOTLAND_CONSTITUENCY = "S14"
 WALES_CONSTITUENCY = "W07"
 
+COUNTRY_PREFIXES = {
+    "E": ENGLAND_CONSTITUENCY,
+    "N": NI_CONSTITUENCY,
+    "S": SCOTLAND_CONSTITUENCY,
+    "W": WALES_CONSTITUENCY,
+}
+
 incomes = incomes[
     np.any(
         [
             incomes["code"].str.contains(country_code)
-            for country_code in [
-                ENGLAND_CONSTITUENCY,
-                NI_CONSTITUENCY,
-                SCOTLAND_CONSTITUENCY,
-                WALES_CONSTITUENCY,
-            ]
+            for country_code in COUNTRY_PREFIXES.values()
         ],
         axis=0,
     )
 ]
 
-full_constituencies = incomes.code
-missing_constituencies = pd.Series(list(set(incomes.code) - set(ages.code)))
+missing_codes = set(incomes.code) - set(ages.code)
 missing_constituencies = pd.DataFrame(
     {
-        "code": missing_constituencies.values,
-        "name": incomes.set_index("code").loc[missing_constituencies].name.values,
+        "code": list(missing_codes),
+        "name": incomes.set_index("code").loc[list(missing_codes)].name.values,
     }
 )
-for col in ages.columns[2:]:
-    # We only have England and Wales demographics- fill in the remaining with the average age profiles among the rest of the UK.
-    missing_constituencies[col] = ages[col].mean()
+
+age_cols = ages.columns[2:]
+
+# Use country-specific mean profiles instead of UK-wide mean (#64).
+# For each missing constituency, find areas in the same country and
+# use their mean age profile. Falls back to UK-wide mean if no areas
+# exist for that country.
+for _, row in missing_constituencies.iterrows():
+    country_letter = row["code"][0]
+    same_country = ages[ages["code"].str.startswith(country_letter)]
+    if len(same_country) > 0:
+        for col in age_cols:
+            missing_constituencies.loc[
+                missing_constituencies["code"] == row["code"], col
+            ] = same_country[col].mean()
+    else:
+        for col in age_cols:
+            missing_constituencies.loc[
+                missing_constituencies["code"] == row["code"], col
+            ] = ages[col].mean()
 
 ages = pd.concat([ages, missing_constituencies])
 ages.to_csv("age.csv", index=False)

policyengine_uk_data/datasets/local_areas/local_authorities/targets/README.md

@@ -1,5 +1,8 @@
 # Data
 
 * Age data is from the [ONS mid-year population estimates](https://www.nomisweb.co.uk/datasets/pestsyoala), with single-year age counts for each local authority district. The raw download should be saved as `raw_age.csv`; running `fill_missing_age_demographics.py` fills missing values and intersects with the income file, writing the processed output to `age.csv`.
+  * The Nomis dataset covers all UK local authorities including the 11 NI local government districts. However, some NI LGDs may have missing age breakdowns in the raw download.
+  * Missing values are filled using **country-specific means** (not UK-wide mean), so NI areas get NI-shaped age profiles. The NI national age distribution comes from [ONS subnational population projections](https://www.ons.gov.uk/peoplepopulationandcommunity/populationandmigration/populationprojections/datasets/tablea21principalprojectionukpopulationinagegroups) via `demographics.csv`.
+  * NI LGD-level data is also available directly from [NISRA mid-year estimates](https://www.nisra.gov.uk/publications/2024-mid-year-population-estimates-northern-ireland-and-estimates-population-aged-85).
 * Employment incomes are from Nomis ASHE workplace analysis, covering all workers (part-time and full-time), from 2023.
 * HMRC total income is from 2021.