Harmonising KS2 reading scores across assessment eras

[aluayeskendir]

Question / Problem statement

I am using ECHILD to study ethnic disparities in first episode psychosis incidence among children and young people in England. As part of a causal mediation analysis, I plan to use KS2 reading attainment as a proxy for English language proficiency, alongside the English as a first language variable from pupil census.

My study includes eight birth cohorts (September 1999 - August 2007), meaning children sat KS2 across three assessment eras:

• Up to 2011/12: Reading test reported as NC levels (READLEV) and raw marks (READMRK 0–50). READFINE is not available until 2012/13.
• 2012/13 to 2014/15: Reading test unchanged but writing moved to TA-only; GPS test introduced from 2012/13; still NC levels. READFINE (2.5–6.5) now available alongside READMRK.
• 2015/16 onwards: New curriculum, reading reported as scaled scores (READSCORE 80-120) and raw marks (READMRK 0-50).

Reading is the only English component with an externally marked test across all eras, which is why I've chosen it. The derived metrics change across eras (levels → fine grades → scaled scores), but READMRK (raw reading marks) is available from 1995/96 onwards.

I am currently considering two complementary strategies:

Strategy A: Harmonise and pool all cohorts using within-year percentile ranks

Since the reading test changed across eras (different total marks possible, different curriculum content), raw marks are not directly comparable across years. Instead, I would rank every child against all other children who sat the same test in the same academic year and convert that rank to a percentile (0–100), using READMRK throughout. For example, a child at the 25th percentile in 2010 and a child at the 25th percentile in 2017 both read less well than 75% of their peers that year. This produces a single, scale-free continuous variable.

The key assumption is that a child's relative position in their year's reading distribution reflects a comparable level of functional English proficiency across eras, despite curriculum changes.

As sensitivity analyses:

• Using era-specific best measures: READFINE for 2012/13–2014/15 and READSCORE for 2015/16+, percentile-ranked within year, as a check on whether the choice of base variable matters

Strategy B: Separate era-specific analyses

Run the mediation model separately for pre-reform cohorts (using READFINE) and post-reform cohorts (using READSCORE), avoiding harmonisation entirely.

The post-reform analysis (birth cohorts 2004–2007, READSCORE) would be the primary analysis given the better-calibrated continuous measure and overlap with my outcome datasets.

The trade-off is reduced statistical power per analysis, particularly for ethnicity-stratified estimates.

I am leaning towards Strategy A as primary with Strategy B as a key sensitivity analysis.

Question

Has anyone harmonised KS2 attainment across the pre-2016 (levels) and post-2016 (scaled scores) eras and is within-year percentile ranking of READMRK a reasonable approach, and are there pitfalls I should be aware of (e.g. changes in total marks available, test duration, or mark distributions across years)?

Compliance checklist

• I confirm I have not entered any information from within the ONS SRS (or only information that has been cleared for release by the ONS SRS output checking team).
• I confirm I have asked only one question in the textbox above.
• I confirm I have checked for similar questions amongst existing Q&As.

[klharron]

Hi Alua apologies for the lack of response on this. I'll check with the team and get back to you asap.

[jwltan]

Hi Alua, we standardised raw reading marks within each academic year - i.e. subtract the mean and divide by the standard deviation for all who took the test that year. This creates a z-score with mean=0 and sd=1. It's similar to what you're proposing for Strategy A. We did not use readfine or readscore.

[aluayeskendir]

Thank you very much!