Skip to content

Methodology and Calculations

Jump to bottom
Calvin Yang edited this page Jun 1, 2026 · 1 revision

Methodology and Calculations

Implements the AIAG Average and Range Method (MSA 4th Ed., 2010). Study variation expressed as 5.15 sigma — covering 99% of a normal distribution.

Variance components

Symbol Name Description
EV Equipment Variation (Repeatability) Within-operator trial-to-trial gage noise
AV Appraiser Variation (Reproducibility) Between-operator systematic offset
GR&R Combined sqrt(EV^2 + AV^2)
PV Part Variation True part-to-part variation
TV Total Variation sqrt(GR&R^2 + PV^2)
ndc Distinct Categories floor(1.41 * PV / GR&R)

Step-by-step calculation

Step 1 — Per-operator stats: For each operator and part: Range = max(trials) - min(trials), Mean = mean(trials). R-bar_o = mean of part ranges; X-bar_o = mean of part means.

Step 2 — R-bar-bar:

R-bar-bar = mean of all operator R-bars
UCL_R     = D4 * R-bar-bar

Ranges above UCL_R are flagged out-of-control.

Step 3 — X-diff:

X-diff = max(operator X-bars) - min(operator X-bars)

Step 4 — Part range:

Rp = max(part averages) - min(part averages)

Step 5 — Variance components (5.15-sigma):

EV  = R-bar-bar * K1
AV  = sqrt( max(0, (X-diff * K2)^2 - EV^2 / (n_parts * n_trials)) )
GRR = sqrt(EV^2 + AV^2)
PV  = Rp * K3
TV  = sqrt(GRR^2 + PV^2)

If AV^2 goes negative, AV is clamped to 0. This means operator variation is indistinguishable from gage noise — a good outcome, not an error.

Step 6 — Percentages and acceptance:

%EV = (EV / TV) * 100       %AV  = (AV  / TV) * 100
%GRR = (GRR / TV) * 100     %PV  = (PV  / TV) * 100
ndc = floor(1.41 * PV / GRR)
%GRR Status
<= 10% ACCEPTABLE
10-30% MARGINAL
> 30% UNACCEPTABLE

Both conditions must pass independently: %GRR within threshold and ndc >= 5.

K-constants (d2* factors)

K1 — Equipment Variation (by trials)

Trials K1
2 4.56
3 3.05
4 2.50
5 2.21

K2 — Appraiser Variation (by operators)

Operators K2
2 3.65
3 2.70
4 2.30
5 2.08

K3 — Part Variation (by parts)

Parts K3 Parts K3
2 3.65 7 1.82
3 2.70 8 1.74
4 2.30 9 1.67
5 2.08 10 1.62
6 1.93

D4 — UCL multiplier (by trials)

Trials D4
2 3.267
3 2.574
4 2.282
5 2.114

Tolerance metrics

When --tolerance or --usl/--lsl is provided:

%Tolerance(EV)  = (EV  / tolerance) * 100
%Tolerance(AV)  = (AV  / tolerance) * 100
%Tolerance(GRR) = (GRR / tolerance) * 100
ndc_tol         = 1.41 * (tolerance / GRR)

Confidence intervals

95% and 90% CIs on EV, AV, GRR via chi-squared distribution (AIAG Appendix C):

sigma_EV = EV / 5.15
CI_lower = sigma_EV * sqrt(df / chi2_upper) * 5.15
CI_upper = sigma_EV * sqrt(df / chi2_lower) * 5.15
df_EV    = n_operators * n_parts * (n_trials - 1)

scipy provides exact quantiles; a hardcoded table (df 1–30) serves as fallback.

Nested GR&R (destructive testing)

Use --study-type nested when parts cannot be re-measured. Each operator measures a unique set of parts; variance is decomposed via nested ANOVA.

Linearity study

--study-type linearity fits: bias = a + b * reference

  • Non-zero slope (p < 0.05) → gage error changes across measurement range
  • Linearity % = |b| * process_range / tolerance * 100
  • Acceptable: slope p-value > 0.05

Bias study

--study-type bias: bias = mean(measurements) - reference

  • %bias = |bias| / tolerance * 100
  • Acceptable: |%bias| <= 1% of tolerance

Clone this wiki locally