Validation Guide Sec. 9.2 — smoke concentration (Fig. 9.49) vs. obscuration (Fig. 9.51) summary plots

[ManuelOsburg]

Hi all,

I've been working through Section 9.2 Smoke of the Validation Guide and wanted to raise a question about the two smoke summary scatter plots — partly to make sure I'm reading them correctly.

Figure 9.49 ("Summary of smoke concentration predictions") reports mass concentration in mg/m³ and combines the NIST/NRC and FM/FPRF Datacenter datasets, with a Model Bias Factor of 2.46. Figure 9.51 ("Summary of smoke obscuration predictions") reports obscuration in %/m for the FAA Cargo Compartments dataset, with a Bias Factor of 1.02. Both figures for reference:

Looking at the experiment data files, all three datasets appear to be based on optical (light-extinction) measurements — the FAA experiment files use LT_* channels in %/m, NIST/NRC has a "Smoke Conc." channel from a laser extinction photometer, and the FM/FPRF data center work also relies on a laser signal. Mass concentration and extinction are linked through $K = K_m · c$, so for a single sample they're essentially interchangeable.

My concern is that the way the two summary plots are set up can leave a reader with the impression that "mass concentration is hard to predict, optical extinction is easy" — when the actual finding (Sec. 9.2.3) is that FDS overpredicts smoke in the closed-door NIST/NRC tests because agglomeration and wall deposition aren't modeled. That's an error in the transported soot inventory, and it would show up under either unit. The two figures differ in two ways at once — the reported quantity and the experiment set — so the unit and the physics end up confounded.

A couple of questions:

1. Is there a specific reason the FAA data is summarized as obscuration while NIST/NRC and FM/FPRF are summarized as mass concentration? I'd guess it's because a reliable $K_m$ isn't available for the FAA fuels, whereas it is for heptane/propylene — but I wanted to confirm.
2. Would it be worth either harmonizing the quantity across both plots, or adding a sentence near the figures clarifying that the difference in bias reflects the experimental configurations (well-ventilated vs. closed/under-ventilated) rather than the choice of unit?

Happy to be corrected if I've misunderstood the intent here. Thanks for all the work on the Guide.

Best regards,
Manuel

[mcgratta]

Usually we predict whatever the experimentalist has measured and reported. I'll have to check, but for smoke, sometimes the measurement pathlength is short so that one can talk about a single concentration over that distance. In other cases, the pathlength is longer in which case we prefer to mimic what was done in the experiment. I have not thought about what you have observed, although I get you point that it would be better distinguish what the code is predicting well and what it is not. Trouble is, one cannot do this with the experimental data. Usually the experimentalist uses some mass extinction coefficient to convert from obscuration to concentration. Ideally, it would be nice to have a direct measurement of particular concentration.

[ManuelOsburg]

Thanks, this is really helpful. The path-length point makes sense as a rationale for the FDS device setup — a long beam is better mimicked by an integrated obscuration than collapsed to a single concentration, and that matches the input files (FAA uses PATH OBSCURATION over a ~3 m beam, NIST/NRC a point DENSITY device).

Going back to the test reports, my reading is that the unit choice is driven less by path length than by what each experiment set out to measure. Path length and reporting unit clearly correlate, but the causal driver seems to be the intended use: the FAA report states % light transmission per foot was chosen "to allow direct comparison with the alarm point units used in TSO C1d" — the aircraft detector thresholds — whereas NIST/NRC wanted a mass concentration. The unit follows the purpose of each experiment, which is really my original point: it doesn't speak to how well the code does.

One related detail worth noting: the x-axis datasets in Fig. 9.49 don't even share a conversion convention — NIST/NRC derives concentration with Kₘ = 8.7 m²/g, FM/FPRF with 10.06 m²/g — while the FDS side outputs QUANTITY='DENSITY' for SOOT directly, with no Kₘ. A ~15 % effect, well within the stated Kₘ uncertainty, so not the driver of the 2.46 bias factor — that's the wall-deposition effect from Sec. 9.2.3 — but it does mean the comparison mixes conventions.

My main question remains the editorial one: even if the quantities stay as each experimentalist reported them, would it be worth one sentence near Figs. 9.49/9.51 noting that the difference in bias factor mainly reflects the experimental configurations (well-ventilated vs. closed/under-ventilated), not the choice of unit? That would pre-empt the "concentration is hard, obscuration is easy" misreading without touching the data or methodology.

Happy to draft such a sentence if useful.

Best regards,
Manuel

[mcgratta]

I'm wondering if the best solution is to plot all results in units of %/m, i.e. obscuration, because that is what is actually measured in the experiment: $-\frac{1}{L} \ln \left( \frac{I}{I_0} \right) $. The model results would have an extinction coefficient multiplied by its predicted particulate mass density, but at least the measurement remains in its original form. That is to say, the model includes both the particulate density and the ext coef, but not the experimental result.

The additional benefit doing it this way is that the more relevant quantity in fire protection is %/m, not soot kg/m3.

[ManuelOsburg]

That sounds like the right solution — cleaner than what I was suggesting. Keeping the measurement in its original form and putting both the particulate density and the extinction coefficient on the model side is the consistent choice, and %/m is also the more relevant quantity for fire protection anyway.

Two practical questions on implementation:

1. Would the model-side extinction coefficient stay at the default 8700 m²/kg for all cases, or be set per case? NIST/NRC and FM/FPRF are at 632.8 nm, the FAA cargo beam at 670 nm — so a single Kₘ would still carry a small wavelength mismatch for the FAA cases. Minor, just worth deciding deliberately.

2. The FM/FPRF data is published as mg/m³ (Table 4-19 in P14042, derived from the aspirated laser with σₑₘ = 10,056 m²/kg). Converting it back to %/m is exact as long as FM's own coefficient is used for the back-calculation — worth flagging so it doesn't accidentally get the 8700 value.

If it's useful, I can help with reworking that part of the Guide.

Best regards,
Manuel

[mcgratta]

I will create an issue from this discussion thread. I'll add Jason Floyd to the issue as well as myself. He is the one who worked with the FM data.

[ManuelOsburg]

Sounds good — thanks for setting that up. I'll watch for the issue and follow up there.