contentvalidR 0.10.1

A documentation release. The README now documents the handoff to nomologR in
its own section, matching the one the nomologR README carries for the
handoff from contentvalidR, so either front page explains how the two
packages connect. Released on GitHub and R-universe; CRAN keeps 0.4.0 until the joint
1.0.

No computed value changes. Across 25 analyses spanning every workflow,
every value 0.10.0 returns is identical in 0.10.1, and no function, argument,
or handoff field changed.

• The README has a section on the handoff to nomologR, just above the
  references: what a handoff carries, the two nomologR calls that read it, how
  to read its decisions, and how the schema is versioned. The nomologR README
  carries the mirror section, so the handoff can be found from either
  package's front page.
• Krippendorff (2011) now links to the copy on the Annenberg School's site,
  because the Penn repository copy had become unreliable to reach: it
  answered in 2 to 10 seconds, returned a server error, and timed out. It is
  the same paper, dated 2011.1.25, with its reading list updated in 2013. The
  expert-panel vignette's reference list now gives the link too.
• The reader in nomologR is now tested against handoffs from contentvalidR
  0.6.0 through 0.10.0, and the README says so.

contentvalidR 0.10.0

Tenth public release, and the last planned before the joint 1.0 release
candidate on 2026-10-17. v0.10.0 adds figures for papers, posters, and
teaching: the item flow diagram and the item evidence profile of
content_evidence(), and the full distribution of a panel's ratings, each in
APA gray or in color. It also states the handoff contract nomologR reads, makes
this package's walkthrough the joint one, and separates the Delphi printout's
caveats from its teaching.

contentvalidR is on CRAN: 0.4.0 was published there on 2026-09-28. CRAN asks
for updates no more often than every one to two months, so 0.10.0 is released
on GitHub and R-universe, and the next CRAN submission is the joint 1.0.

The package continues to declare Imports: stats only.

No computed value changes. Across 25 analyses spanning every workflow,
every value 0.9.0 returns is identical in 0.10.0. Relevance fits gain
details$ratings. Some printed wording changes: the Delphi printout's method
paragraphs move into its key, anova_content() prints within 80 columns, the
printed handoff advises passing the whole handoff to nomologR, and the
citation a handoff carries for stability = "percent_change" reads
"Scheibe et al. (1975/2002)".

No breaking changes. The plot methods gain arguments after their existing
ones: type, apa, and labels for expert-panel fits, and
which = "distribution", apa, and labels for Delphi fits.
agreement_summary() remains deprecated and is removed in 1.0.0.

Figures for papers, posters, and teaching

Each new figure draws what the fits and handoffs already decided, and says
whether it follows a published form or is this package's own design.

• content_evidence() (new) brings the handoffs from several review
  stages together, such as a relevance panel and then an item sort. It prints
  one row per item with each stage's decision beside the statistic its rule
  read, and draws two figures:
  • plot(x, type = "flow"), the item flow diagram, modeled on the PRISMA
    2020 flow diagram (Page et al., 2021): what each stage reviewed, what it
    held back and the number behind each decision, and what went forward;
  • plot(x), the item evidence profile, this package's own design: one
    panel per stage with each statistic, its interval, and its criterion, so
    agreement and disagreement between methods can be seen item by item.
• The distribution view draws every rating behind an index as diverging
  stacked bars (Heiberger & Robbins, 2014), split at the cut the decision rule
  counts: plot(fit, type = "distribution") for a relevance panel and
  plot(fit, which = "distribution") for a Delphi study, one bar per round. It
  shows what the I-CVI cannot: two items at 1.00, one rated relevant with 4s
  and the other with 3s. Relevance fits now keep their ratings in
  details$ratings to draw it.
• apa switches these figures between gray (TRUE, the default, as an
  APA figure is printed) and a colorblind-safe scheme for slides and posters
  (FALSE): teal for evidence that met its criterion, brown for evidence under
  review. Symbols carry the decision either way, so no reading depends on
  color.
• A relevance panel for the walkthrough items, walkthrough_relevance.csv,
  gives the twelve items a second source of content evidence.
  vignette("reporting-examples") uses it to show both figures, with a sample
  caption. The ratings are constructed, and a separate script,
  data-raw/build-walkthrough-panel.R, says what each is built to show. The
  three walkthrough files that nomologR ships are unchanged.
• The printed handoff and vignette("handoff-to-empirical-validation") now
  pass the whole handoff to nomologR, nomo_screen(data, items = handoff),
  rather than handoff$items, so the keying and the reasons for anything held
  back travel with the items. The vignette no longer calls the handoff reader
  in nomologR future work.

On CRAN

• contentvalidR 0.4.0 was published on CRAN on 2026-09-28. The README now
  gives install.packages("contentvalidR") first, carries a CRAN badge, and
  explains why CRAN's version can trail the newest release on R-universe:
  CRAN asks packages to update no more than every one to two months.

Finishing touches before 1.0

• The Delphi printout separates what changes a reading from what teaches
  it. Two short caveats always print: read kappa as a trend beside the
  share of experts who kept their rating, not against a cut-off; and with
  fewer than about 40 experts the kappa intervals cover less than their
  stated 95%. The fuller explanation (the weights, why kappa falls as a panel
  converges, Klar et al.'s coverage figures) now sits in the key under "How
  the stability statistic works", so an instructor's students see it and a
  researcher can hide it with options(contentvalidR.show_key = FALSE). The
  critique printed for an alternative stability method still always prints.
  The example printout goes from 100 lines to 92, or 58 with the key hidden.
• anova_content() prints within 80 columns. The strongest competitor, which
  htd() and the construct-rating workflow already show, stays in the
  strongest_competitor column.
• The website's Articles menu lists every guide under a heading, in the order
  a course would use them: Start here, Workflow guides, Planning and
  reporting, and After content review. The workflow guides were hidden under
  "More articles".
• Scheibe et al. is cited from the edition the package's method was checked
  against: the editors' 2002 web edition of Linstone and Turoff, with the
  chapter's pages (257–281), a link, and "(Original work published 1975)", as
  APA 7 requires for a republished work. In-text citations read
  "Scheibe et al. (1975/2002)", including the citation a handoff carries for
  stability = "percent_change". The 1975 printing's pages could not be
  checked.

The handoff contract, stated for readers

Asked for by nomologR before handoff schema 1 becomes a 1.0 promise. The
schema and every handoff are unchanged.

• ?content_handoff has a new section, "Reading the decisions": take each
  item's decision from carried and status, never re-derive it by
  comparing value with criterion or by branching on the producer version.
  The example is the nine-expert correction in 0.8.0, where an item's I-CVI
  stayed at .778 while its decision changed. The section also restates that
  keying of NA means unknown, never forward-worded.
• ?content_handoff says an item sits in at most one scale, the one its
  item_evidence$scale names. A test holds the sort, rating, and congruence
  workflows to it, and checks that each refuses an item with two targets.

The joint walkthrough lives here

• vignette("one-item-set-both-stages") is the joint walkthrough for
  contentvalidR and nomologR, as the maintainer decided for the joint 1.0.
  It now says so, passes the handoff itself to nomo_screen() and
  nomo_run() rather than h$items, and links to the section of the
  nomologR guided workflow that continues from content review. Two stale
  sentences are corrected. The handoff reader in nomologR, which the vignette
  called future work, has shipped. And the nomologR guided workflow starts
  from the same handoff but simulates its own responses, so its screening
  numbers differ from these.
• The walkthrough links to the companion article in nomologR, "From content
  review to empirical screening", which continues it on the same responses.
  nomologR ships the item and response files of the walkthrough unchanged,
  with the script that generates them.

contentvalidR 0.9.0

Ninth public release. v0.9.0 is about how the package presents itself before
1.0. Every reference list is in APA 7 form; printed output is shorter and says
what it means; the component functions and report tables print in APA style;
the figures show the criterion and the uncertainty; and the README is a front
page rather than a manual.

The package continues to declare Imports: stats only.

No computed value changes. Across 25 analyses spanning every workflow,
every value 0.8.0 returns is identical in 0.9.0. Some wording changes: the
scale-level summaries of the item-sort and construct-rating workflows name the
Colquitt band of the weaker index, counts read "1 item" or "3 items", and the
note when judge severity cannot be estimated no longer blames near-total
agreement when one dissenting judge causes it.

Breaking changes.

• content_report() writes an APA table by default. For the numeric table
  that used to be the default, pass format = "data.frame"; its p values
  now keep three decimals whatever digits is.
• The component functions' results (compute_psa(), aikens_v(), cvr(),
  and eleven others) gain a class, so they print as formatted tables. Every
  value is unchanged and as.data.frame() returns the plain data frame; only
  code that tests class(x) == "data.frame" exactly would notice.

Deprecated. agreement_summary() warns and will be removed in 1.0.0; use
panel_agreement(), which takes raters in rows like every other ratings
function.

New: content_report(format = "apa"), and csv_binom_test() returns its
inputs (n_target, N, p0, alpha) beside its result.

Published on GitHub and R-universe. 0.4.0 is still in CRAN's review queue, and
CRAN policy asks that no further version be submitted while one is pending.

A README that is a front page

• The README is a front page rather than a manual. It ran to about 1,400
  lines of printed output: every workflow in full, the component functions,
  and six copies of the key. It now shows one first analysis (the printout,
  its APA table, and its plot), a table matching each question put to judges
  or experts to its workflow, the sources it is built on, and its guide, then
  how to read the output, what every workflow returns, and what you can rely
  on. The worked examples it held live in the vignettes.
• vignette("design-and-reporting") gains the planning examples the README
  held: expert_power(), with why a fourth or fifth expert lowers the chance
  of clearing Lynn's criterion, and gtheory_content(), with how many judges a
  design would need. Its method list now names all six workflows, and it has a
  reference list.

Plots show the criterion and the uncertainty

No computed value changes; this is about what the figures draw.

• The item-sort Psa plot draws each item's interval, and a dashed mark at
  the share of judges the exact test needs for that item, so an item is
  retained when its point reaches its mark. It used to show bare points.
• The expert relevance plot shows Aiken's V and I-CVI side by side, each
  with its interval, and a dashed line at the I-CVI criterion when every
  item had the same number of experts ("I-CVI criterion (5 of 6)"). Before,
  only V had an interval, and an I-CVI point could hide behind V's.
• Legends list only what the figure draws and sit in headroom above the
  data. The item plots offered a "No data" symbol no item used, the profile
  plot a "Review gap" line it had not drawn, and the essentiality key sat on
  its own zero line. The content map's key is titled "Cluster" or "Blueprint
  cell", where it showed bare numbers.
• Axis labels say what is measured ("Psa: share of judges choosing the
  target") in place of "Psa correspondence", and axes for statistics that
  cannot exceed 1 are labeled as APA prints them (.25, .50). No figure leaves
  an empty band for a title unless main is given, and each restores the
  graphics settings it changes. Item rows run top to bottom in the order of
  the results table, and scale means are triangles, clearly distinct from
  retained items.
• Every figure in the README and the vignettes has alt text describing
  what it shows; 17 of 19 had none.

Shorter keys, formatted components, and APA report tables

No computed value changes; this is about what is printed.

• The key under each result explains only what it shows. It ran 12 to 36
  lines and was often longer than the results. Each column now gets one line
  after its full name, and "What the decisions mean" lists only the decision
  words that appear, saying what each means in that workflow ("Retain --
  met the exact target-assignment criterion"). The old legend listed all
  four shared statuses and left readers to match Retain or Typical to them.
  contentvalid_glossary() keeps the full definitions and now lists every
  workflow's decision words. The item-sort example in the README prints 58
  lines instead of 71.
• The component functions print as formatted tables. compute_psa(),
  compute_csv(), htc(), htd(), anova_content(), aikens_v(), cvr(),
  ioc(), interpret_colquitt(), colquitt_benchmarks(), csv_binom_test(),
  signal_detection(), reproducibility_phi(), and similarity_from_sort()
  printed raw data frames and lists, such as a p of 1.941920e-16 beside
  seven-digit decimals. They now print as cvi() already did: a titled table
  in APA style with a short note. anova_content() reports its tests as
  F(1.71, 32.57) = 108.55. The values are unrounded; each result only gains a
  class, as.data.frame() returns the plain data frame, and workflow results
  are unaffected. csv_binom_test() also returns its inputs.
• Breaking: content_report() writes APA tables by default. The new
  default format = "apa" gives readable headings, counts such as 18/20,
  p to three decimals, and [LL, UL] intervals, and prints without row
  names; format = "markdown" writes the same table and prints as the
  table. For the numeric table that used to be the default, pass
  format = "data.frame"; its p values now keep three decimals whatever
  digits is. The construct-rating table shows the contrast p beside the
  omnibus p, since a Review item can have an omnibus p below .001.

agreement_summary() is deprecated

• Deprecated in 0.9.0, to be removed in 1.0.0. Use panel_agreement().
  agreement_summary() is the only function that takes items in rows; every
  other ratings function, panel_agreement() included, takes raters in rows,
  so one matrix passed to both was read two different ways.
  panel_agreement() also reports an interval and needs no other package.
  agreement_summary() still works in 0.9.0 and warns when called. Removing
  it at 1.0.0 leaves nothing deprecated past 1.0, as the stability policy in
  ?contentvalidR sets out.

References follow APA 7 everywhere

• One reference list, installed in BibTeX form. REFERENCES.bib held 13
  of the works the package cites, and one it no longer cited; it now holds all
  53, generated from the reference list in the README, with author names and
  details checked against the Crossref record for each DOI. A test keeps the
  two in step. The README calls it a reference list rather than a
  bibliography.
• Every reference list is in alphabetical order, as APA 7 requires, in the
  README, every help page, and every vignette. They had been ordered by topic.
  Page ranges use en dashes.
• Corrections found by the check. De Boeck and Wilson (2004) are the
  editors of Explanatory item response models, not its authors. Book titles
  are in sentence case. Four works gain the DOI they lacked (Brennan, 2001;
  De Boeck & Wilson, 2004; Newcombe, 1998; Vach & Gerke, 2023). Colquitt et
  al. (2014) and the irr package, both cited in help pages, join the list in
  the README.
• citation("contentvalidR") prints the package reference in APA form, with
  its version and "[Computer software]".

Printed output says what it means

• The expert-panel print opens with its verdict, as the other workflows'
  prints do: how many items met the criterion, then the flagged items by name.
  0.8.0's release notes said every print did this; the expert-panel print
  showed only a tally. Items rated by fewer than three experts are named too,
  where the tally left them out.
• The key no longer says modified kappa runs from 0 to 1. It falls below 0
  whenever fewer experts agree than chance predicts: with four experts, an
  item none of them rates relevant has a modified kappa of -.07.
• An essentiality panel with no usable ratings no longer prints "an item needs
  at least NA rating it essential".
• The scale-level summaries of the item-sort and construct-rating workflows
  name the band the weaker index falls in ("The weaker of Psa and Csv falls in
  the Moderate band of published scales"), where they spoke of "normative
  standing". The construct-rating print names its benchmark set the way the
  item-sort print does.
• Every summary heading has the same form and rule; counts read "1 item" or
  "3 items" rather than "item(s)"; the handoff says held-back items are listed
  only when some are; and the note when judge severity cannot be estimated no
  longer attributes it to near-total agreement when one judge rejecting the
  items the others accept causes it.

No computed value changes.

contentvalidR 0.8.0

Eighth public release. v0.8.0 is about reading the output and the literature
correctly. Every printed result now reads as a report in APA style, every
source is cited where its method is used, and the earlier published rules can
be shown beside each decision for teaching, each with its shortcomings stated.
One criterion now matches its source, and the last deprecation is completed,
so nothing deprecated is carried into 1.0.

The package continues to declare Imports: stats only.

Results that change. Values computed by 0.7.0 and 0.8.0 were compared on
25 analyses spanning every workflow. The only differences are these:

• Lynn's criterion at nine experts. An item endorsed by 7 of 9 experts now
  meets the I-CVI criterion, as Lynn's (1986) Table 2 says; 0.7.0 required 8.
  cvi_criterion now shows each panel size's exact cutoff, such as .875 for
  eight experts, where it showed .78 throughout. Beyond ten experts, where her
  table stops, the package holds her 7 of 9. That lowers the requirement by
  one expert at every multiple of nine (14 of 18, 21 of 27, 28 of 36, and so
  on) and at some panels above 58. Verdicts at other panel sizes are
  unchanged.
• judge_validity()'s influence check. It removes one judge at a time, so
  a ten-judge panel was checked at nine, where 0.7.0 applied the wrong
  criterion. On a ten-judge panel this could flag every judge as influential.
  It now applies Lynn's.
• A label. A judge whose fit falls below the range is now Too predictable, not Erratic. The status is Review either way.
• One removed column. anova_content() no longer returns posthoc_pass;
  read contrast_pass, which holds the same value.
• Handoff rule text reads as counts and APA numbers. The schema is
  unchanged.

One display default changes. Print methods now default to digits = 2, as
APA reports statistics; print(x, digits = 3) restores the previous
precision. Stored values are unaffected.

New arguments: sort_validity(legacy = , n_constructs = ),
expert_validity(legacy = ), and legacy in their print() methods.

Published on GitHub and R-universe. 0.4.0 is still in CRAN's review queue, and
CRAN policy asks that no further version be submitted while one is pending.

The last deprecation is completed before 1.0

• Breaking change: anova_content() no longer returns posthoc_pass. It
  was a duplicate of contrast_pass, documented as deprecated in 0.7.0. The
  written policy requires one minor release of notice and allows removal in a
  minor release only before 1.0, so this is the last release that can remove
  it without waiting for 2.0. Read contrast_pass, which holds the same
  value. It was the last column, so no other column moves.
• No deprecation is in progress now, so nothing deprecated is carried into
  1.0. ?contentvalidR records both of anova_content()'s completed cycles.

Earlier methods, for comparison

For teaching, the way a methods text reports eta-squared beside omega-squared,
sort_validity() and expert_validity() now compute the earlier published
rules beside their own. legacy = TRUE prints them under "Earlier methods, for
comparison (not used for the decision)", and print(fit, legacy = TRUE) shows
them for any fit. They are stored in details$earlier_methods and never change
a decision. Each follows its source as published, and the tests check each
against the numbers its source prints.

• Item sort.
  • Anderson and Gerbing's (1991) critical Csv, from their Equations 5 and 6:
    .50 for 20 judges at .05. The printout explains why it is not used to
    decide: it assumes every judge who misses the target picks the same rival,
    so when they spread across several constructs it can pass an item the
    exact test does not.
  • Yao, Wu and Yang's (2008) cutoffs, Psa and Csv both at least .30, set for a
    four-domain sort.
  • A labeled package extension of Yao et al.'s rule to other numbers of
    constructs: chance plus .05, so Psa and Csv both at least 1/k + .05,
    which is their .30 at four. It prints in its own column marked * and is
    described as "a contentvalidR extension, not a published rule". The new
    n_constructs argument states how many constructs judges were offered;
    by default the package counts the ones they used, and says so.
  • A count of how often each earlier rule agrees with the decision, and a
    note that pooling every rival into Csv gives a different index.
• Expert essentiality. Lawshe's (1975) Table 1 minimum CVR, the
  recalculation by Wilson, Pan and Schumsky (2012), z / sqrt(N), and Lawshe's
  content validity index, the mean CVR of the items his table retains. A panel
  size Lawshe did not tabulate gets no minimum rather than an interpolated one.
  When a CVR prints equal to a cutoff it misses, such as 8 of 9 (.778) against
  Lawshe's .78, the printout says so.
• Expert relevance. Fleiss' (1971) kappa on the relevant/not-relevant
  decision, computed by the package and checked against the irr package, and
  the benchmarks of Polit and Beck (2006): .90 for S-CVI/Ave, which is the
  average congruency percentage, and .80 for S-CVI/UA.
• Expert relevance: the content validity coefficient (Ccv) of
  Hernández-Nieto (2002), beside Aiken's V, read from the full text of the
  book's chapter.
  It is shown because it is widely cited, and never informs a decision,
  because of shortcomings the printout, ?expert_validity and the
  expert-panel vignette all set out:
  • It uses only each item's mean, so it cannot reflect agreement, although
    the book says it does. The book's own Table 7 gives ratings of 1, 3, 4,
    5, 2 and of 3, 3, 3, 3, 3 the same .60.
  • Its chance correction, (1/J)^J, depends only on the number of judges. It
    is .00032 for five of them.
  • On a scale starting at 0 it is Aiken's V; on a scale starting at 1 it
    cannot fall below one over the maximum.
  • Its .80 and .90 bands are not derived, and the book's own Example 11
    calls .7968 acceptable.
• The item-sort vignette section that explained why Anderson and Gerbing's rule
  was "not exposed" now shows it for comparison, using the shipped example
  where it and the exact test disagree. The expert-panel vignette shows
  Lawshe's anomaly at nine experts.

Every vignette lists what it cites

• Four vignettes cited works without a reference list, and the expert-panel
  vignette cited Lynn (1986) without listing it. Each now has an APA reference
  list, and vignette("reading-output"), which explains the APA number rules,
  cites the Publication Manual. Yao et al. (2008) joins the README references.

Printed output reads as a report, in APA style

No statistic changes. Only printed output changes, and results still holds
every value at full precision.

• Numbers follow the APA Publication Manual, seventh edition. A statistic
  that cannot exceed 1 prints without a leading zero (Psa = .90, p = .021,
  kappa = -.13). One that can exceed 1 keeps it (a chi-square of 0.31, a
  severity of -0.27 rating points). Values have two decimals, and p values
  have three, with anything below .001 printed as < .001 rather than 0.000
  (Section 6.36).
  Intervals print as 95% CI [.70, .97] in a column headed with their level
  (Section 6.43). digits now defaults to 2 in every print method.
• Every print opens with its verdict. A sentence says how many items met
  the criterion and names the ones flagged, before any table. The decision
  appears beside each item. Anything that is the same on every row, such as the
  I-CVI criterion for a panel of one size, is stated once in a sentence instead
  of repeated down a column.
• No more walls of numbers. Tables are laid out to fit an 80-column
  console (long item names can still widen them), and prose wraps to the
  console width. The summary() methods show flagged items in the same columns
  as print(), where the expert-panel summary used to print its whole results
  table. An explanation shared by several items, such as a
  Delphi note or a judge's flag, is printed once with every item it applies
  to.
• Planning tables have one row per panel size. expert_power() and
  sort_power() put each assumed probability in its own column, so ten panel
  sizes take ten rows instead of thirty.
• The key names columns by the heading you see (Psa, 95% CI, kappa), not
  by the name of the column in results.
• Headings are words. n_target and n print together as judges (18/20),
  severity(pts) as severity, differentiation as scale use, and the
  recommendation column as decision.
• delphi_validity() prints the degrees of freedom beside a chi-square, as APA
  reports it. judge_validity() prints infit beside outfit, since either can
  flag a judge, and states the rules that flag one.
• The README and vignettes follow the same rules, and the vignette on reading
  output explains them.

Two labels now say what they mean

• judge_validity() no longer calls a too-predictable judge "Erratic". A
  fit mean square above the range means ratings noisier than the model expects;
  one below it means ratings more predictable than it expects, the opposite
  problem. Both were labeled Erratic. The second is now Too predictable,
  with its own interpretation. The status is Review either way, and which
  judges are flagged does not change. Code that matched recommendation == "Erratic" to find every misfitting judge should also match "Too predictable", or use status.
• The handoff states the I-CVI criterion as a count. Its rule text read
  I-CVI >= 0.8333333. It now reads at least 5 of 6 experts rate the item relevant (I-CVI >= .83; Lynn, 1986), since the decision compares counts, and
  the handoff now cites Lynn (1986). Rule text is prose for a person and
  outside the frozen schema, which is unchanged.

The I-CVI criterion now matches Lynn's table

• **Changed resul...

contentvalidR 0.7.0

Seventh public release. v0.7.0 is about being safe to depend on. It says in
writing what code can rely on from one version to the next, freezes the
handoff object that another package reads, and settles the one deprecation
that a 1.0 release would otherwise lock in. It also adds a worked example that
carries a single item set from an expert panel into response data, and it
corrects the stated basis of the Delphi kappa interval.

The package continues to declare Imports: stats only.

One breaking change. anova_content() no longer accepts posthoc, and
passing it is an error. The argument has had no effect on any result in any
release, and it has warned in every one since 0.1.0. See below.

No default changes, and no computed value changes. Across all six workflows,
every Delphi stability method, and the bootstrap intervals, every value 0.6.0
returns is identical in 0.7.0. Handoff objects gain four columns (note,
keying, response_min, response_max), all appended within schema
version 1, so a reader written for 0.6.0 still reads them.

New arguments: content_handoff(reverse_keyed = , response_scale = ). New
vignette: vignette("one-item-set-both-stages"). New example data: three
walkthrough_*.csv files in inst/extdata.

Published on GitHub and R-universe. 0.4.0 remains in CRAN's review queue, and
CRAN policy asks that no further version be submitted while one is pending.

The one deprecation in flight is resolved

• Breaking: anova_content()'s posthoc argument is removed
  (#60). Passing it is now
  an error rather than a warning.
  • Replacement: none is needed. Tukey/Duncan post-hoc testing was dropped
    before the first release because the Hinkin-Tracey question is answered
    directly by planned target-versus-orbiting contrasts, which
    anova_content() has always run. The argument has had no effect on any
    result in any version.
  • It warned from the beginning. The deprecation warning was in the
    initial commit and stood through 0.1.0, 0.2.0, 0.3.0, 0.3.1, 0.4.0, 0.5.0,
    and 0.6.0 — seven releases against the policy's minimum of one.
  • Why now. The stability policy removes a deprecation in a minor release
    before 1.0, and only in a major release from 1.0 onward. This is the last
    release that can do it without waiting for 2.0.
  • The argument sat seventh in the signature, ahead of alpha, so anyone
    who passes arguments by position past target_map should check their call.
• posthoc_pass is now documented as deprecated. It is a duplicate of
  contrast_pass and has been returned silently since the first release,
  without ever being marked. It is still returned and still correct, since
  the policy requires a release in which reading it keeps working; read
  contrast_pass instead. It is the last column, so removing it later cannot
  move another.
• Both events are recorded here, which is the point. The policy says
  NEWS.md records a deprecation and its removal with the replacement; the
  posthoc deprecation was never recorded at all until now. ?contentvalidR
  now cites posthoc as the completed cycle and posthoc_pass as the one in
  progress, and the test suite checks that both are in the state the policy
  claims.
• csv_binom_test() moves from Tier 3 to Tier 2. Tier 3 describes helpers
  "kept for continuity with older analyses", which are "not recommended
  workflows" and "may be deprecated and removed". That is the wrong description
  of the Howard and Melloy (2016) exact test that sort_validity() runs on
  every item. The tiers become promises at 1.0, so a load-bearing function had
  to stop being filed under the one that promises least. No code changes.

One item set carried through both stages

• New vignette("one-item-set-both-stages") follows twelve items from an
  expert panel through the handoff and into response data
  (#55), so the pair of
  stages is demonstrated rather than asserted. Every step runs: the second half
  uses stats only, and the nomologR call is shown but not run, since
  neither package depends on the other.
• The point of the walkthrough is the disagreement between the stages, and it
  runs in both directions.
  • Two items fail content review, for opposite reasons: one keeps its target
    and misses the criterion, the other loses its target to a competing facet.
  • Two pass content review and then misbehave. EF4 is sorted to effort
    regulation by eighteen of twenty judges and carries almost no common
    variance; TF4 is sorted to one facet and loads on both. A panel cannot
    see either, which is the reason for running the second stage.
  • EF3 is the reverse, and the sharper case: an empirical screen flags it
    and it is worth keeping. Nearly everyone endorses "I finish the assignments
    that count toward my grade" — 96% answer in the top two categories — so its
    variance is a third of every other item's and it cannot correlate strongly
    with anything. Its correlation looks as bad as EF4's, and only the
    distribution tells them apart. It is also the only item covering the
    completion of required work, so dropping it would narrow the domain the
    panel defined. Added at the nomologR maintainers' suggestion, so that both
    packages' articles make the same point about it.
  • Meanwhile the item that met the content criterion by a single judge behaves
    perfectly well, so a borderline content result is not read as a
    prediction.
  • EF2 and TF2 are written the other way round, and the walkthrough
    passes that to content_handoff(reverse_keyed = ) along with the
    respondents' one-to-five scale. It then shows why the handoff has to carry
    both: before recoding, TF2 correlates at -0.51 with its own facet, which
    looks like the strongest evidence against an item that item analysis
    produces and is in fact a coding error. Recoded with the keying and the
    scale limits the handoff carries, it is +0.51. Added together with the
    keying field, at the nomologR maintainers' request, so that no reader ever
    receives reverse-worded data without the means to know it.
• Three new files in inst/extdata: walkthrough_items.csv (the items, their
  facet, their stems, what each one was built to do, and whether it is
  reverse-worded),
  walkthrough_sort.csv (twenty judges), and walkthrough_responses.csv (400
  respondents, twelve items, a two-level cohort variable, and the rejected
  items still present). They are simulated. No participant was involved and
  no real instrument is reproduced.
• data-raw/build-walkthrough-data.R writes all three and states the
  generating model in full: the two correlated facets, every loading, the
  response thresholds, and the single cohort shift. It uses base R only and is
  deterministic. nomologR mirrors the response file from the same script, so
  the two packages cannot drift.
• A test file holds the vignette to its claims: which two items the panel
  flags, which met the criterion by one judge, that EF4 is the weak one and
  TF4 the ambiguous one, that EF3 is flagged for restricted variance rather
  than for lacking common variance, and that TF6 is the only item that shifts
  by cohort. If the data are regenerated and a claim stops holding, the suite
  fails rather than the vignette quietly becoming wrong.

Handoff schema version 1 is frozen

• ?content_handoff gains two sections. What version 1 freezes names every
  field and column a reader can rely on, states that each keeps its name,
  position, and type, and lists what is deliberately not frozen: which rows
  appear, the labels in the statistic column, the prose in note, rule,
  recommendation, and citation, the contents of settings and design,
  and the printed output. If the schema ever changes gives the version-2 path
  and the two-line version check a reader should gate on.
• The frozen contract is now data rather than prose, so the tests and the
  documentation cannot drift apart. Handoffs from all five workflows that can
  produce one are checked against it, column by column, along with the
  guarantee that makes the freeze useful: every handoff carries every column,
  so a statistic with no interval carries NA in all four interval columns and
  a workflow with no panel coefficient returns a zero-row panel_statistics
  with the full column set. Handoffs from different workflows therefore bind
  without reconciling them first.
• Before the freeze closed, it was checked against every reader.
  • nomologR confirmed the frozen list covers everything its reader consumes,
    and named two gaps, each blocking a specific computation rather than
    expressing a preference. Both are added.
  • solomonR was asked too
    (solomonR#32): its
    fit_solomon_sem_latent() takes item names as character vectors, which is
    exactly what a handoff carries.
• content_handoff() gains reverse_keyed and response_scale, recorded
  per item in three new item_evidence columns: keying (1 forward, -1
  reverse-worded), response_min, and response_max.
  • Keying, because an even-odd consistency index must recode reverse-worded
    items first or a consistent respondent looks careless, and because a
    negative corrected item-total correlation means a coding error on an item
    that was never recoded and evidence against the item on one that was.
  • The response scale, because screening for out-of-range answers needs the
    scale's limits rather than the observed ones: a category nobody used is
    still a legal answer.
  • Both come from the analyst, never from the fit. A panel's lo and hi
    are the scale the experts rated relevance on, usually 1 to 4, not the scale
    responde...

contentvalidR 0.6.0

Sixth public release. v0.6.0 finishes the Delphi workflow: a Delphi study can
now be handed off to empirical validation, carrying for each item the round it
settled in, and plot() shows the two trends a Delphi is about. It also adds
a test layer that checks the package's printed claims against what the code
actually computes.

The package continues to declare Imports: stats only.

No default changes. Existing calls return the same values as in 0.5.0.

New method: plot() for delphi_validity(). content_handoff() accepts a
Delphi fit.

Published on GitHub and R-universe. 0.4.0 remains in CRAN's review queue, and
CRAN policy asks that no further version be submitted while one is pending.

Seeing a Delphi's trends

• plot() for delphi_validity()
  (#44), so the Delphi
  workflow matches the others in having one.
  • which = "consensus" (default) draws each item's share of experts
    agreeing, round by round. A line that stops early belongs to an item that
    settled and was set aside, so the picture shows when each item left the
    study.
  • which = "stability" draws the stability statistic per pair of rounds,
    with the share of experts who kept their rating as open circles.
  • Nothing is drawn that the analysis did not apply. The consensus
    threshold appears only when one was set, and kappa gets no shaded bands,
    because its verbal benchmarks are arbitrary and it falls as a panel
    converges.
  • Item labels sit at the end of each line and are nudged apart when items
    finish at the same height.

Printed claims are tested against what the code computes

• A new test file checks the package's printed output against the fitted
  objects it describes, rather than against strings copied from the print
  methods (#41). A test
  that asserts what the documentation claims can pass while being false. It
  checks that:
  • every number printed for an item is the number the object holds;
  • the printed settings are the settings that ran;
  • a method's critique appears when that method ran, and not otherwise;
  • percentages written into prose match the computed proportions;
  • a claim about how a statistic was computed holds of the numbers. The
    note that quadratic weighting makes kappa an intraclass correlation is
    checked against the intraclass correlation itself, and is absent under
    linear weights, where it would be false;
  • compare_rounds() claims comparability only when the settings match;
  • the status legend's example words are real recommendation values in some
    workflow;
  • statistics stay inside the ranges the glossary claims, under unanimous,
    rejected, and evenly split panels.
• .print_key() now fails on an unknown term instead of dropping it. A typo
  in a print method's key used to remove a column's explanation in silence.

A Delphi study can hand off

• content_handoff() now accepts a delphi_validity() fit
  (#42). It used to
  refuse one, and its message did not even mention Delphi.
  • Each item travels with its own last round. A Delphi settles one item
    at a time: an item that reached consensus early was set aside before the
    final round. round now holds that round's index, so it varies between
    items. This is the first workflow where it carries real information.
  • Relevance evidence comes from the item's last round, taken from that
    round's expert_validity() fit, with intervals: I-CVI against the
    consensus threshold, Aiken's V, and modified kappa. Modified kappa
    carries no criterion, because a Delphi decides on the consensus
    threshold rather than on the 0.74 rule.
  • Stability travels as evidence, never as a carry decision:
    proportion unchanged plus the statistic that ran, named for its
    method, such as weighted kappa (quadratic) or Goodman-Kruskal lambda. The chi-square methods add stability p_value. The names are
    qualified so a downstream report cannot misread them
    (nomologR#46).
  • round cannot be set by hand for a Delphi fit, since the fit supplies it.
• No schema change. These are new values in existing columns, so a schema
  version 1 reader still reads the object.

contentvalidR 0.5.0

Fifth public release. v0.5.0 adds a workflow for Delphi studies, where the same
expert panel rates items over successive rounds. It reports consensus and
stability as separate questions, and every published stability method is
available with its limits printed. The handoff to empirical validation now
carries each statistic's interval, so a downstream analysis can see how much
panel evidence stands behind a number.

The package continues to declare Imports: stats only.

No default changes. Existing calls return the same values as in 0.4.0.
content_handoff() objects gain four interval columns in item_statistics
and a panel_statistics table, within schema version 1.

New function: delphi_validity(). New article: vignette("delphi-rounds").

This release is published on GitHub and R-universe. 0.4.0 is still in CRAN's
review queue, and CRAN policy asks that no further version be submitted while
one is pending, so 0.5.0 will be submitted to CRAN later as an update.

Delphi consensus and stability

• New workflow delphi_validity() for an expert panel rated over successive
  Delphi rounds (#20).
  It reports two questions separately. Consensus: whether enough experts
  agree now. Stability: whether experts are still changing their ratings.

  • Consensus is the share of experts rating an item at or above
    agree_cut, which on a relevance scale is the I-CVI. It is judged
    against a consensus_threshold fixed before the study. There is
    deliberately no default: without a threshold, results are descriptive
    (Diamond et al., 2014).

  • Stability always reports the share of experts who kept their rating.
    Beside it goes one statistic:

    • the default, weighted kappa between rounds (Holey et al., 2007), read
      as a trend. Quadratic weights, the default, make it the intraclass
      correlation of the two rounds (Fleiss & Cohen, 1973); linear weights
      are available (Cohen, 1968);
    • Chaffin & Talley's (1980) lambda;
    • Chaffin & Talley's individual chi-square;
    • Dajani, Sincoff & Talley's (1979) group chi-square;
    • Scheibe, Skutsch & Schofer's (1975) 15% net-change rule.

    Each alternative's limits are printed. The two chi-square tests read in
    opposite directions, and the printout says which is which.

  • No verbal kappa labels. Landis & Koch (1977) call their divisions
    arbitrary, and kappa falls as a panel converges. When a round is
    unanimous, kappa is 0 however many experts kept their rating; the output
    flags this and points to the share unchanged.

  • A bootstrap interval for kappa, resampling experts, marked as this
    package's extension.

  • Round-by-round relevance evidence. Each round is also fitted with
    expert_validity(), so compare_rounds() can read the whole process.

• New vignette, vignette("delphi-rounds"), walks through a three-round
  panel for a graduate reader. Its worked example covers:
  • a statement set aside after reaching consensus;
  • an expert leaving the panel;
  • consensus against a statement;
  • a stable split;
  • a panel converging to unanimity, where kappa drops to 0;
  • experts swapping ratings while the overall distribution stays still,
    which only individual stability detects.
• contentvalid_glossary("delphi") defines the new columns.
• Every published value used to design the workflow is a test: Cohen's
  (1968) Table 1, Fleiss & Cohen's equivalence, and the worked examples of
  Dajani et al. and Chaffin & Talley.

Interval bounds in the handoff

• content_handoff() now carries each statistic's interval
  (#33), so a reader can
  tell a unanimous four-judge panel from a unanimous twenty-judge one. A
  unanimous I-CVI from four experts has a Wilson 95% lower limit near 0.51.
  • item_statistics gains lower, upper, interval_method, and
    interval_level. They hold the Penfield-Giacobbi score interval for
    Aiken's V, and whichever proportion_ci method was chosen for I-CVI and
    Psa.
  • A new panel_statistics table carries the panel agreement coefficient
    and its bootstrap interval, which are not per item.
  • NA in all four columns means the statistic has no interval, never
    missing data.
• These additions stay within schema version 1, as agreed with nomologR on
  nomologR#46.
  • Existing fields keep their names, types, meaning, and column positions.
  • A reader should check that the new columns are present before using
    them.
• The handoff reports intervals only. It does not turn them into priors or
  weights.

contentvalidR 0.4.0

Fourth public release.

v0.4.0 connects content validation to what comes after it. A content-validity pretest ends with a decision about which items go forward into data collection. Until now that decision travelled as retyped item names, and the reasons behind it stayed behind. content_handoff() packages the decision, so the item set and its evidence move together into empirical scale development.

The package still declares Imports: stats only, so it installs without a compiler toolchain. No default changes: existing calls return the same values as in 0.3.1.

Install

install.packages(
  "contentvalidR",
  repos = c("https://juhalt.r-universe.dev", "https://cloud.r-project.org")
)

Handoff to empirical validation

handoff <- content_handoff(fit)
handoff$items            # the carried items, ready for the next stage
handoff$item_evidence    # every reviewed item, with the rule that decided it

The handoff carries:

• the item names that survived content review;
• the construct each belongs to, where the design defines one;
• a per-item evidence table with the effective number of judges and the decision rule, in words;
• the statistics behind each decision, with the criterion each was compared against;
• the provenance of the analysis: workflow, settings, method, citations, version, and date.

Items that did not qualify are not dropped. They stay in the evidence table with carried = FALSE, because review is not deletion. The printed output says plainly that surviving content review is evidence about relevance and representation, not a prediction that an item will behave well empirically.

The object shape is schema version 1, agreed with nomologR, the companion package for the empirical stage, after a round of counter-proposals between the two. Every field is a base type, so neither package depends on the other. Two refusals are built into the agreement:

• contentvalidR never labels a held-back item as "dropped".
• nomologR never invents construct membership for an expert panel that rated a single item set.

New vignette: From Content Validity to Empirical Validation.

A stricter parallel analysis criterion

v0.3.0 made Horn's (1965) parallel analysis the default for qfactor_content(). Glorfeld (1995) found Horn's mean criterion still tends to retain one or two factors too many, and proposed comparing each eigenvalue against an upper percentile of the simulated distribution instead.

qfactor_content(ratings, parallel_criterion = "percentile")   # Glorfeld, 95th percentile

Horn's mean stays the default. Both criteria read the same simulation, so one seeded run compares them directly, and the result records which criterion produced the retained number of factors.

Packaging

Both 0.3.x CRAN submissions were returned by the incoming checks, each for a problem neither the local --as-cran run nor the CI matrix could see:

• 0.3.0 was returned because README links pointed at files excluded from the built package. 0.3.1 fixed that.
• 0.3.1 was returned because its tarball, built from a git worktree, carried a .git file that R CMD build does not drop. .Rbuildignore now excludes it, and release tarballs are now listed to confirm they contain no hidden files before upload.

0.4.0 goes to CRAN with both fixes.

Benched, with reasons

Bayesian content-validity extensions stay out of this package. Bayesian Instrument Development needs participant response data and MCMC, which puts it past the handoff and outside the zero-dependency profile. It is recommended for nomologR, where the interesting version would use content-validity evidence as informative priors. The reasoning is recorded in #2.

Verification

• R CMD check --as-cran on the built 0.4.0 tarball: 0 errors, 0 warnings. Its two notes are "New submission" and a component missing on the local check machine.
• The built tarball contains no hidden files and no build-ignored files, confirmed by listing its contents rather than inferred from a passing check.
• A clean-library install of the tarball ran the examples for all 40 help topics, all eight vignettes, and the citation.
• The handoff has 14 tests covering all three item-level workflows, both congruence modes, the construct mapping, per-workflow judge counts and decision rules, and refusals. Its output was shared with nomologR as real objects rather than prose.
• The percentile criterion is tested to be no less strict than Horn's mean on the same simulation, and to retain no more factors on pure noise across replications.

Full changelog: https://juhalt.github.io/contentvalidR/news/index.html

contentvalidR 0.3.1

Patch release for the CRAN submission. Package code and results are unchanged from 0.3.0.

• README links to the license and the roadmap now use full URLs. The relative links pointed at files that are not part of the built package, which CRAN's incoming checks report as invalid file URIs.

Everything new in v0.3 is described in the 0.3.0 release notes: selectable score intervals for I-CVI and Psa, panel-level agreement for expert panels, and evidence-based factor retention.

Install

install.packages(
  "contentvalidR",
  repos = c("https://juhalt.r-universe.dev", "https://cloud.r-project.org")
)

Full changelog: https://juhalt.github.io/contentvalidR/news/index.html

contentvalidR 0.3.0

Third public release, and the first prepared for CRAN.

v0.3.0 is built on evidence. Every addition rests on published, verifiable methodology, and one design rule runs through the release: where more than one published method exists, you choose through an argument whose default is the best-supported option. Methods with published evidence against them stay available so earlier work can be reproduced, but they are never the default, and choosing one prints the critique.

The package still declares Imports: stats only, so it installs without a compiler toolchain.

Install

install.packages(
  "contentvalidR",
  repos = c("https://juhalt.r-universe.dev", "https://cloud.r-project.org")
)

Two defaults change results

• qfactor_content() now chooses the number of factors by parallel analysis. Use retention = "kaiser" to reproduce earlier results.
• expert_validity() relevance mode now bootstraps an interval for panel agreement. Set seed when printed output must be reproducible.

Other existing results are unchanged. The new interval and agreement columns are additions.

Intervals for I-CVI and Psa

I-CVI and Psa are proportions from small panels, and until now they appeared as bare numbers. cvi() and compute_psa() now report an interval for each, and expert_validity() and sort_validity() carry it through. The Wilson score interval is the default, following Newcombe (1998); Agresti-Coull and Clopper-Pearson exact intervals are available when a study needs to match earlier work.

Output names the method and level, and says what an interval cannot tell you: it reflects how few ratings an item received, not whether the right judges were chosen. With eight experts, an item every expert rated relevant still has a lower limit well below 1.

Panel-level agreement

panel_agreement() reports one coefficient for how consistently a panel rated the whole item set, and expert_validity() relevance mode now includes it. Modified kappa still describes each item; this is the panel-level view it never gave.

• Krippendorff's alpha is the default, at a measurement level you choose: ordinal (default), nominal, or interval. It handles any number of experts and missing ratings, which Zapf et al. (2016) recommend it for.
• Gwet's AC1 is available but never the default. Vach and Gerke (2023) show it rises as ratings concentrate in one category even when agreement does not change, and choosing it prints that critique.
• The interval resamples whole items with all of their ratings, following Zapf et al. (2016), who found Krippendorff's original bootstrap reached only about 60% coverage.

Alpha can be low on a panel that agrees closely. When nearly every rating is the same value, chance predicts very little disagreement, so a few dissenting ratings pull alpha down. The output reports the share of identical rating pairs next to alpha and says plainly that a low alpha with many identical pairs is not by itself evidence of a poor panel. In the README's three-item example, alpha is near zero while 71% of rating pairs are identical.

Factor retention for Q-factor content adequacy

qfactor_content(), which implements the content-adequacy approach of Schriesheim et al. (1993, 1999), used Kaiser's eigenvalue-greater-than-1 rule to pick the number of factors. Zwick and Velicer (1986) found that rule severely overestimates the number of components. The default is now Horn's (1965) parallel analysis, simulated with the same size and missing cells as your ratings and reproducible with seed. signal_detection() now cites the Anderson and Gerbing (1991) validation design it follows.

Verification

• Krippendorff's alpha reproduces the worked example in Krippendorff (2011) at all three measurement levels (.743, .815, .849) and matches irr::kripp.alpha() to 1e-10.
• AC1 reproduces the Wongpakaran et al. (2013) table, and its handling of missing ratings matches Gwet's own implementation.
• Wilson and exact intervals match base R's prop.test() and binom.test().
• Parallel analysis recovers a simulated two-factor structure; on pure noise, the Kaiser rule retains on average more than three factors beyond it.
• R CMD check --as-cran is clean across Windows, macOS, and Linux on R release, oldrel and devel, including a job that treats any NOTE as a failure. A clean-library install of the built tarball runs the examples for every help topic, all seven vignettes, and the citation.

Notes

CRAN submission follows this release and is tracked in #14. Evidence-reviewed methods that did not meet the bar for v0.3.0, including Bayesian extensions, COSMIN appraisal, Delphi consensus, and the 95th-percentile parallel analysis criterion, are recorded with their reasons in the Future research milestone.

Full changelog: https://juhalt.github.io/contentvalidR/news/index.html